Jul 01 2010

Eclipse 3.6 Helios: WTP + Ivy and Deployment Assembly… Working reliably at last?

Category: Eclipse,Software DevelopmentPhil @ 12:01 am
I have a love/hate relationship with the IvyDE plug-in when it comes to WTP project types. I have literally wasted hours of time trying to get IvyDE to publish into my web application. It is very strange, no pattern at all; it works fine for days at a time and then decides to stop working… I don’t believe that the team has actually developed a “reproducible pattern” of forcing WTP to publish the Ivy dependencies; it is a continual “trial and error” of the following steps, clean the project, Ivy resolve, clean Tomcat, un-deploy the application, redeploy the application; one of these will eventually cause WTP to push the dependencies over. Can you say, frustrating?

Some developers just give up and use Ivy retrieve to dump the dependencies into the WEB-INF/lib directory; this will work, but is not a very elegant solution. To make this work, you are required to refresh the workspace after the Ivy resolve/retrieve completes, which pushes the dependencies into the web application; nasty.

This problem may have been answered in Eclipse 3.6 (Helios). I tested some of the 3.6 milestone releases, but was fooled into thinking that the IvyDE and WTP integration was broken; the J2EE module dependency option was no longer visible on the project properties. I was too busy to investigate (or look at other menu options!), I just assumed that it was just broken. To make a long story short, there is a new menu option called Deployment Assembly. The new option is a much simpler concept to comprehend; it is more flexible and actually seems to work consistently with Ivy. The most obvious change is the presentation; it gives a better picture of how the web application will be assembled. The old J2EE Module Dependency manager allowed you to choose components, but might have been more tailored for building an EAR file, rather than a WAR. Another issues is the inability to exclude any of the “source” folders. If you manage your unit test classes in a separate source folder, they would also be included in the application (WAR). Not necessarily a problem, but not very clean and could possibly hide unintended dependencies between the two source trees.

When using IvyDE, the first step is to add an Ivy classpath container. This will add Ivy dependencies to the classpath of your Eclipse project. If you have a WTP project, you will also need these dependencies copied to the WEB-INF/lib directory. This is easily accomplished by clicking the Add button under Deployment Assembly (under project properties). You can add other projects and external jars as well. Here is where it gets interesting, there appears to be two different options. If you have already added the Ivy classpath container to your build path, click the “Referenced Classpath Entries” option and select the ivy.xml file from your project. There might be a little display problem, as the “Deploy Path” for the classpath container does not appear correct, but it does put the libraries in the correct directory location.

The second option is to remove the Ivy classpath container from the build path, and re-add it under the new Deployment Assembly properties. This option displays properly and copies the dependencies to the WEB-INF/lib directory. I actually prefer this option, but there is a significant problem; there is no way to re-resolve the dependencies; it appears you need to delete the container and re-add it, if you change your ivy.xml file. Rather annoying!

I started writing this blog a couple of weeks ago and was under the impression that the new version was working perfectly. However, I’m currently back to where I started, with inconsistent, unexplainable behavior. Time to submit a question to the Ivy support list. Hopefully, I will have something to report back soon.

https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png


Jun 07 2010

What motivates you?

Category: Software DevelopmentPhil @ 12:01 am

Kent Beck tweeted this a couple of weeks ago, I thought it was interesting from multiple perspectives… First, the overall “presentation” concept was super cool and very creative. Secondly, the content was excellent and thought provoking. I think many of the points seem to get lost in corporate America. The video is 10 minutes long, but well worth the time; grab a coffee and enjoy! If you liked this video, there are actually numerous other topics from RSA Animate on YouTube.

RSA Animate – Drive: The surprising truth about what motivates us

https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png


Jun 06 2010

Ivy Dependency Management – Lessons Learned and Ant 1.8 Mapped Resources

Category: Continuous Integration,Software DevelopmentPhil @ 1:59 pm

I have been using Ivy for a couple of years now. Strangely, of all of the developers that I have introduced Ivy too, I think I’m the only one really jazzed by this approach. Some actually question the value of the tool; change is a hard thing! I think it is hard to see the real value of dependency management, especially when you only work on a single application, for an extended period of time. After all, once an application is wired up with the right libraries, what is the big deal? I tend to work on multiple projects and take a more board view of development issues. I seem to be the guy creates the Ant scripts,  upgrades the libraries, and manage the dependencies between all of these tools; hence I’m a huge fan of dependency management! I see no better way to document the external dependency requirements of an application. From a corporate or even organization perspective, Ivy can provide additional value beyond development, it can support both compliance and configuration management activities. The value of a single, centrally managed, browsable repository of reusable components should not be overlooked. I believe that significant number of person-hours are lost each year on the simple task of component (open source and commercial) version management. Just think about the amount of time spent by a project to download new component versions (and their dependencies), update build scripts, manage them in a version control system, creating tags and branches for items that never change! Now, multiple that effort by the number of applications your company manages. I have to imagine this amount of time is more than insignificant. OK, my Ivy sermon is over; on to something that might actually be valuable.

Once you have your repository structured, Ivy works pretty well. Converting an Ant-based application to use Ivy is pretty trivial and always seems to work perfectly. On the other hand, the IvyDE plug-in has always been a little bit of pain, especially when you are working with a WTP project. After converting the first project to use Ivy, it was basically a cut and paste exercise for every other project. Unfortunately, there were a few side effects of using Ivy that I just finally got around to addressing. Not that these were huge issues, more a matter of cleanliness.

Use multiple Ivy configuration files rather than one single file.

We always use a single Ivy configuration file per project; it contains the dependencies for all activities: compiling, execution, jUnit, Clover, FindBugs, PMD, Checkstyle, etc. This made the Ivy configuration a little difficult to comprehend, as it was not 100% obvious which libraries were actually required to simply build the application. Additionally, this made the Ant build files and file system rather messy; we ended up with hundreds of jar files in the Ivy retrieve directory, not knowing if a tool or component was dependent on them.  If the project happened to create a WAR file, we had to “hand pick” the files from the retrieve directory and provide a custom fileset of required files for the WEB-INF/lib directory. This was a continual maintenance issue, as someone would add a new dependency to Ivy and forget to update the custom path object in the Ant XML. To simplify and eliminate this problem, we decided to create two Ivy configuration files, one for the required components (ivy.xml), and one for the continuous integration / build environment (ivy.tools.xml).  It was now much easier to visualize and manage the run-time components required to execute the application, independently from components needed to build and/or test the application.

Use Ivy resolve Ant task rather than Ivy retrieve task, or better yet, use the Ivy cachepath task.

The simplest Ivy strategy is to dump all the components into a single directory using the Ivy:retrieve Ant task. This approach gives direct access to the files and enables the creation of multiple filesets for specific uses. This is the easiest way to convert an existing project to Ivy, just re-point the build scripts to the new location. This approach aggravated some developers on the team, as we could end up with the same jar file in at least 3 different sub-directories on your machine; a copy in your Ivy cache, another copy in the Ivy retrieve directory, and another copy in the required location (WEB-INF/lib for example). The IvyDE plug-in uses files directly from the Ivy cache, no extra copies of the files are created. I should have picked up on this approach sooner, as it seems much more elegant. There are several post resolve tasks that can be used to implement this approach, namely the Ivy:cachepath task. This task creates Ant path objects with direct references to the file’s location within the Ivy cache. If you need a fileset object, you can utilize the Ivy:cachefileset task.

As you can see from the following example, it is easy to create multiple path objects from within your Ant build process. It is even possible to have extremely fine grain control, you can create path objects for individual modules, one for each specific tool if you so desired; I  have not used this strategy yet, but it does seem interesting.

<ivy:settings file="ivysettings.xml" />
<ivy:cachepath pathid="classpath.CORE" conf="runtime" resolveId="CORE" file="ivy.xml" />
<ivy:cachepath pathid="classpath.CI" conf="runtime" resolveId="CI" file="ivy.tools.xml" />
<ivy:cachepath pathid="classpath.COMMON"organisation="beilers.com" module="common" revision="1.0" inline="true" conf="debug" />

Once you have your path objects created, you can use them just like normal in other tasks, nothing special here. It is that point that what makes Ivy so attractive from a build (Ant) perspective, in that Ivy does not require you to change the way you think or typically use Ant.

<javac source="1.5" destdir="${build.dir}" debug="${javac.debug.option}" verbose="no">
     <compilerarg value="-Xlint:unchecked" />
     <src path="src" />
     <classpath>
           <path refid="classpath.COMMON" />
           <path refid="classpath.CORE" />
           <path refid="classpath.CI" />
      </classpath>
</javac>

I did run into one issue with the Ant <war> task using the Ivy:cachepath task. When the WAR file was created, the WEB-INF/lib directory did not get flattened. All of the dependent jar files were placed in sub-directories that mirrored the Ivy repository structure. Unfortunately, the <war> task <lib> section does not support the “flatten” option. (An Ant bug was actually filed in 2004 for this problem.) Luckily, the post gave me the appropriate Ant way of resolving the situation, using a new feature in Ant 1.8 called mapped resources. The XML seems a little convoluted looking, but works perfectly. I don’t think I would have ever figured this out without Google!

By combining all of these ideas into the Ant build build system, I can take advantage of the Ivy cache and eliminate all extraneous copies of jar files. The real value is provided by the multiple Ivy configuration files, making it completely unnecessary to know what jar files are included in each dependency. This approach makes the Ant build system significantly less error prone and eliminates all future maintenance concerning  jar files.

<war destfile="${war.full.path}" webxml="WebContent/WEB-INF/web.xml" manifest="${manifest.path}">
    <fileset dir="WebContent">
     </fileset>
    <classes dir="${build.dir}"/>

    <mappedresources>
      <restrict>
        <path refid="classpath.CORE"/>
        <type type="file"/>
      </restrict>
      <chainedmapper>
        <flattenmapper/>
        <globmapper from="*" to="WEB-INF/lib/*"/>
      </chainedmapper>
    </mappedresources>

    <zipfileset dir="src" prefix="WEB-INF/classes">
         <include name="**/resources/**/*.properties" />
         <include name="**/resources/**/*.xml" />
    </zipfileset>
</war>
https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png


Apr 25 2010

New Agile…

Category: Software Development,TestingPhil @ 5:50 pm

I might be dating myself, but when I was a kid, I remember my parents talking about New Math. I don’t believe that I was ever exposed to New Math, it might not have made it to the small towns in northwestern Ohio! The New Math Wiki was actually very interesting; taking us back to the cold war with Russia, attempting to drive up the American scientific and mathematical skills. For better or worse, New Math was not able to change the status quot and was abandoned.

Not really the point of this blog, but New Math actually stressed the mathematical concept of set theory. This is actually a concept that one of my mentors actually spoke about quite often. Something that I suggest everyone do a little reading on, especially if you are not familiar with the concept.

So, why New Agile?  A friend happened to snap a picture from a white board at his place of employment, illustrating their new 14 week Agile process. Oh, where to begin…. I going to skip the obvious 14 week concept and tie this back to one of my previous posts on testing. I asked the question: Should testing be the limiting factor in the software development process? I am not exactly sure what numbers were used per cycle, as it seemed like it was a discussion point, but it does highlight an interesting trend.
I threw together this little a table to illustrate my point. Based on either combination of values, the project was going to allocate between 57% and 64% of it’s project schedule to testing. Using the worst case numbers, we are talking about 3 times longer to test the code, than actually develop the code. Is it just me or is that crazy? What if you had two (2) months worth of development? Simple math, that would require six (6) additional months to validate!  I realize there are or can be several factors contributing to this equation, but I’m pretty sure that we are not talking about a team of twelve (12) developers to one (1) tester, creating an unbalanced capacity situation.  Just for fun, I threw a Theory column into my table. I can’t remember where I learned or read this, but I’m sure everyone knows this fact as well… the cost of fixing bugs. We should be spending the vast majority of our software development resources on upfront planning; this would include requirements, analysis and design. It would seem that if developers knew exactly what there building and testers knew exactly what they were testing, both teams could do a much higher quality job, probably in less time!

Unfortunately, we seem to be spending too much of our time in the wrong SDLC phase and on the wrong artifacts. I can’t tell you how many times I have seen a hundred (100) page requirements document for a seemingly trivial system. The SDLC mandates that these documents be produced, irregardless of the content’s quality. This is no fault of the business, they simply try to document their needs, given what they know. Because requirements are typically not allowed to evolve and are seldom interpreted correctly by the IT team, the business never gets what they really need. Additionally, the time spent by the testing teams to provide the traceability back to the excessively verbose requirements, while generating and documenting their test cases could almost be considered SDLC overhead. The testers are rarely given a chance to actually automate their test cases, which in my opinion, is the most valuable artifact of their job.

If we could produce smaller, more concise, valuable requirement documents, spend more time doing actual analysis and design work, we could actually take steps toward my Theory column. This approach could and should enable the later phases of the SDLC to be completed with less effort, fewer issues, and in less time. So two questions:

  1. Is this not common sense?
  2. Why is this approached not embraced by technology organizations?  The benefits seem unquestionable…
https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png


Apr 20 2010

Making up your own methodology… WetAgile or W-agile

I was reading the Software Development Times last week and found an interesting article about development methodologies. I almost think Agile has turned into a “techno-fad”; a meaningless buzzword, right up there next to architect! Nobody wants to admit they are using the waterfall approach; they would have be considered an technical dinosaur; just look at the history highlights below!  Unfortunately, many projects seem to choose Agile for the wrong reason; there seems to be some kind of coolness, prestige, or promise of advancement associated with using Agile, rather than a true embracement of the methodology.

Can you believe…

  • The waterfall model was first documented in 1970 by Winston Royce.
  • The spiral model was defined by Barry Boehm in 1986 article “A Spiral Model of Software Development and Enhancement”.
  • The Extreme Programming model was introduced by Kent Beck in 1999. I recently suggested following Kent here
  • Interesting fact, Ken Schwaber, one of the SCRUM creators stated that 75% of organizations using SCRUM do not get the benefits they had hoped for using this approach.

So, everyone claims to be “doing Agile”; I think reality tells a slightly different story. I recently observed a group define the allowable methodology options for new projects; they chose Waterfall, Iterative, and Agile. Waterfall and Agile are pretty to definable, but Iterative seemed a little vague to me. After reading this article, I think I actually figured out what it was, WetAgile. Per the author, WetAgile is a term used to describer Waterfall shops that either aspire to be Agile or are in the transition process.
I actually found two versions of the document by the author, Steve Pieczko. The SD Time version seemed to be at a little higher level, while the Enterprise Management Quarterly version has a little more substance.

The real question is why do teams only implement so few aspects of Agile? The author suggest that we typically implement the pieces of Agile that we most understand or believe the Waterfall teams can easily embrace. It seems like the XP techniques are much easier for the development team to implement, specifically because they control their own space; whereas embracing the complete Agile (SCRUM) management process is a little more involved. I believe the problem is that too many people have to be involved to make it work. Throwing a bunch of people in a room and having a daily stand-up meeting does not exactly imply you are doing “Agile”. I find it disappointing that the most valuable part of SCRUM (my opinion), the creation and management of the backlog, is rarely  done. Probably because project management is still very “Waterfall” centric; you start with some requirements and end with a date, nice and simple. Additionally, it is not very easy to build a backlog when requirements, including defects and enhancements, are managed by processes that have nothing to do with the software development life cycle.

Here is the root of the problem, most projects (applications) are managed as series of individual releases, where each release has a concise beginning and end. Each release is a discrete and isolated project. I don’t think applications are ever viewed as perpetual, ongoing, evolving entities.  The subtle difference is that enhancements and defects are usually managed independently and are rarely viewed and prioritized together. The backlog, if it exists is virtual. The business seems to “divine” their most relevant needs out of thing air, ask for some LOEs, and then define the next release based on what can be implemented by the some magical date. That is a far cry of creating, evolving, and managing a complete list of features that ultimately should be built in a application, using a iterative release stream.

I think another significant problem is the perceived cost of creating a release. How great would it be to deliver new functionality to your customer every month? Unfortunately, with today’s audit and traceability requirements, the actual cost of creating a release can outweigh the value. Too many meetings, too many sign-offs, just too much overhead.; Not to bash the testing community, but it seems that the effort required to test and certify a release is also cost prohibitive. There is no possible way that testing teams could manage monthly release cycles using their traditional approaches. Should testing actually be the limiting factor in the software development process? What is really driving up the cost? What is burning up all of the time? Something I will have to research! I would hope that testing could be accomplished just like development, as developers attempt to do test driven development, testers should be following the exact same process. The continuous integration process should be running their suite, same as the developers suite. Would this not be sweet? The next question is, do we really need “User Acceptance Testing”? If the users are truly involved with the SDLC process and all testing is automated and quantifiable, is that not good enough? We seem to add multiple levels of testing, choosing an “accumulative testing strategy”, rather than a single, cohesive plan.

I found an interesting post on Agile Coaching, called “Agile or W-agile“. I think the blog is kind of unique, a series of drawings done on note cards and white boards… actually a pretty good way to get out a message!

Extra Credit Reading…
This article got me thinking: what is the next advancement in software development methodologies? It seems like eXtreme Programming and Agile have been around for many, many years; even though, maybe not widely adopted. I did a quick Google and found an interesting post titled: After Agile, What’s next? There are some interesting questions and several good links.
https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png


Apr 01 2010

Who should write unit tests?

Category: Software Development,TestingPhil @ 12:01 am

I have been a big fan of Kent Beck for many years, mostly from all of his work on jUnit and extreme programming.  I have two recommendations for you:

  1. Follow him on Twitter. I cannot tell you how many thought provoking tweets he has provided me, I’m sure that I get at least one good nugget each day!
  2. Read his personal blog at Three Rivers Institute.  It is a really good source of design and development information. Typically, not a quick read, but well worth the investment.

Earlier this month, Kent tweeted about the “Pragmatic Magazine”. I had been to the “Pragmatic Bookshelf” many times in the past, but did not know about their magazine. This turned out to be complete and valuable educational exercise for me.

Tangent…
You can download the “magazine” in multiple formats: PDF, ePub,  or mobi. Since I do all of my reading on my cell phone (Motorola Droid), I downloaded the Aldiko ebook reader. Within a couple of minutes, I was able to open up the magazine on my phone, too cool! My poor Kindle is so neglected; I should try to download the mobi format and see how the reading experience compares. I’m sure it will be much nicer on the Kindle DX, but I always have my phone with me, so the Droid wins!

I was really surprised by the featured articles. I gravitated the to the JSON article; I’m a huge JSON fan, but see it totally ignored in my professional world… I have not figured why this is true, but the article did provide an interesting perspective. One of the other articles that grabbed my attention was on Testing. You have to read it, it is very interesting…

Quoted from the article…
The developers who understand the importance of unit tests and who write comprehensive and subtle tests are exactly the group that need them the least. The “average” or inexperienced developer probably doesn’t  believe in unit tests and will write bad or pointless tests if forced to do so.

So, the question is: Who should write your unit tests? This answer to this question really emphasizes something that I always try to stress to project teams: Do not test your implementation,  test the behavior, the contract, you need to support. So many times, especially if testing is not second nature to the developer and the team uses a code coverage tool, there is a very bad tendency to write tests which are specific to the implementation. To make matter’s worse, some teams do in-package testing of their Java code to exercise and validate the implementation details. I was so fired up on this point last year, so I wrote a blog on it! Needless to say, I think this is a really bad idea and creates unnecessary and undesirable coupling between your unit test and implementation.

The author goes on to suggest that your best developers should only create the interfaces and associated unit tests, let the other developers create the implementation behind the interface. Pretty cool idea. Unfortunately,  I don’t see this taking off in the corporate world. In my experience, most developers tend to write their own unit tests or unit testing is completely outsourced to some third party (way after the fact). Additionally, the “first tier” developers are expected to write the hard code, and that is what they really want to do anyway; the “second tier” developers usually get the leftovers, like unit test cleanup and shell scripts.

I think the whole value system is skewed in the typical corporate software development process. Where should you put the good people to get the most value?  In the past, if you did not cut it as a developer, you would get promoted to the CM or testing team.  Many times, these teams were not valued by the organization, and hence not staffed with top talent. I would prefer the best and brightest developers in the testing space. No matter how good the code is, if the testers are sub-par, then the application is destined to fail. Conversely, if you have marginal developers and a strong test team, I think you have at least a chance of success (providing you have a good design).

I read another interesting article about bug tracking and system testing.  Think about this… rather than creating a problem ticket for issues found in system test, why not create some type of unit / integration test to demonstrate the problem?  The test get checked in by the test team and breaks the continuous integration build process. Now, the development staff just has to fix the problem. Seems kind of streamlined and efficient; I guess it will never work! But I do like the concept!

https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png


Mar 31 2010

Parkinson's Law and Bikeshed Bureaucracy

Category: Software DevelopmentPhil @ 8:45 pm

I was doing some research on Subversion the other day and found an interesting item on the FAQ about a bike shed. The gist of the article is that it is easier to get approval to build an atomic power plant than build a bike shed. It is a pretty good read about bureaucracy.

So, if you like Dilbert, you will relate to Parkinson’s Laws. Amazingly, these laws were written almost 40 years ago… and seem to be in full force today!

Enjoy!

https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png


Mar 30 2010

Unit Testing Not Required…

I assume that everyone performs some kind of unit testing… The real question is what does unit testing mean to you?  For me, it is some form of repeatable, assertion-based approach that can be re-executed by other developers and a continuous integration process. Unit testing is just one piece in my software confidence puzzle. That sounds like a great topic for my next blog! Unfortunately, some developers think that generating some log messages to inspect or stepping through the code with a debugger accomplishes the very same thing. Depending on the competency of the developer, I believe this can be partially true; however what about the next developer who needs to change the code?

Next, you need to place some kind of value on your unit test strategy. Is testing an investment, an expense, or a liability?  I think that unit testing typically starts as some type of organizational mandate. It quickly turns into an expense and ultimately a liability. I have seen very few projects truly embrace and believe in the value of unit testing. It seems that more often than not, a true believer (or zealot) establishes the process and everything goes along smoothly. After a while, the “pressure of the release date” eliminates the need for creating new unit tests; or even worse, the existing unit test are no longer maintained. They become brittle, broken and ignored. As the number of failed unit test grows, is seems very hard to justify the cost of fixing them. At this point, you have a liability problem; the existing failures get in the way of identifying and managing new failures. Who will really notice the the one new failure when there are already fifty other failures? If ignoring unit test failures is allowed by the continuous integration process, the typical developer will have no incentive to fix them. We have now created an environment where the correctness of the unit test only matters at the point when the code is checked in. That statement might not even be true, as there is no guarantee or reason for the developer to ensure that their new unit test executed properly in the continuous integration environment. Chances are, that after some period of time,  that new test will decay and be added to the every growing, ignored list of broken unit tests.

Wow, I’m painting a pretty bleak picture of unit testing! To make things even more discouraging, I recently lost the debate on the merits of unit testing with one of my mentors. I think he is one of the smartest and most practical architect/developer/person that I have ever worked with, yet he saw little to no value in unit testing. To this day, I still cannot completely follow his logic; surely there are other, external factors that could minimize the value of unit testing, such as team competency, cohesiveness, maturity, and the general environment, but I don’t believe unit testing can (or should be) completely eliminated.

Sad, but true…
On a previous project, one of my teammates said that he would not create any unit tests, unless the project manager added additional line items to the project plan, specifically for unit testing his code. I was amazed… Why was this not considered a standard development procedure? It is pretty easy to see why one could argue that unit testing is doomed.

Fortunately, I have worked on a few projects where we considered unit testing an investment and an asset. These projects mandated zero unit test failures and even required some level of code coverage. The teams never considered or required the creation of a unit test to be called out on the project plan; it was simply part of their job. More than once, these unit test suites gave us the confidence to make the right change (re-factor), rather than the typical Band-Aid Software Methodology typically employed to address problems. Sounds like another good blog topic!

Final thoughts…
Here is another good read, Design to Test; old, but still valid today! There are so many good points in this article, they deserve to be reiterated again and again. If you are familiar with the Spring Framework, they should really resonate with you…

  • Test the interface, not the implementation
  • Composition over inheritance
  • Singleton avoidance

Here was one more blog, “Tips for Testing“, that reinforces the increased ROI of unit testing over time. You should at least skim his tips, he seems to have an interesting perspective…

https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png


Nov 22 2009

Self-Documenting Code… Interesting thoughts about comments…

Category: Java,Software DevelopmentPhil @ 2:30 pm

I recently found an interesting web site called Making Good Software. I really liked this guy’s site, he might just save me from writing down a lot of my own thoughts! Unlike my typical post, his are very short and easy to read. I highly recommend checking it out and popping through his other recent posts too. Nothing revolutionary, but just great reminders of how we should be thinking as we write new software. The following blog entries caught my attention:

The Comments are Evil statement really hit home with me… I have always said that code should basically be self-documenting. This statement is usually frowned upon some developers and managers; some people might think I’m just lazy. Farthest thing from the truth! By enforcing good coding practices with automation like Checkstyle, PMD, and FindBugs, combined with placing a high value on class and variable names, placing logic in the proper place, and keeping methods and classes small, most comments are redundant and typically provide little value. It was very interesting how the author associated comments with the DRY (Don’t Repeat Yourself) principal. Interesting thought, when your comments simply repeat what the code is already saying!

I also place a huge value on refactoring. I borrowed the image at the right to further highlight the point. There are many articles on the “Red Green Refactor” approach worth reading, helping to tie this all together. I guess one of my personal goals is always to create that elegant (simple and clean) solutions, which never happens the first time! Looping through R-G-R a few time helps achieve that elegance, and you end up with code that requires less comments.

Some of the comments in the Making Good Software blog are very interesting has well; many people throwing in their two cents, both pro and con on the value of comments. The author also states that comments cannot be completely eliminated. The code we write is never perfect and probably needs a little extra context to explain the hopefully infrequent imperfections.  I think the bottom line is to focus on quality code; eliminate the obvious, limited value comments, and comment the pieces that are truly intricate. If you are writing a lot of comments, then you better go back and look at your design, as something is probably not quite right.

https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png


Nov 16 2009

Code Presentation or Inspection?

Category: Software DevelopmentPhil @ 8:18 pm

Several months ago, I got back on the code review bandwagon and jotted down some thoughts. As I worked with other teams doing “code reviews”, the lack of structure (from my perspective) got me energized to go back down this road.  We have been playing with the Crucible Code Review Tool for a couple of weeks now, but I’m not completely sold on what I’ve seen; I feel like I’ve missed something, but still think this is the right direction to be heading.

I first want to share a couple of links to some vendor papers that I think are worth noting, Effective Code Reviews (Atlassian.com), and Best Practices (SmartBear.com). I found two points that I wanted to highlight from these documents, both of which are about “preparation”. The following graphics were pulled from referenced links.
This first one really drives me crazy, but  unfortunately, this appears to be the way that most teams perform their “code reviews”.  First, the presenter/reader sets up a phone line for all of the participants to dial into, even if they all sit together in the same area! Next, the presenter sets up a “net meeting” and walks the team through the code; adding comments or making small fixes along the way. Maybe I have too limited of an attention span, but I don’t really commit to this type of code review, nor do many others from my observations. Inevitably, someone comes by with a question, or you get an interesting email or instant message; something to draw you away from what is being presented. To make matters worse, you have probably never even seen the code before and maybe don’t even understand the design. How much can you really contribute, other than noticing the most obvious issues? The above graph shows the range of code reviews, from a formality perspective. Not that the process has to be this rigid, but I think it is important to discuss the stages and roles of an inspection:

  • An inspection follows a well-defined procedure that includes six stages: planning, overview, individual preparation, inspection meeting, rework,and follow-up. Certain participants have assigned roles: author, moderator, reader, and recorder.

Depending on the team, there is usually only one stage (meeting) and one role (reader). I personally feel it is much more valuable to explore the code and actually read the code in advance,  committed to providing feedback; I think the reviewer needs to physically participate, whatever that means: print off the code or browse the code on line, actually understanding the code, be engaged!

My second point is about preparation. I generally don’t think that most developers are very effective at “thinking on their feet”. It is really not about intelligence, but more about communication. Teams are typically very diverse,  both technically and it the way they consume (process) information. It can be very difficult to present code at a level which will be effective for everyone to understand and process. In my opinion, no preparation is equivalent to GIGO.

Previously, I thought it was more of the reviewers responsibility to make the process meaningful. I think the reviewers should (must) prepare before the meeting, just like they were studying for a test. Feedback must be returned before the “meeting”;  otherwise, there is no need to schedule the meeting. The purpose of the meeting is to discuss comments and determine the best solution for highlighted issues; this approach also helping to get everyone on the same page. The amount of preparation done by the presenter/reader can vastly impact the review as well.  If the reader simply “wings it”, it will be a generally unfocused and undirected presentation.
As illustrated by the graph to the right, the reader’s preparation has a dramatic effect on the code review process. The process of the author reviewing and re-thinking their implementation before submitting the code for review, provides another opportunity to discover problems before the inspection process actually starts, making the process even more efficient. How many times have you looked at your own code and said… “What was I thinking? This is not good at all…”  So, why know incorporate a “self-review” into the process as well?

I will wrap this up, so you don’t get too bored, my next blog will be on our Crucible evaluation. I did find a short blog that had a nice picture of the “Crucible Workflow“. This picture is where I’m striving to be, from a process perspective: collaboratively and iterative and ultimately actionable.

https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png


« Previous PageNext Page »