Jul 20 2010

Eclipse Project Configurations – Shared or Personal?

Category: Continuous Integration,Eclipse,Software DevelopmentPhil @ 12:01 am

To be honest, this was not a topic I was actually planning to write about. However, because I received a couple of interesting comments on a recent blog entry, Unfriendly Developer Practices, I thought I should clarify my position.

Question of the Day
Should your IDE (Eclipse) configuration files be checked in as part of your project?

One of the comments suggested that there should be should be no dependency between the build system and the IDE. Another person suggested the disconnect between the IDE and the build system creates a convenient place for dependency inconsistencies to develop unchecked.

I completely agree with both of those statements and have seen both problems manifested several times. However, there are two simple tools can make this a non-issue: Continuous Integration and Dependency Management.

  • Continuous Integration is obvious; if that lazy developer forgets to check in a dependency or update the build scripts, the build fails. Not perfect, but the problem is immediately detected and the relevant people are notified of the situation. The problem can be resolved within minutes.
  • The Silver Bullet for me, was the addition of the Ivy Dependency Management tool into our build process. Because the build system and IDE share the same dependency configuration, the project’s dependencies were now managed in single place. Using the IvyDE plug-in, Eclipse simply worked, with no additional configuration. Using the externalized dependencies and basic Ant targets, the project could create a robust, change resilient build system. To achieve this level of robustness, I took advantage of the Ivy post resolve tasks. It was not until I discovered how easily they could isolate the build scripts from the actual dependencies, did it all come together. The real beauty of this approach is that no files (dependencies) are directly referenced. The post resolve tasks create variables which contain all of the appropriate files; the build script can then treat these variables generically, without concern. Nice and clean!

It was an unstated, fundamental requirement to have no “direct” dependency on Eclipse; such that we could all revert back to the wonderful world of Emacs tomorrow, should the need arise. I don’t think which IDE a project chooses to use, is really that important. I am apparently an Eclipse snob; but all I really care about is having Emacs key-bindings! If a majority of the team works on the same platform, I do believe there is real value around this continuity; that just happens to be Eclipse for me!

I have several other reasons for checking in configuration files:

  • It quickly highlights wrong doing! If a developer checks in something specific to their environment, they will break everyone on the team. My goal is for complete project neutrality: check out on any machine, in any directory, and the project is guaranteed to build and deploy.  Additionally, check the project out in Eclipse and it should build with no issues, within minutes.
  • Not all developer’s actually care about tools. Some developers simply want their environment setup for them. They have no desire to figure out how Eclipse formats code when you save a file or how to configure and run quality checks after each build; implementing business solutions is their primary concern.
  • Enhanced team productivity. If everyone’s world (environment) is the same, it is so much easier to spin up a new developer or help a teammate with a problem. Would you really want each developer to go through the discovery process of setting up the project? In the big picture, isn’t this really just wasted time?
  • Helping to ensuring quality coding practices and standards. We also check in the Checkstyle, PMD, and Findbugs configuration and rule sets. Taking advantage of the sharable configuration files, both the Eclipse plug-ins and Ant tasks work from the same rule sets, ensuring complete consistency across the team, no matter where the rules are executed.

I appreciate all of the recent comments; thanks for taking the time to reply. They enable me think about and reconsider the decisions I have made, giving me yet another opportunity to learn from my mistakes!  As far as Eclipse configuration files are concerned, I strongly believe there is far more project value gained by including them, as compared to requiring each developer manage their own environment. One final note, I have no issue with developers wanting to manage their own world, more power to them!  I would hope that these efforts would be to make the overall, shared environment a little better; after all, everyone should be contributing to all aspects of the project! The real point is that everyone should not be required to configure a project, unless they really want or need to.


Jul 11 2010

Unfriendly Developer Pactices – Adding uncessary environmental complexities

A couple of years ago, I began focusing on construction and deployment complexities. I had just joined a new team and began looking for the commonalities between their architectures and environments. Needless to say, they were all different. The most disturbing issue was the amount of effort required for a new developer to start working with an application. If there was any project documentation, it was probably out of date; resulting in hours or even days of trial and error to actually build and deploy the system. In this post, I’m talking about is building and deploying J2EE applications. The following list represents the top developer unfriendly practices I have observed.

Developer Unfriendly Practices
  1. Without first running a setup task in Ant, the project does not build in Eclipse.
  2. The Eclipse .project and .classpath files are not checked into version control system with the source.
  3. The project’s Ant scripts are generic and non-deterministic; they have to be edited for your individual environment before building the project.
  4. The project requires specific libraries or packages to be installed on your machine before it can be built.
  5. Eclipse project not configured as an WTP project.
  6. Minimal jUnits and no continuous integration.
  7. J2EE container dependencies built into the project.

After experiencing these challenges, I began a quest of environmental simplification. I understand there are numerous reasons why projects end up with these unfriendly characteristics,  but leaving them unaddressed was just not in my character! I made it my personal mission to ensure that each project that I worked on, I would try to leave it in a little better shape than when I arrived. This is a never ending activity; I hope that those that come after me will have a similar philosophy and continue my quest. It is amazing how easy and fast an application can atrophy, eliminating all of the positive changes that had been previously applied. To keep things simple, I came up with following three project requirements. I try to weave some aspect of them into the development process and architecture of each project that I work on.

Project Principles
  1. Self-Containment, No External Configuration.
  2. Environmental Awareness
  3. Change Resilient

I hope that most of these principles seem like common sense and are nothing new. Much like many of the XP principals, they are not new or revolutionary, just good reminders of often overlooked practices. I will try to elaborate on each principal in a future post, hopefully, in the near future!


Jun 06 2010

Ivy Dependency Management – Lessons Learned and Ant 1.8 Mapped Resources

Category: Continuous Integration,Software DevelopmentPhil @ 1:59 pm

I have been using Ivy for a couple of years now. Strangely, of all of the developers that I have introduced Ivy too, I think I’m the only one really jazzed by this approach. Some actually question the value of the tool; change is a hard thing! I think it is hard to see the real value of dependency management, especially when you only work on a single application, for an extended period of time. After all, once an application is wired up with the right libraries, what is the big deal? I tend to work on multiple projects and take a more board view of development issues. I seem to be the guy creates the Ant scripts,  upgrades the libraries, and manage the dependencies between all of these tools; hence I’m a huge fan of dependency management! I see no better way to document the external dependency requirements of an application. From a corporate or even organization perspective, Ivy can provide additional value beyond development, it can support both compliance and configuration management activities. The value of a single, centrally managed, browsable repository of reusable components should not be overlooked. I believe that significant number of person-hours are lost each year on the simple task of component (open source and commercial) version management. Just think about the amount of time spent by a project to download new component versions (and their dependencies), update build scripts, manage them in a version control system, creating tags and branches for items that never change! Now, multiple that effort by the number of applications your company manages. I have to imagine this amount of time is more than insignificant. OK, my Ivy sermon is over; on to something that might actually be valuable.

Once you have your repository structured, Ivy works pretty well. Converting an Ant-based application to use Ivy is pretty trivial and always seems to work perfectly. On the other hand, the IvyDE plug-in has always been a little bit of pain, especially when you are working with a WTP project. After converting the first project to use Ivy, it was basically a cut and paste exercise for every other project. Unfortunately, there were a few side effects of using Ivy that I just finally got around to addressing. Not that these were huge issues, more a matter of cleanliness.

Use multiple Ivy configuration files rather than one single file.

We always use a single Ivy configuration file per project; it contains the dependencies for all activities: compiling, execution, jUnit, Clover, FindBugs, PMD, Checkstyle, etc. This made the Ivy configuration a little difficult to comprehend, as it was not 100% obvious which libraries were actually required to simply build the application. Additionally, this made the Ant build files and file system rather messy; we ended up with hundreds of jar files in the Ivy retrieve directory, not knowing if a tool or component was dependent on them.  If the project happened to create a WAR file, we had to “hand pick” the files from the retrieve directory and provide a custom fileset of required files for the WEB-INF/lib directory. This was a continual maintenance issue, as someone would add a new dependency to Ivy and forget to update the custom path object in the Ant XML. To simplify and eliminate this problem, we decided to create two Ivy configuration files, one for the required components (ivy.xml), and one for the continuous integration / build environment (ivy.tools.xml).  It was now much easier to visualize and manage the run-time components required to execute the application, independently from components needed to build and/or test the application.

Use Ivy resolve Ant task rather than Ivy retrieve task, or better yet, use the Ivy cachepath task.

The simplest Ivy strategy is to dump all the components into a single directory using the Ivy:retrieve Ant task. This approach gives direct access to the files and enables the creation of multiple filesets for specific uses. This is the easiest way to convert an existing project to Ivy, just re-point the build scripts to the new location. This approach aggravated some developers on the team, as we could end up with the same jar file in at least 3 different sub-directories on your machine; a copy in your Ivy cache, another copy in the Ivy retrieve directory, and another copy in the required location (WEB-INF/lib for example). The IvyDE plug-in uses files directly from the Ivy cache, no extra copies of the files are created. I should have picked up on this approach sooner, as it seems much more elegant. There are several post resolve tasks that can be used to implement this approach, namely the Ivy:cachepath task. This task creates Ant path objects with direct references to the file’s location within the Ivy cache. If you need a fileset object, you can utilize the Ivy:cachefileset task.

As you can see from the following example, it is easy to create multiple path objects from within your Ant build process. It is even possible to have extremely fine grain control, you can create path objects for individual modules, one for each specific tool if you so desired; I  have not used this strategy yet, but it does seem interesting.

<ivy:settings file="ivysettings.xml" />
<ivy:cachepath pathid="classpath.CORE" conf="runtime" resolveId="CORE" file="ivy.xml" />
<ivy:cachepath pathid="classpath.CI" conf="runtime" resolveId="CI" file="ivy.tools.xml" />
<ivy:cachepath pathid="classpath.COMMON"organisation="beilers.com" module="common" revision="1.0" inline="true" conf="debug" />

Once you have your path objects created, you can use them just like normal in other tasks, nothing special here. It is that point that what makes Ivy so attractive from a build (Ant) perspective, in that Ivy does not require you to change the way you think or typically use Ant.

<javac source="1.5" destdir="${build.dir}" debug="${javac.debug.option}" verbose="no">
     <compilerarg value="-Xlint:unchecked" />
     <src path="src" />
     <classpath>
           <path refid="classpath.COMMON" />
           <path refid="classpath.CORE" />
           <path refid="classpath.CI" />
      </classpath>
</javac>

I did run into one issue with the Ant <war> task using the Ivy:cachepath task. When the WAR file was created, the WEB-INF/lib directory did not get flattened. All of the dependent jar files were placed in sub-directories that mirrored the Ivy repository structure. Unfortunately, the <war> task <lib> section does not support the “flatten” option. (An Ant bug was actually filed in 2004 for this problem.) Luckily, the post gave me the appropriate Ant way of resolving the situation, using a new feature in Ant 1.8 called mapped resources. The XML seems a little convoluted looking, but works perfectly. I don’t think I would have ever figured this out without Google!

By combining all of these ideas into the Ant build build system, I can take advantage of the Ivy cache and eliminate all extraneous copies of jar files. The real value is provided by the multiple Ivy configuration files, making it completely unnecessary to know what jar files are included in each dependency. This approach makes the Ant build system significantly less error prone and eliminates all future maintenance concerning  jar files.

<war destfile="${war.full.path}" webxml="WebContent/WEB-INF/web.xml" manifest="${manifest.path}">
    <fileset dir="WebContent">
     </fileset>
    <classes dir="${build.dir}"/>

    <mappedresources>
      <restrict>
        <path refid="classpath.CORE"/>
        <type type="file"/>
      </restrict>
      <chainedmapper>
        <flattenmapper/>
        <globmapper from="*" to="WEB-INF/lib/*"/>
      </chainedmapper>
    </mappedresources>

    <zipfileset dir="src" prefix="WEB-INF/classes">
         <include name="**/resources/**/*.properties" />
         <include name="**/resources/**/*.xml" />
    </zipfileset>
</war>


Apr 20 2010

Making up your own methodology… WetAgaile or W-agile

I was reading the Software Development Times last week and found an interesting article about development methodologies. I almost think Agile has turned into a “techno-fad”; a meaningless buzzword, right up there next to architect! Nobody wants to admit they are using the waterfall approach; they would have be considered an technical dinosaur; just look at the history highlights below!  Unfortunately, many projects seem to choose Agile for the wrong reason; there seems to be some kind of coolness, prestige, or promise of advancement associated with using Agile, rather than a true embracement of the methodology.

Can you believe…
  • The waterfall model was first documented in 1970 by Winston Royce.
  • The spiral model was defined by Barry Boehm in 1986 article “A Spiral Model of Software Development and Enhancement”.
  • The Extreme Programming model was introduced by Kent Beck in 1999. I recently suggested following Kent here
  • Interesting fact, Ken Schwaber, one of the SCRUM creators stated that 75% of organizations using SCRUM do not get the benefits they had hoped for using this approach.

So, everyone claims to be “doing Agile”; I think reality tells a slightly different story. I recently observed a group define the allowable methodology options for new projects; they chose Waterfall, Iterative, and Agile. Waterfall and Agile are pretty to definable, but Iterative seemed a little vague to me. After reading this article, I think I actually figured out what it was, WetAgile. Per the author, WetAgile is a term used to describer Waterfall shops that either aspire to be Agile or are in the transition process.
I actually found two versions of the document by the author, Steve Piecko. The SD Time version seemed to be at a little higher level, while the Enterprise Management Quarterly version has a little more substance.

The real question is why do teams only implement so few aspects of Agile? The author suggest that we typically implement the pieces of Agile that we most understand or believe the Waterfall teams can easily embrace. It seems like the XP techniques are much easier for the development team to implement, specifically because they control their own space; whereas embracing the complete Agile (SCRUM) management process is a little more involved. I believe the problem is that too many people have to be involved to make it work. Throwing a bunch of people in a room and having a daily stand-up meeting does not exactly imply you are doing “Agile”. I find it disappointing that the most valuable part of SCRUM (my opinion), the creation and management of the backlog, is rarely  done. Probably because project management is still very “Waterfall” centric; you start with some requirements and end with a date, nice and simple. Additionally, it is not very easy to build a backlog when requirements, including defects and enhancements, are managed by processes that have nothing to do with the software development life cycle.

Here is the root of the problem, most projects (applications) are managed as series of individual releases, where each release has a concise beginning and end. Each release is a discrete and isolated project. I don’t think applications are ever viewed as perpetual, ongoing, evolving entities.  The subtle difference is that enhancements and defects are usually managed independently and are rarely viewed and prioritized together. The backlog, if it exists is virtual. The business seems to “divine” their most relevant needs out of thing air, ask for some LOEs, and then define the next release based on what can be implemented by the some magical date. That is a far cry of creating, evolving, and managing a complete list of features that ultimately should be built in a application, using a iterative release stream.

I think another significant problem is the perceived cost of creating a release. How great would it be to deliver new functionality to your customer every month? Unfortunately, with today’s audit and traceability requirements, the actual cost of creating a release can outweigh the value. Too many meetings, too many sign-offs, just too much overhead.; Not to bash the testing community, but it seems that the effort required to test and certify a release is also cost prohibitive. There is no possible way that testing teams could manage monthly release cycles using their traditional approaches. Should testing actually be the limiting factor in the software development process? What is really driving up the cost? What is burning up all of the time? Something I will have to research! I would hope that testing could be accomplished just like development, as developers attempt to do test driven development, testers should be following the exact same process. The continuous integration process should be running their suite, same as the developers suite. Would this not be sweet? The next question is, do we really need “User Acceptance Testing”? If the users are truly involved with the SDLC process and all testing is automated and quantifiable, is that not good enough? We seem to add multiple levels of testing, choosing an “accumulative testing strategy”, rather than a single, cohesive plan.

I found an interesting post on Agile Coaching, called “Agile or W-agile“. I think the blog is kind of unique, a series of drawings done on note cards and white boards… actually a pretty good way to get out a message!

Extra Credit Reading…
This article got me thinking: what is the next advancement in software development methodologies? It seems like eXtreme Programming and Agile have been around for many, many years; even though, maybe not widely adopted. I did a quick Google and found an interesting post titled: After Agile, What’s next? There are some interesting questions and several good links.


Mar 30 2010

Unit Testing Not Required…

I assume that everyone performs some kind of unit testing… The real question is what does unit testing mean to you?  For me, it is some form of repeatable, assertion-based approach that can be re-executed by other developers and a continuous integration process. Unit testing is just one piece in my software confidence puzzle. That sounds like a great topic for my next blog! Unfortunately, some developers think that generating some log messages to inspect or stepping through the code with a debugger accomplishes the very same thing. Depending on the competency of the developer, I believe this can be partially true; however what about the next developer who needs to change the code?

Next, you need to place some kind of value on your unit test strategy. Is testing an investment, an expense, or a liability?  I think that unit testing typically starts as some type of organizational mandate. It quickly turns into an expense and ultimately a liability. I have seen very few projects truly embrace and believe in the value of unit testing. It seems that more often than not, a true believer (or zealot) establishes the process and everything goes along smoothly. After a while, the “pressure of the release date” eliminates the need for creating new unit tests; or even worse, the existing unit test are no longer maintained. They become brittle, broken and ignored. As the number of failed unit test grows, is seems very hard to justify the cost of fixing them. At this point, you have a liability problem; the existing failures get in the way of identifying and managing new failures. Who will really notice the the one new failure when there are already fifty other failures? If ignoring unit test failures is allowed by the continuous integration process, the typical developer will have no incentive to fix them. We have now created an environment where the correctness of the unit test only matters at the point when the code is checked in. That statement might not even be true, as there is no guarantee or reason for the developer to ensure that their new unit test executed properly in the continuous integration environment. Chances are, that after some period of time,  that new test will decay and be added to the every growing, ignored list of broken unit tests.

Wow, I’m painting a pretty bleak picture of unit testing! To make things even more discouraging, I recently lost the debate on the merits of unit testing with one of my mentors. I think he is one of the smartest and most practical architect/developer/person that I have ever worked with, yet he saw little to no value in unit testing. To this day, I still cannot completely follow his logic; surely there are other, external factors that could minimize the value of unit testing, such as team competency, cohesiveness, maturity, and the general environment, but I don’t believe unit testing can (or should be) completely eliminated.

Sad, but true…
On a previous project, one of my teammates said that he would not create any unit tests, unless the project manager added additional line items to the project plan, specifically for unit testing his code. I was amazed… Why was this not considered a standard development procedure? It is pretty easy to see why one could argue that unit testing is doomed.

Fortunately, I have worked on a few projects where we considered unit testing an investment and an asset. These projects mandated zero unit test failures and even required some level of code coverage. The teams never considered or required the creation of a unit test to be called out on the project plan; it was simply part of their job. More than once, these unit test suites gave us the confidence to make the right change (re-factor), rather than the typical Band-Aid Software Methodology typically employed to address problems. Sounds like another good blog topic!

Final thoughts…
Here is another good read, Design to Test; old, but still valid today! There are so many good points in this article, they deserve to be reiterated again and again. If you are familiar with the Spring Framework, they should really resonate with you…
  • Test the interface, not the implementation
  • Composition over inheritance
  • Singleton avoidance

Here was one more blog, “Tips for Testing“, that reinforces the increased ROI of unit testing over time. You should at least skim his tips, he seems to have an interesting perspective…


Sep 09 2009

Hudson Task Scanner Plug-in

Category: Continuous Integration,Software DevelopmentPhil @ 8:21 pm

Ok, that is just about enough Hudson for this month, unless I get fired up about Selenium and try out the Hudson integration! I did find one more useful plug-in this week, the Task Scanner.  Because I tend to check in code early and often, my code is typically in an incomplete state; you will tend see a lot of TODO or FIXME comments in the code. Managing these task inside of Eclipse is easy enough, but I thought it might be interesting to also monitor them on Hudson’s dashboard.

The plug-in is very easy to configure and allows you to specify thresholds on the number of open tasks. Like most other plug-ins, it generates configurable trend graphs and provides a convenient task browsing screen.

Here was an interesting blog post that shows how you could use the plug-in to determine how messy your SVN merges will be… kind of cool..


Sep 08 2009

Continuous Deployment with Hudson…

Since I was looking through the Hudson plug-in list this week, I decided to try a few more of them. We currently have Hudson continuously deploying a simple web application to a Tomcat instance. Since both Hudson and Tomcat are running on the same server, it was pretty simple to have Tomcat point to the WAR file generated during the build process. Tomcat notices the new WAR file and automatically redeploys the application. Pretty simple and pretty handy; this is about a 100 times easier and faster than dealing with our standard Weblogic deployment process!

So, how do you deploy your application if Hudson is running on a different machine than your Tomcat instance? Fortunately, Hudson provides a plug-in to solve this problem, simply named the Deploy Plug-in. It supports a variety of containers, taking advantage of the Cargo Framework. Can’t get much easier than this, just point to your WAR file and fill in the container information. That’s it! I tried it out today and it worked perfectly. This was a much cleaner approach than adding the logic to the build process.

Having the current code set continuously deployed is a great benefit all by itself. This helps ensure that your application is built correctly and is deployable; How many times have you forgot to add that new JAR file to the WAR, only to discover the failure several days later? Continual deployment also makes the application constantly available to the analysis and test teams. This allows everyone to see the current state of development and bounce through the application to see how it will ultimately work. This early exposure also gives developers a change to resolve issues much earlier in the development cycle. We can achieve more value by adding the integration or system testing to the process as well… Just think how much we could reduce our development and testing cycles with this level of testing?

I have been starting to read about Selenium on the Hudson mailing list and found this interesting blog by Ben Rometsch on “Easy Automated Testing with Hudson and Selenium“. I really liked this picture from the article (hopefully it will spark your interest too!) and truly believe this is what we should be working towards… Maybe this can be one of my next educational adventures!!!


Sep 07 2009

Hudson Plug-in for Custom Documentation, integrating an Ivy Report

Category: Continuous Integration,Software DevelopmentPhil @ 7:14 am

One of the best things about the Hudson Continuous Integration platform, is the amount of activity. People are constantly fixing problems and adding new functionality. I’m sure that some would say this is a bad thing, but even with the frequent releases, it is amazingly stable.

I have been moving projects to the Ivy Dependency Manager for the last couple of years. We have established a multi-repository solution, that allows us to control access to the repositories without impeding the progress of the development teams.  Introducing multiple repositories and transitive dependencies takes a little getting used to, so I have been adding an ivy.report task to each of the project’s build scripts.  Here is an example of Ivy dependency report. I think it is quite handy on multiple levels, but the primary benefit is displaying the dependencies of each component. Side benefits include showing the different types licenses used by each component, kind of nice for corporate environments. The report also shows where the components are resolved from, such as the legacy or sandbox repositories (indicating that you might need to do some promotion or upgrade work before releasing the project).

My problem was, how to tie the Ivy dependency report into my continuous integration process. Building the report was easy, but how could I make it available on the main Hudson project page? Unfortunately, there was no plug-in to make this happen; I was actually thinking about writing my own plug-in this summer, but never quite got around to it (or blogging for that matter!) Fortunately, this appears to have been a common need and someone was nice enough to release a really nice plug-in this past July, called DocLinks. It allows you to add multiple links to items generated by the build process. The process could not be simpler.  The links will automatically show up on your main project page, but will not be active until the next build completes.

This also solved a problem that I have with the Testability Explorer plug-in. I have about a 50% success rate with this plug-in. I am able to generate both the XML and HTML files for all projects without issue, however, the Hudson Plug-in reports nothing for half of the projects; I have yet to figure out the pattern. Anyway, the DocLinks plug-in now lets me add a link to the Testability Explorer HTML report, so that everyone can always see the results, independent of the Hudson integration.

Suggestion!There is one area that the Hudson community could be a little more helpful. The Hudson tool is very good about showing you when updates are available for itself or installed plug-ins. However, there appears to be now good process for discovering new plug-ins. I even follow the Hudson mailing lists, and there are no notices about new or updated components; the only way I know about upgrades is to click the “configure” link every so often, when I actually remember! To find out about new plug-ins is even worse, I have to look at the Plug-in page, and notice if there is something new. There are so many plug-ins now, it is almost impossible to know if something has been added. Hopefully, they will add some kind of search for recently added or updated feature to their page; I’m sure that I’m not the only one that would appreciated it!


Jun 10 2009

Hudson Continuous Integration and Eclipse Update Sites

I was looking for a development build of the IvyDE plug-in for Eclipse tonight and I stumbled across the Apache Hudson server. You have to check this out. Pretty cool, Hudson running over 150 jobs!  Their build farm consist of both Solaris and Ubuntu servers. What I found most interesting was how they used Hudson for creating an Eclipse update site. I subscribe to the Hudson mailing list, summarized here, to follow what is going on with the project. People seem to use Hudson for a variety of different purposes, such as a cross platform testing tool and even as a deployment tool. I have only been able to get people interested in Hudson as a “Cruise Control” replacement…. but at least it is a good start!

This site also demonstrates how multiple projects can be linked together, triggering builds of dependent downstream projects.  I’m not exactly sure how these jobs were assembled, but at a high-level, Ivy and IvyDE each have its own build loop (job). Each job is responsible for creating build artifacts. After a successful build of either job, a third job is triggered which creates/updates and Eclipse update site.  Pretty slick. (As any FYI, the dev plug-in looks better, new presentation and several new new options)

Another small plug for Continuous Integration. We recently set up something kind of cool (trivial, but cool)! As we continue trying to convince the powers that be, to move away from Weblogic towards Tomcat (or even JBoss), we found a really nice little feature. Since Tomcat and JBoss can hot deploy a new version of the WAR file, just by noticing a difference on the file system, combined with the fact that we can spin up a Tomcat instance in about 2 minutes, we actually created an auto deployment mechanism.  Our Weblogic environment seems so cumbersome in comparison…. We could have built the WAR and deployed in in our build loop, but that would have minimally added another 25 minutes to the loop (let’s just say we have an interesting deployment process). With Tomcat, we did a one time deployment to the location of the WAR file inside the Hudson workspace. Now, with each series of check-ins, we build the WAR as normal and Tomcat instantly deploys it, in seconds! If we were a little more sophisticated, we would write some jUnits to bang against it, making sure we did not break anything or create any new dependencies.  Maybe I will throw that in during my next project!!! I know this integration was possible before (with Weblogic), but no way as simple!  Even better Continuous Integration… on my last project I had Ant spin up an embedded Jetty web server, ran my jUnits against the Clover instrumented code, and finally had Ant shut down Jetty…. Pretty slick, from a code coverage collection perspective…


Mar 07 2009

Setup Hudson for Continuous Integration

Category: Continuous IntegrationPhil @ 8:24 pm

I have been experimenting with Hudson for several weeks now and I like the tool much better than Cruise Control. Not to say it does not have its issues, it generally works very really.  I have mostly had issues getting reports to show up in the project. For example, it appears that on a new project, jUnit test results will not show up until after the first completely successfully loop; kind of strange. One issue that is really frustrating me, the Testability Explorer plug-in;  I just can’t seem to get the report integrated, all of the other plug-ins have worked as expected.

There is one big difference between Cruise Control and Hudson, which I have not exactly come to grips with yet: the handling of build errors. Hudson introduces the the concept of Job Stability. Traditionally, I relied on Ant to determine the success or failure of the build, but with Hudson, it is also possible to determine the stability or health. Most plug-ins allow you to provide custom ranges to indicate the job’s stability, based on the number of errors or warnings generated by the component. On most of my projects, we did not allow any PMD or Checkstyle errors, so job stability was never much of an issue. However, there are many projects that allow Checkstyle, PMD, and even jUnit errors; this is where the stability metric can be really helpful. This would also be useful when introducing a new tool into a project and it takes time to cleanup the issues.

I found a very well written article for setting up a Hudson server.  If you are really ambitious, the article walks you through setting up a Tomcat or JBoss installation.  I originally setup Hudson using Tomcat, but most of my experimental work, I have used the built-in Servlet Container; it is a little easier and makes the frequent updates as simple as a mouse click.


Next Page »