I might have said this in a previous post, but I believe that many developers use Agile as an excuse to avoid following institutionalized processes. They prefer to make up their own extremely lightweight methodology, doing only the activities they enjoy or believe they have time for. By adopting an Agile process, many claim they can eliminate the need for analysis, design, and even code reviews! Personally, I don’t believe this was the intention of the Agile methodology. However, given the substantial number of quality gates and sign-offs that organizations institute to ensure the elimination of mistakes, it is no wonder that Agile becomes the easiest method to subvert corporate mandates and deliver software. I completely understand the theory and goal of these mandates, but it does seem that by the time they are institutionalized, the overhead of the process is more complicated than the actual business to be solved! And before anyone corrects my association of Agile and Pair-programming, I do realize that Pair-programming is a tenant of Extreme Programming, not Agile. Unfortunately, many people seem to commingle the Extreme Programming principles with Agile; XP tends to generate negative connotations in some groups, while Agile is complete accepted.

Several months ago, I discovered Scott Hanselman. He happens to work for Microsoft, but don’t hold that against him. He records several podcasts each week or so; I subscribe to two of them and they are both excellent. Scott focuses on a variety of software development issues and technologies. Hanselminutes is more technology oriented, while This Developers Life covers a variety of personal issue, from motivation, drive, to the Egyptian Revolution. I highly recommend going back and listening to all of the old This Developer’s Life episodes; I can’t tell you how insightful and reflective they are. As an added bonus, you get to hear some rather interesting musical choices! The people that he interviews are so interesting and have some really funny commentary. You will hear stories from different types of developers, both famous and infamous. You will learn how they navigated through their careers, sometimes successfully and other times, not so well. All I can say, is they are definitely worth the time! I hope you enjoy them as much as I did.

I had been a loyal Firefox user for many years. Firefox provided a nice cross-platform experience between Windows and Linux, and was highly extensible. This extensibility was one of the early benefits of Firefox, the ability to add new behavior to the browser through “Add-ons”. Amazingly, I was never a big “Add-on” user, I used a few of them, but did not take advantage or even explore what they could do for me. I used the Delicious Add-on for my bookmarking needs, but recently moved to the Diigo Add-on. I also use Firefox Sync for browser synchronization, including my tabs, bookmarks, history, etc. For posting to my blog, I’m a huge fan of the ScribeFire Add-on.  If you happen to do web page development, you have to try the Firebug Add-on. That is basically the extent of my Add-on usage, I did not ask too much from my browser!

Before jumping ship, I upgraded all of my machines to the Firefox 4 Beta. There were numerous technical improvements, but I was primarily focused on pure usability and how the browser could help me be more efficient. Start-up time was one of my biggest Firefox complaints; the browser seemed to have a tendency to bog down over time. The new version seems to have gone through a pretty dramatic user interface overhaul and addressed multiple performance issues, including start-up.

I was pretty happy with the UI changes, preferring the new, but controversial new tab location. The tabs are now located over-top of the navigation tool-bar; there was apparently quite a bit of debate on this little change! I prefer having two control rows at the top of the browser window, one row for tabs and the another row for navigation, apps, and widgets. I have seen a lot of content about these “web apps”, but it seems a little like pure marketing to me! The Firefox implementation, App Tabs, appear to be little more than a space saving short-cut; however, I can see them providing value for highly used web sites.

I had installed Chrome a couple of years ago, but was not too excited by it; I saw no compelling reason to change browsers. Wikipedia has an interesting graph of web browser usage; I was really amazed to see how the Chrome market share has taken off in the last twelve months. Even on my own blog, Chrome accounts for almost 25% of the traffic. I installed the newest version of Chrome last week and was immediately hooked. Unfortunately, I have become a true Google convert. It started with the purchase of my Android phone and there was no looking back. I am not saying that the following activities can or cannot be done in Firefox, I’m simply saying that I like everything better in Chrome!

A simple, but extremely cool feature is the “New Tab” behavior. It obviously opens a new tab, but its contents are quite different than you would expect. It is basically divided into three sections, Apps, Most visited, and Recently closed. Under the Apps section, you will see the Web Store icon; does everyone need to have their own app store these days? Anyway, the Web Store is a very well done site, that makes searching and installing new behavior extremely easy, using either applications or extensions.

Applications seem like fancy bookmarks, but from my reading, they can be (are) a lot more sophisticated. I looked at the SlideRocket app, it was genuinely cool… however, you can also run the app in Firefox! The app concept seems analogous to a rich user interface experience, one that performs like a real desktop application, rather than a collection of old-fashioned HTML pages.

My favorite feature of Chrome has to be the Extensions. Extensions add additional behavior to the browser itself. You can see from the picture to the right, I have added quite a few of them! They integrate into the Navigation Bar and look very nice, consuming minimal space while providing significant functionality. They look similar to the icons found in cell phones; many of the extensions have little indicators that track the number of items you need to address. You can find extensions for all of the standard Google tools: Gmail, Reader, Calendar, and even eBay. The Calendar extension is extremely helpful; it will tell you how long until your next appointment and when you mouse over it, it shows you the event details. The WP Stats is another personal favorite; it tells me how many people have looked at my site! Clicking on some icons will navigate you into the corresponding website, much like a short cut. Other icons have specific behavior, such as showing you detailed web site access statistics or an enhanced view of your search history. My favorite blogging tool, ScribeFire is also available in Chrome, but the spell checker is not working! I like the placement and interaction of the Chrome extensions much better than the traditional Firefox “Add-on” view, which is typically at the bottom of the browser window; Chrome make the extensions feel more integrated with the browser and part of the actual user experience.

My final Chrome praise is the synchronization with my Google account. It is pretty cool to watch an extension get automatically installed on my Windows machine, simply by installing it on my Linux machine. No restart or refresh required, it just automatically shows up!  I did notice one small oddity, I still had to configure the extension on the Windows machine. This seems rather strange, maybe it is a bug…. I assumed that Chrome would save each of the extension’s settings and synchronize them too. Even with this little shortcoming, there is no going back to Firefox for me, I hope you give it a try too!

I have probably described many of the same experiences you have encountered with Ant-based projects; they can be even more exaggerated when you start managing multiple projects. One possible evolution path is to define a “standard” directory structure and develop a collection of “shared” Ant scripts that work with the standard directory structure. This allows each individual project’s build XML to be little more than a set of properties. This is exactly where we ended up, utilizing an Ivy-like externalized repository for build scriptlets and Ant’s URL import capability. Hold on, a standard directory structure with reusable build components? Almost sounds like some other tool…

Changing a project’s build tool has a rather large impact and needs be considered from multiple perspectives. I have approached the problem by breaking it down into five (5) different spaces. This might not be the optimal or preferred approach, but does show the high-level differences and addresses the major build challenges. Surely, my next project will be done completely differently, but you have to start learning somewhere!

Perspective	Changes
Dependencies	To end up with an application that compiles in Eclipse, this change and the next have to be done in parallel. The easiest approach is to create a POM and focus on the dependencies section. If you are already using Ivy to manage your dependencies, this will not be that big of a change. Using a public Maven repository like MVN Repository to locate dependencies is very simple and saves a ton of time. The easiest way to create a POM is to let Eclipse make it for you; you will need to install the Maven plug-in first.  Once you have your project loaded into Eclipse, simply “Enable Dependency Management” in your project’s properties. You can also add dependencies directly from the  plug-in; I’m a little old school and prefer to edit the POM manually, but either approach works fine.
Local Development	I never liked the required Maven directory structure; I must have subconsciously adopted the Eclipse project structure as my personal standard. Converting the project’s directory structure was trivial and soon realized it was not that big deal after all; I aways separate my source and test files anyway. The main difference is everything lives under the source root, including the WebContent directory. I’m not sure what the “preferred” process is for loading a Maven project into Eclipse, probably using the Maven Eclipse Plug-in, which will create the appropriate Eclipse property files. Since I was doing a conversion, I don’t think this plug-in would be too helpful. Because I’m fairly Eclipse savvy, I simply wired everything up myself. Having the Eclipse Maven Plug-in will enable your dependencies  to be automatically added to the build path. With the addition of the Eclipse “Deployment Assembly” options dialog, it is extremely easy manually configure your project. You need to have a Faceted Dynamic Web Module for the “Deployment Assembly” option to be visible, but that should be a fairly simple property change as well. At this point, you should have a fully functional, locally deployable web application.
The Build	Now we can ignore Eclipse, and focus on building our WAR file using Maven. Even with the minimalist POM shown to the left (plus your dependencies section), it is possible to compile the code and create a basic WAR file. Use mvn compile to ensure that your code compiles. Using the Maven package goal, the source code is compiled, the unit tests are compiled and executed, and the WAR file is built. All of this functionality, without writing one line of XML!  One of the more time consuming parts of an Ant-based build is integrating all of the “extras” typically associated with a project, making them available to the continuous integration server. The extras include: unit test execution, code coverage metrics, and quality verification tools such as Checkstyle, PMD, and FindBugs. This tools are all typically easy to setup, but every project implements them slightly different and never put the results into a standard place! The general process for adding new behavior (tools) to a build appears to be the same for most tools. You simply add the plug-in to the POM and configure it to fire the appropriate goals at the appropriate time in the Maven lifecycle. Ant does not have this lifecycle concept, but it seems like a very elegant way to add behavior into the build. From the following example, I added the Checkstyle tool to the POM. The <executions> section controls what and when the plug-in will be executed. In this example, the check goal will be executed during the compile phase of the build process. Simply executing the compile goal, will cause Checkstyle to be invoked as well. This seems like a very clean integration technique, one that does not cause refactoring ripples. The Cobertura integration is another very good example. I have been a fan of the Clover code coverage for many years. Since losing access to Clover, I needed to look for an open-source alternative,  and had never tried EMMA or Cobertura. They both seemed like capable tools, but I had more success integrating Cobertura with Jenkins. I’m highlighting this point, as doing code coverage with Ant and Clover can sometimes be a little tricky and usually messy. The Cobertura plug-in takes compete responsibility for instrumenting the code, executing the unit test, and generating the report; completely non-intrusive.
Continuous Integration	Most development teams also want their project monitored by a Continuous Integration (CI) process. Modern CI tools such as Hudson/Jenkins provide excellent dashboards for reporting a variety of quality metrics. As I previously stated, it is rather time consuming to develop and test the Ant XML required to generate and publish these metrics; combine that with configuring each CI server job to capture these metrics and you have added a fair amount of overhead. I knew there was support for Maven-based projects within Hudson/Jenkins, but never took the time to understand why it would be beneficial. The main benefit is right there in the description, had I bothered to read it! Configuring a Maven-based job is little more than clicking a few check boxes. No need configure them in Jenkins, using the information provided by Maven, it is basically automatic. This is one interesting aspect of the Hudson-Jenkins fork. Sonatype, the creator of the Nexus Maven Repository manager and the Eclipse Maven plug-in, have chosen the Hudson side of the battle. I wonder what this means for Maven support on the Jenkins side. Obviously, it will not go away, but that might end up being a Hudson advantage in the long run. I still believe that the Jenkins community will quickly out pace the Hudson community.
Deployment	I’m not going into much detail on this point, other than to say that I would like to see the deployment process completely separated from the build process, maybe even done by two different people or organizations. I have seen too many projects combine building and deploying into one system, creating artificial dependencies, time consuming, unreliable, and unmaintainable deployment processes. I have also experienced complete deployment overkill, causing simple deployments to take over thirty (30) minutes, with no real guarantee that it would be successful. Hopefully teams (developer and operations) will strive for a minimal deployment overhead/process, one that provides just enough security and control to enable continuous, rapid deployments.

I have barely touched upon the real power of Maven and know that some people are going to say my requirements are too simple to be valid. I’m sure there are projects far more complicated than mine, but I do feel that this project is highly representative of the 80-20 rule. If developer’s can have a little self control, conform, and do what is required verses what would be really cool, simple Maven POMs should satisfy a large number of corporate projects with very little overhead. That is the goal, correct?  I can even see using Maven for my toy projects; why do I want to waste time writing and maintaining a bunch of useless build scripts? I never realized that you could ease into Maven; I think that is the real point of the post! And, it is not actually a huge investment.

When I was Googling for this topic, I found many interesting articles. I truly believe the biggest reason preventing teams from evolving and adopting new approaches is simply fear, fear of change and the unknown. I still think Maven is a rather large “pill to swallow”, but have now experienced enough value to continue investing time into this technology. I took the typical developer approach and dove into Maven with little preparation or understanding… I would not suggest this approach, unless you to have lots of time to waste! If you are unfamiliar with Maven, I hope the following collection of articles will provide a quick overview of the basic concepts. I realize that some of them are a little old, but you should start with the Maven basics, and those really have not changed.

An Introduction to Maven2
Should you move to Maven2
Maven vs Ant: Stop the Battle
How to Migrate from Ant to Maven: Project Structure
Top Ten Reasons to Move to Maven 3
Maven over Ant + Ivy
Automated Deployment with Maven – Going the whole nine yards

I found this title to be a very interesting question… I’m not sure it was intended as an open-ended question, as I don’t think there is a right or wrong answer. It really depends on your perspective. Are you looking at it from a purely technical or logical (modeling/solution) perspective?   Even though I might be considered very technical, I prefer to stay out of the really low level stuff. Sure it is fun, to get down to the bare metal once in a while, but I get more enjoyment out of the thought process. Claiming to be a very Object Oriented person and stumbling on this question really bothered me… I came home and did some Googling, but really never found the answers that seemed to live up my expectations. Next, I sent an mail to one of my pals who just loves the nuts and bolts… In his typically witty way, he rattled off: int, float, boolean, double, long, and char. A completely valid answer! He then have me six other bullets which I have merged into the table below and tried to expand upon.

My thoughts might not be completely baked, but I need to keep on moving; so please let me know if you think I missed the point! Generally speaking, I have been very happy with Java as a implementation language. All languages have their warts, and Java is no different. The vast number of tools and frameworks allow Java applications to be effectively developed, tested, and deployed.  I also believe that Java is pretty forgiving, allowing developers of all levels to build solutions; from being barely object oriented to insanely obscure. Hopefully, we end up in the middle with simple elegance. I hope to have the opportunity to explore some of these Java flaws in a Ruby, so please check back and see what I have learned!

One of the comments suggested that there should be should be no dependency between the build system and the IDE. Another person suggested the disconnect between the IDE and the build system creates a convenient place for dependency inconsistencies to develop unchecked.

I completely agree with both of those statements and have seen both problems manifested several times. However, there are two simple tools can make this a non-issue: Continuous Integration and Dependency Management.

• Continuous Integration is obvious; if that lazy developer forgets to check in a dependency or update the build scripts, the build fails. Not perfect, but the problem is immediately detected and the relevant people are notified of the situation. The problem can be resolved within minutes.
• The Silver Bullet for me, was the addition of the Ivy Dependency Management tool into our build process. Because the build system and IDE share the same dependency configuration, the project’s dependencies were now managed in single place. Using the IvyDE plug-in, Eclipse simply worked, with no additional configuration. Using the externalized dependencies and basic Ant targets, the project could create a robust, change resilient build system. To achieve this level of robustness, I took advantage of the Ivy post resolve tasks. It was not until I discovered how easily they could isolate the build scripts from the actual dependencies, did it all come together. The real beauty of this approach is that no files (dependencies) are directly referenced. The post resolve tasks create variables which contain all of the appropriate files; the build script can then treat these variables generically, without concern. Nice and clean!

It was an unstated, fundamental requirement to have no “direct” dependency on Eclipse; such that we could all revert back to the wonderful world of Emacs tomorrow, should the need arise. I don’t think which IDE a project chooses to use, is really that important. I am apparently an Eclipse snob; but all I really care about is having Emacs key-bindings! If a majority of the team works on the same platform, I do believe there is real value around this continuity; that just happens to be Eclipse for me!

I have several other reasons for checking in configuration files:

• It quickly highlights wrong doing! If a developer checks in something specific to their environment, they will break everyone on the team. My goal is for complete project neutrality: check out on any machine, in any directory, and the project is guaranteed to build and deploy.  Additionally, check the project out in Eclipse and it should build with no issues, within minutes.
• Not all developer’s actually care about tools. Some developers simply want their environment setup for them. They have no desire to figure out how Eclipse formats code when you save a file or how to configure and run quality checks after each build; implementing business solutions is their primary concern.
• Enhanced team productivity. If everyone’s world (environment) is the same, it is so much easier to spin up a new developer or help a teammate with a problem. Would you really want each developer to go through the discovery process of setting up the project? In the big picture, isn’t this really just wasted time?
• Helping to ensuring quality coding practices and standards. We also check in the Checkstyle, PMD, and Findbugs configuration and rule sets. Taking advantage of the sharable configuration files, both the Eclipse plug-ins and Ant tasks work from the same rule sets, ensuring complete consistency across the team, no matter where the rules are executed.

I appreciate all of the recent comments; thanks for taking the time to reply. They enable me think about and reconsider the decisions I have made, giving me yet another opportunity to learn from my mistakes!  As far as Eclipse configuration files are concerned, I strongly believe there is far more project value gained by including them, as compared to requiring each developer manage their own environment. One final note, I have no issue with developers wanting to manage their own world, more power to them!  I would hope that these efforts would be to make the overall, shared environment a little better; after all, everyone should be contributing to all aspects of the project! The real point is that everyone should not be required to configure a project, unless they really want or need to.

Unfortunately, it seems that in our current economic environment, it is more important to prevent mistakes rather than take action to address the underlying problems. This strategy can even be taken to the extreme, putting so many gates into the process, that it actually makes it impossible to move forward. It becomes difficult to fix existing issues and nearly impossible to development new functionality. The number of signatures and the amount of evidence required to prevent errors has become more of an accountability tool, rather than a quality tool.

Writing this post made me think of another technique that we always seem to overlook, the “5 Whys“. If you are like me and have not looked at this lately, it is perfect time for a refresher! Just click the link!

After experiencing these challenges, I began a quest of environmental simplification. I understand there are numerous reasons why projects end up with these unfriendly characteristics,  but leaving them unaddressed was just not in my character! I made it my personal mission to ensure that each project that I worked on, I would try to leave it in a little better shape than when I arrived. This is a never ending activity; I hope that those that come after me will have a similar philosophy and continue my quest. It is amazing how easy and fast an application can atrophy, eliminating all of the positive changes that had been previously applied. To keep things simple, I came up with following three project requirements. I try to weave some aspect of them into the development process and architecture of each project that I work on.

Project Principles

Self-Containment, No External Configuration. Environmental Awareness Change Resilient

I hope that most of these principles seem like common sense and are nothing new. Much like many of the XP principals, they are not new or revolutionary, just good reminders of often overlooked practices. I will try to elaborate on each principal in a future post, hopefully, in the near future!

I started addressing the problem my making templates out of my scripts and placing them into a Subversion repository. They could easily be included into a project using the Ant <get> task. A little ugly, but it worked well enough. It did required us to land the XML files in the project directory, such that they could be “imported” into the running build.  Once again, not a problem if you remember to use SVN ignore on the .ant directory; we might have been able to put them in /tmp, but never tried it.

<property name="ant.xml.location" value="http://svn:1234/Ivy/Ant/Common/trunk/main/1.0/xml" />
<mkdir dir=".ant" />
<get src="${ant.xml.location}/ant.ivy.xml" dest="${basedir}/.ant/ant.ivy.xml" usetimestamp="true" />
<get src="${ant.xml.location}/ant.checkstyle.xml" dest="${basedir}/.ant/ant.checkstyle.xml" usetimestamp="true" />

<import file=".ant/ant.ivy.xml" />
<import file=".ant/ant.checkstyle.xml" />

With Ant 1.8, you can actually accomplish this in a much cleaner method. Look at the import task for some quick examples. Our first test was a very simple implementation, but worked like a champ.

I personally like to break my Ant scripts into multiple files, each with it own purpose. Some of my coworkers hate this approach and would prefer a single XML file. In the long run, the multi-file approach seems easier to support, especially when you factor in multiple maintainers. You would never put all of your Java code in a single class, why would you put all of your XML in a single file? I will ultimately create a single file to import; it will import all of the individual XML files. This should make the consumers of the templates a little happier!

<property name="ant.xml.location" value="http://svn:1234/Ivy/Ant/Common/trunk/main/1.0/xml" />
<import>
       <url url="${ant.xml.location}/ant.ivy.xml" />
       <url url="${ant.xml.location}/ant.checkstyle.xml" />
       <url url="${ant.xml.location}/ant.pmd.xml" />
       <url url="${ant.xml.location}/ant.clover.xml" />
       <url url="${ant.xml.location}/ant.findbugs.xml" />
       <url url="${ant.xml.location}/ant.testability.xml" />
</import>

I think the javaresource approach would be the perfect way to solve my problem. Unfortunately, I could not find any way to combine a remote URL with a classpath attribute. It is a neat little trick to remember, maybe this feature could be added to the next version of Ant.

<import>
    <javaresource name="common/targets.xml">
      <classpath location="common.jar">
<!--  <url url="http://somewhere/common.jar/> -->
    </classpath>
  </javaresource>
</import>

One final word about completeness. My personal goal is to make the build system completely self-contained, meaning no external dependencies (outside of Ivy), no custom Ant installations, no Ant-lib requirements, or any manual configuration. Ideally, a developer should be able to check out the project from the version control system and get to work. It should automatically build and execute from Eclipse, and as long as Ant is installed, the developer should just be able to type Ant <target> to accomplish where ever is required.

I have been changing all of my templates to make use of the ivy:cachepath and ivy:cachefileset tasks. In prior Ant implementations, I used Ivy retrieve to access all of the dependencies; my new goal is to only use files from the Ivy cache. Additionally, I think the Ant templates should do their own Ivy dependency management. Now applications teams only need to worry about their own dependencies, not what is required by the environment or build system. The follow example demonstrates how to add Clover to the build process, and ensure that the Clover license is added to your class path, all without really knowing where the files are… Pretty cool!

<ivy:cachepath pathid="classpath.CLOVER" conf="runtime" organisation="atlassian" module="clover" revision="3.0.2" inline="true" />
<ivy:cachefileset setid="fileset.CLOVER" conf="license" organisation="atlassian" module="clover" revision="3.0.2" inline="true" />
<pathconvert pathsep="${line.separator}-> " property="clover.license.path" refid="fileset.CLOVER" setonempty="" />
<dirname file="${clover.license.path}" property="clover.license.dir" />

<taskdef resource="cloverlib.xml">
     <classpath>
          <path refid="classpath.CLOVER" />
          <path location="${clover.license.dir}" />
      </classpath>
</taskdef>