Soapbox Rants and Raves

To be honest, this was not a topic I was actually planning to write about. However, because I received a couple of interesting comments on a recent blog entry, Unfriendly Developer Practices, I thought I should clarify my position.

One of the comments suggested that there should be should be no dependency between the build system and the IDE. Another person suggested the disconnect between the IDE and the build system creates a convenient place for dependency inconsistencies to develop unchecked.

I completely agree with both of those statements and have seen both problems manifested several times. However, there are two simple tools can make this a non-issue: Continuous Integration and Dependency Management.

• Continuous Integration is obvious; if that lazy developer forgets to check in a dependency or update the build scripts, the build fails. Not perfect, but the problem is immediately detected and the relevant people are notified of the situation. The problem can be resolved within minutes.
• The Silver Bullet for me, was the addition of the Ivy Dependency Management tool into our build process. Because the build system and IDE share the same dependency configuration, the project’s dependencies were now managed in single place. Using the IvyDE plug-in, Eclipse simply worked, with no additional configuration. Using the externalized dependencies and basic Ant targets, the project could create a robust, change resilient build system. To achieve this level of robustness, I took advantage of the Ivy post resolve tasks. It was not until I discovered how easily they could isolate the build scripts from the actual dependencies, did it all come together. The real beauty of this approach is that no files (dependencies) are directly referenced. The post resolve tasks create variables which contain all of the appropriate files; the build script can then treat these variables generically, without concern. Nice and clean!

It was an unstated, fundamental requirement to have no “direct” dependency on Eclipse; such that we could all revert back to the wonderful world of Emacs tomorrow, should the need arise. I don’t think which IDE a project chooses to use, is really that important. I am apparently an Eclipse snob; but all I really care about is having Emacs key-bindings! If a majority of the team works on the same platform, I do believe there is real value around this continuity; that just happens to be Eclipse for me!

I have several other reasons for checking in configuration files:

• It quickly highlights wrong doing! If a developer checks in something specific to their environment, they will break everyone on the team. My goal is for complete project neutrality: check out on any machine, in any directory, and the project is guaranteed to build and deploy.  Additionally, check the project out in Eclipse and it should build with no issues, within minutes.
• Not all developer’s actually care about tools. Some developers simply want their environment setup for them. They have no desire to figure out how Eclipse formats code when you save a file or how to configure and run quality checks after each build; implementing business solutions is their primary concern.
• Enhanced team productivity. If everyone’s world (environment) is the same, it is so much easier to spin up a new developer or help a teammate with a problem. Would you really want each developer to go through the discovery process of setting up the project? In the big picture, isn’t this really just wasted time?
• Helping to ensuring quality coding practices and standards. We also check in the Checkstyle, PMD, and Findbugs configuration and rule sets. Taking advantage of the sharable configuration files, both the Eclipse plug-ins and Ant tasks work from the same rule sets, ensuring complete consistency across the team, no matter where the rules are executed.

I appreciate all of the recent comments; thanks for taking the time to reply. They enable me think about and reconsider the decisions I have made, giving me yet another opportunity to learn from my mistakes!  As far as Eclipse configuration files are concerned, I strongly believe there is far more project value gained by including them, as compared to requiring each developer manage their own environment. One final note, I have no issue with developers wanting to manage their own world, more power to them!  I would hope that these efforts would be to make the overall, shared environment a little better; after all, everyone should be contributing to all aspects of the project! The real point is that everyone should not be required to configure a project, unless they really want or need to.

A couple of years ago, I began focusing on construction and deployment complexities. I had just joined a new team and began looking for the commonalities between their architectures and environments. Needless to say, they were all different. The most disturbing issue was the amount of effort required for a new developer to start working with an application. If there was any project documentation, it was probably out of date; resulting in hours or even days of trial and error to actually build and deploy the system. In this post, I’m talking about is building and deploying J2EE applications. The following list represents the top developer unfriendly practices I have observed.

After experiencing these challenges, I began a quest of environmental simplification. I understand there are numerous reasons why projects end up with these unfriendly characteristics,  but leaving them unaddressed was just not in my character! I made it my personal mission to ensure that each project that I worked on, I would try to leave it in a little better shape than when I arrived. This is a never ending activity; I hope that those that come after me will have a similar philosophy and continue my quest. It is amazing how easy and fast an application can atrophy, eliminating all of the positive changes that had been previously applied. To keep things simple, I came up with following three project requirements. I try to weave some aspect of them into the development process and architecture of each project that I work on.

Project Principles

Self-Containment, No External Configuration. Environmental Awareness Change Resilient

I hope that most of these principles seem like common sense and are nothing new. Much like many of the XP principals, they are not new or revolutionary, just good reminders of often overlooked practices. I will try to elaborate on each principal in a future post, hopefully, in the near future!

I have a love/hate relationship with the IvyDE plug-in when it comes to WTP project types. I have literally wasted hours of time trying to get IvyDE to publish into my web application. It is very strange, no pattern at all; it works fine for days at a time and then decides to stop working… I don’t believe that the team has actually developed a “reproducible pattern” of forcing WTP to publish the Ivy dependencies; it is a continual “trial and error” of the following steps, clean the project, Ivy resolve, clean Tomcat, un-deploy the application, redeploy the application; one of these will eventually cause WTP to push the dependencies over. Can you say, frustrating?

Some developers just give up and use Ivy retrieve to dump the dependencies into the WEB-INF/lib directory; this will work, but is not a very elegant solution. To make this work, you are required to refresh the workspace after the Ivy resolve/retrieve completes, which pushes the dependencies into the web application; nasty.

This problem may have been answered in Eclipse 3.6 (Helios). I tested some of the 3.6 milestone releases, but was fooled into thinking that the IvyDE and WTP integration was broken; the J2EE module dependency option was no longer visible on the project properties. I was too busy to investigate (or look at other menu options!), I just assumed that it was just broken. To make a long story short, there is a new menu option called Deployment Assembly. The new option is a much simpler concept to comprehend; it is more flexible and actually seems to work consistently with Ivy. The most obvious change is the presentation; it gives a better picture of how the web application will be assembled. The old J2EE Module Dependency manager allowed you to choose components, but might have been more tailored for building an EAR file, rather than a WAR. Another issues is the inability to exclude any of the “source” folders. If you manage your unit test classes in a separate source folder, they would also be included in the application (WAR). Not necessarily a problem, but not very clean and could possibly hide unintended dependencies between the two source trees.

When using IvyDE, the first step is to add an Ivy classpath container. This will add Ivy dependencies to the classpath of your Eclipse project. If you have a WTP project, you will also need these dependencies copied to the WEB-INF/lib directory. This is easily accomplished by clicking the Add button under Deployment Assembly (under project properties). You can add other projects and external jars as well. Here is where it gets interesting, there appears to be two different options. If you have already added the Ivy classpath container to your build path, click the “Referenced Classpath Entries” option and select the ivy.xml file from your project. There might be a little display problem, as the “Deploy Path” for the classpath container does not appear correct, but it does put the libraries in the correct directory location.

The second option is to remove the Ivy classpath container from the build path, and re-add it under the new Deployment Assembly properties. This option displays properly and copies the dependencies to the WEB-INF/lib directory. I actually prefer this option, but there is a significant problem; there is no way to re-resolve the dependencies; it appears you need to delete the container and re-add it, if you change your ivy.xml file. Rather annoying!

I started writing this blog a couple of weeks ago and was under the impression that the new version was working perfectly. However, I’m currently back to where I started, with inconsistent, unexplainable behavior. Time to submit a question to the Ivy support list. Hopefully, I will have something to report back soon.

I seem to work on a variety of Java applications and find that unit testing is one of the most varied (implementation-wise) pieces of the development process. These applications, created by different development staffs, many which have evolved over the pass few years are all very unique… kind of makes me wonder why? That is a topic for another post!

One debate I always seem to encounter is around where to actually save the unit tests? Should we do in-package testing or rely on the public API. My preferences are pretty simple:

• Unit test should not be stored with the code, but rather created under a secondary source root. On my early projects, I created a sub-package under each package, called test, to manage the relevant files (tests and data). I later realized that I could simplify the Ant build process and enable in-package testing by creating a separate source tree. I have seen projects commingle the actual source code, unit tests, and even the test data files, all within the single source tree. This defaults the testing strategy to in-package testing, and discourages (or even prevents) API testing. I find this approach rather messy and unclear. This approach also complicates the Ant build process; at some point during the process, the unit tests, supporting classes and files should be separated from the actual deployable content.
• I like to name my secondary source root tdd. I know it is not widely practiced, but my hope is that if the developers see that TDD directory every time the open up their IDE, the concept might actually wear off! Maybe someday, one developer (hopefully more) will actually be challenged to write their unit tests first. With Eclipse and jUnit 4.x annotations, I seem to always start with my unit test, and sometimes even refactor code from the unit test into the actual class; kind of a high-bread TDD process, but the thought is always there!
• I also prefer the public API testing strategy verses in-package testing. This line of thinking always takes me back to the Testability Explorer. The concepts behind this metric enforce the idea public API testing and is worth a quick read. Add the Spring Framework to the mix, enabling dependency injection and I see little need for in-package testing. This is typically accomplished by creating a “test” package as the root of the secondary source tree, with sub-packages reflecting the package hierarchy of your classes to be tested. I truly believe that defaulting to in-package testing allows the developers to be very sloppy and even unaware that they are testing a specific internal implementation, rather than externalized behavior presented by the API… This is VERY bad practice in my opinion.
• If there is a need for in-package testing, then the sub-package structure of the classes to be tested can be recreated under the secondary source tree, maintaining a clean separation between the source and test cases. These test should be considered an exception to the norm, rather than common practice. I would hope for a 90-10 or 80-20 ratio, with a majority of the test falling under the test package root (in-package is the minority).

I am kind of tied to Eclipse as my IDE, but do play with NetBeans every so often. I think I could switch to NetBeans, my only real requirement is that the IDE must have Emacs key bindings; some habits are just too hard to break! NetBeans looks like an pretty good tool and seems to be very responsive on my Ubuntu box; I especially like the way it manages plug-ins.  One interesting thing that NetBeans does (maybe a little better than Eclipse), is manage unit testing. NetBeans will automatically create the secondary source tree, but seems to default to in-package testing. It is very easy to add the additional test package into the package structure to enable API testing. I also like the way that NetBeans separates the Test Libraries from the regular Libraries. I’m not sure how well this would work when you use a tool like Ivy, but does make it more obvious, as to which libraries are used for execution verses unit testing… If you do happen to generate a unit test from an existing class, NetBeans will generate more code than Eclipse. I’m not sure how useful this code is, but it does try to create the object under test, invoke the get() methods and perform assertions on the returned values. Might be more noise that it is worth, and anyway, you should be writing the unit test first!

The bottom line is that Unit testing should be easy and valuable. If it becomes too hard or complicated, it might be time to re-address how the unit test strategy is being implemented.

This was making me crazy the other night… I did not spend too much time on it, thinking is was just me. But, after upgrading my laptop and encountering the exact same problem, I knew there was more too it. It seemed like only the cancel buttons actually worked; all of the next and finish buttons did absolutely nothing, other then beep!  I soon realized that I could use the tab key and press enter to move to the next screen; but that did not make me very happy.

A quick Google search turned up several people with the same exact problem. It was kind of interesting. The Ubuntu developers did not want to take responsibility for the problem and said it was an Eclipse issue. The Eclipse community said they never had to do anything special for Eclipse to run on Ubuntu; it was looking like a stand off! I did find the official bug report filed with Launchpad.net, it says the problem is resolved. However, without overriding the following variable, Eclipse is not very usable.

export GDK_NATIVE_WINDOWS=1

It also appears that this problem is affecting many other tools on the new 9.1 release as well. This simple fix will get you back into action, without having to wait for the real fix.

The Eclipse developers did a really nice upgrade to the Plug-in manager in the new Galileo release. Managing plug-in sites seems much easier and cleaner with the new Available Software Sites menu option in the main preferences area. Not really a new feature, but seemingly more obvious, is the ability to import and export update sites to and from your team mates. I have exported my settings and linked them to the XML file icon on the left. Simply download the XML file and import them into Eclipse and you are ready to go. I can’t believe I never used this feature in the previous versions!

Also, don’t forget that you can also have Eclipse automatically find new updates for you. This is not a new feature either, but one that I always seem to forget! I just hit the Check for updates option every so often; not nearly as convenient! You can find this option under the Install/Update preferences panel as well.

Once you have your software sites configured, it is time to install or update your configuration.  Under the main Help option, select Install New Software.  You can select multiple plug-ins for installation  or use the new Work with: option. You can filter your plug-ins based on a specific site’s configuration file. I’m not exactly sure why I like this better, but the old multi-tab Ganymede version was just too busy. I think the new screens are much cleaner and easier to work with.

I have to give credit to a co-worker who overheard me complaining about Eclipse one day… Ever since upgrading to the Ganymede version of Eclipse, the launch buttons for executing and debugging applications and unit tests were tied to the currently active editor. This was so annoying, I could not believe the Eclipse developers changed this behavior. Every time I clicked on the button, Eclipse would ask me what I wanted to do…. No, I don’t want to execute the XML file I was looking! No, I don’t want to execute the random Java file that I just fixed; I want to run the last unit test!

Pre-Ganymede versions of Eclipse simply ran the previously launched application or unit test. It was so convenient to just click the button and rerun the test. Apparently, I was not smart enough to look at all of the available options in the preferences menu; I conditioned myself to click on the little arrow next to the launch buttons and select the test I wanted to run from the menu. To my surprise, there was a new menu option that controlled the behavior of these buttons! Unfortunately, the default behavior is the un-desirable one, in my opinion; but with a quick click of the mouse, the world is right again! Hopefully, this little tip will make your Eclipse usage a little happier and smoother too. Thanks Jason!

I might be in the minority, but I’m just not that excited by this release of Eclipse. I started using the RC3 version for real last week, and have had no issues at all; it seems very solid.  I attempted to use with the first couple release candidates, but they did not seem to play well with the Subversion plug-ins. I’m sure it was user error; it could not have been that messed up! For whatever reason, Eclipse completely toasted my work area (multiple attempts) after every commit. Needless to say, I did not even bother using them.  I have Eclipse integrated with the Spring IDE, Checkstyle, PMD, and Ivy plug-ins. I heavily use the Web Tools Platform for deploying web applications to Tomcat.  At least for what I have been doing lately, there is nothing to get me really excited. I think the best part is, that everything still just works! I notice a new little feature every so often, but nothing that says that’s a keeper! I have been using the Implementors plug-in for many years and was glad to see they finally implemented this feature within Eclipse. It works pretty well,  but they only implemented half of the functionality! With the Implementors plug-in, you can click on the interface method and ask to see all of the concrete implementations.  You can, sort of, get the same functionality through the type hierarchy view, but not as nicely. After seeing the Open Implementation feature, I thought this was a new feature too; I did not know that you could just click on a URL and have it open up in the browser, kind of handy!

One other thing that seems new, is the ability to search using partial camel-case entries. If you are really lazy or can not spell worth a crap, then you will probably use this feature a lot. Simple type in mixed case letters in the entry field, and Eclipse will automatically match to those classes. I thought the bold-ing of the matched letters was a nice touch.

There are two features that need some work, especially when you are moving between (upgrading) Eclipse versions.

1. I thought the Clean Up and Save Actions functionality were excellent additions to the Ganymede release. I still can not figure out why I have to enter my configurations on both option panels. You can import/export the configuration on the Clean Up panel, but not on the Save Actions panel; you have to re-enter everything. I understand that some people might want different behaviors, but you should at least be able to import/export your settings on the Save Actions panel too; allowing us to reuse the other panel’s settings. I thought for sure this would have been fixed, small  bummer.
2. The Content Assist Favorites drives me crazy, especially because I’m a big jUnit believer/implementer. Because I only use jUnit 4.x annotations and static import assertions in my unit tests, Eclipse is not smart enough to add the imports when I hit <CTRL><SHIFT>o or ask for the content assist. Maybe I learned something new tonight, from this blog.  I typically enter all of the jUnit assertion signatures (method names only) that I typically use. It looks like their might be an easier way, just by specifying the Type, I will have to try this tomorrow! Personally, I think jUnit assertions should be added by default, but that is just me!

If I’m missing out on a new killer Eclipse feature that was added to Galileo, please let me know! I don’t want to take away any of the great work that was done on this release, unfortunately, I don’t use any of those new additions! But I don’t think you can go wrong with this upgrade.

I was looking for a development build of the IvyDE plug-in for Eclipse tonight and I stumbled across the Apache Hudson server. You have to check this out. Pretty cool, Hudson running over 150 jobs!  Their build farm consist of both Solaris and Ubuntu servers. What I found most interesting was how they used Hudson for creating an Eclipse update site. I subscribe to the Hudson mailing list, summarized here, to follow what is going on with the project. People seem to use Hudson for a variety of different purposes, such as a cross platform testing tool and even as a deployment tool. I have only been able to get people interested in Hudson as a “Cruise Control” replacement…. but at least it is a good start!

This site also demonstrates how multiple projects can be linked together, triggering builds of dependent downstream projects.  I’m not exactly sure how these jobs were assembled, but at a high-level, Ivy and IvyDE each have its own build loop (job). Each job is responsible for creating build artifacts. After a successful build of either job, a third job is triggered which creates/updates and Eclipse update site.  Pretty slick. (As any FYI, the dev plug-in looks better, new presentation and several new new options)

Another small plug for Continuous Integration. We recently set up something kind of cool (trivial, but cool)! As we continue trying to convince the powers that be, to move away from Weblogic towards Tomcat (or even JBoss), we found a really nice little feature. Since Tomcat and JBoss can hot deploy a new version of the WAR file, just by noticing a difference on the file system, combined with the fact that we can spin up a Tomcat instance in about 2 minutes, we actually created an auto deployment mechanism.  Our Weblogic environment seems so cumbersome in comparison…. We could have built the WAR and deployed in in our build loop, but that would have minimally added another 25 minutes to the loop (let’s just say we have an interesting deployment process). With Tomcat, we did a one time deployment to the location of the WAR file inside the Hudson workspace. Now, with each series of check-ins, we build the WAR as normal and Tomcat instantly deploys it, in seconds! If we were a little more sophisticated, we would write some jUnits to bang against it, making sure we did not break anything or create any new dependencies.  Maybe I will throw that in during my next project!!! I know this integration was possible before (with Weblogic), but no way as simple!  Even better Continuous Integration… on my last project I had Ant spin up an embedded Jetty web server, ran my jUnits against the Clover instrumented code, and finally had Ant shut down Jetty…. Pretty slick, from a code coverage collection perspective…

I’ve ran across several blogs that listed their favorite Eclipse plug-ins.  Since I am such a huge Eclipse fan and am always preaching about the power and benefits, it seemed like a good idea for me to document my preferences and experiences. First, you need to start with the Eclipse Java EE version, probably the Ganymede version (3.4), but Galileo (3.5) will be released next week. This Eclipse bundle will give you more than you need, but is the best starting point. Out of the box, Eclipse has many useful features:

• Mylyn – Task focused development support; here is some background information.
• Web Standard Tools – Gives you all of the nice editors for XML, HTML, JSP, etc.
• J2EE Standard Tools – Provides support for J2EE projects, such was the Servlet and EJB.
• Remote System Explorer – Excellent plug-in for accessing remote systems. It enables you to remotely edit files and provides shell access, including SSH.

Here is my personal Top 10+ list of Eclipse Plug-ins:

Just a random link of several more plug-ins, http://www.plentyofcode.com/2007/07/most-useful-top-50-eclipse-plug-ins.h6tml There were a few plug-ins on the list that looked interesting, maybe you will think so too. Make sure you read the comments!