Jul 20 2010

Eclipse Project Configurations – Shared or Personal?

Category: Continuous Integration,Eclipse,Software DevelopmentPhil @ 12:01 am

To be honest, this was not a topic I was actually planning to write about. However, because I received a couple of interesting comments on a recent blog entry, Unfriendly Developer Practices, I thought I should clarify my position.

Question of the Day
Should your IDE (Eclipse) configuration files be checked in as part of your project?

One of the comments suggested that there should be should be no dependency between the build system and the IDE. Another person suggested the disconnect between the IDE and the build system creates a convenient place for dependency inconsistencies to develop unchecked.

I completely agree with both of those statements and have seen both problems manifested several times. However, there are two simple tools can make this a non-issue: Continuous Integration and Dependency Management.

  • Continuous Integration is obvious; if that lazy developer forgets to check in a dependency or update the build scripts, the build fails. Not perfect, but the problem is immediately detected and the relevant people are notified of the situation. The problem can be resolved within minutes.
  • The Silver Bullet for me, was the addition of the Ivy Dependency Management tool into our build process. Because the build system and IDE share the same dependency configuration, the project’s dependencies were now managed in single place. Using the IvyDE plug-in, Eclipse simply worked, with no additional configuration. Using the externalized dependencies and basic Ant targets, the project could create a robust, change resilient build system. To achieve this level of robustness, I took advantage of the Ivy post resolve tasks. It was not until I discovered how easily they could isolate the build scripts from the actual dependencies, did it all come together. The real beauty of this approach is that no files (dependencies) are directly referenced. The post resolve tasks create variables which contain all of the appropriate files; the build script can then treat these variables generically, without concern. Nice and clean!

It was an unstated, fundamental requirement to have no “direct” dependency on Eclipse; such that we could all revert back to the wonderful world of Emacs tomorrow, should the need arise. I don’t think which IDE a project chooses to use, is really that important. I am apparently an Eclipse snob; but all I really care about is having Emacs key-bindings! If a majority of the team works on the same platform, I do believe there is real value around this continuity; that just happens to be Eclipse for me!

I have several other reasons for checking in configuration files:

  • It quickly highlights wrong doing! If a developer checks in something specific to their environment, they will break everyone on the team. My goal is for complete project neutrality: check out on any machine, in any directory, and the project is guaranteed to build and deploy.  Additionally, check the project out in Eclipse and it should build with no issues, within minutes.
  • Not all developer’s actually care about tools. Some developers simply want their environment setup for them. They have no desire to figure out how Eclipse formats code when you save a file or how to configure and run quality checks after each build; implementing business solutions is their primary concern.
  • Enhanced team productivity. If everyone’s world (environment) is the same, it is so much easier to spin up a new developer or help a teammate with a problem. Would you really want each developer to go through the discovery process of setting up the project? In the big picture, isn’t this really just wasted time?
  • Helping to ensuring quality coding practices and standards. We also check in the Checkstyle, PMD, and Findbugs configuration and rule sets. Taking advantage of the sharable configuration files, both the Eclipse plug-ins and Ant tasks work from the same rule sets, ensuring complete consistency across the team, no matter where the rules are executed.

I appreciate all of the recent comments; thanks for taking the time to reply. They enable me think about and reconsider the decisions I have made, giving me yet another opportunity to learn from my mistakes!  As far as Eclipse configuration files are concerned, I strongly believe there is far more project value gained by including them, as compared to requiring each developer manage their own environment. One final note, I have no issue with developers wanting to manage their own world, more power to them!  I would hope that these efforts would be to make the overall, shared environment a little better; after all, everyone should be contributing to all aspects of the project! The real point is that everyone should not be required to configure a project, unless they really want or need to.

https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png


Jul 19 2010

Self-destructive Tendencies and Root Cause Analysis

Category: Software DevelopmentPhil @ 12:01 am

One of my favor blogs, 37Signals, had an interesting post. It quoted Ed Catmull, President of Pixar, on the topic of creative vision and leadership. The full interview can be read here; it is actually rather interesting. What caught my attention was the quote concerning self-destructive tendencies, “if everyone is trying to prevent error, it screws things up. It’s better
to fix problems than to prevent them
“.

Unfortunately, it seems that in our current economic environment, it is more important to prevent mistakes rather than take action to address the underlying problems. This strategy can even be taken to the extreme, putting so many gates into the process, that it actually makes it impossible to move forward. It becomes difficult to fix existing issues and nearly impossible to development new functionality. The number of signatures and the amount of evidence required to prevent errors has become more of an accountability tool, rather than a quality tool.

Writing this post made me think of another technique that we always seem to overlook, the “5 Whys“. If you are like me and have not looked at this lately, it is perfect time for a refresher! Just click the link!

https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png


Jul 11 2010

Unfriendly Developer Pactices – Adding uncessary environmental complexities

A couple of years ago, I began focusing on construction and deployment complexities. I had just joined a new team and began looking for the commonalities between their architectures and environments. Needless to say, they were all different. The most disturbing issue was the amount of effort required for a new developer to start working with an application. If there was any project documentation, it was probably out of date; resulting in hours or even days of trial and error to actually build and deploy the system. In this post, I’m talking about is building and deploying J2EE applications. The following list represents the top developer unfriendly practices I have observed.

Developer Unfriendly Practices
  1. Without first running a setup task in Ant, the project does not build in Eclipse.
  2. The Eclipse .project and .classpath files are not checked into version control system with the source.
  3. The project’s Ant scripts are generic and non-deterministic; they have to be edited for your individual environment before building the project.
  4. The project requires specific libraries or packages to be installed on your machine before it can be built.
  5. Eclipse project not configured as an WTP project.
  6. Minimal jUnits and no continuous integration.
  7. J2EE container dependencies built into the project.

After experiencing these challenges, I began a quest of environmental simplification. I understand there are numerous reasons why projects end up with these unfriendly characteristics,  but leaving them unaddressed was just not in my character! I made it my personal mission to ensure that each project that I worked on, I would try to leave it in a little better shape than when I arrived. This is a never ending activity; I hope that those that come after me will have a similar philosophy and continue my quest. It is amazing how easy and fast an application can atrophy, eliminating all of the positive changes that had been previously applied. To keep things simple, I came up with following three project requirements. I try to weave some aspect of them into the development process and architecture of each project that I work on.

Project Principles

  1. Self-Containment, No External Configuration.
  2. Environmental Awareness
  3. Change Resilient

I hope that most of these principles seem like common sense and are nothing new. Much like many of the XP principals, they are not new or revolutionary, just good reminders of often overlooked practices. I will try to elaborate on each principal in a future post, hopefully, in the near future!

https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png


Jul 05 2010

Ant 1.8, Reusable XML Scriptlets, and Ivy for Self-Containment

Category: Software DevelopmentPhil @ 8:33 am

I started reading about EasyAnt sometime ago on the Ivy developer forums. It was a very interesting concept, storing reusable Ant targets in your Ivy Repository and pulling them into projects at build time. Ivy works great for jar files, so why not Ant build scripts?  I have a set of Ant scripts in my toolbox that seem to get copied from project to project; it bothers me every time I copy them! We gave EasyAnt a quick look, but it had one show stopper; it required every developer and builder to have EasyAnt installed (rather than just Ant). This is a little thing, but when you are working with dozens of projects and hundreds of developers, it seemed like too much of a hurdle.  I wanted Ant to be the only installation requirement and be “self contained”. Just type “ant…” Nice and simple.

I started addressing the problem my making templates out of my scripts and placing them into a Subversion repository. They could easily be included into a project using the Ant <get> task. A little ugly, but it worked well enough. It did required us to land the XML files in the project directory, such that they could be “imported” into the running build.  Once again, not a problem if you remember to use SVN ignore on the .ant directory; we might have been able to put them in /tmp, but never tried it.

<property name="ant.xml.location" value="http://svn:1234/Ivy/Ant/Common/trunk/main/1.0/xml" />
<mkdir dir=".ant" />
<get src="${ant.xml.location}/ant.ivy.xml" dest="${basedir}/.ant/ant.ivy.xml" usetimestamp="true" />
<get src="${ant.xml.location}/ant.checkstyle.xml" dest="${basedir}/.ant/ant.checkstyle.xml" usetimestamp="true" />

<import file=".ant/ant.ivy.xml" />
<import file=".ant/ant.checkstyle.xml" />

With Ant 1.8, you can actually accomplish this in a much cleaner method. Look at the import task for some quick examples. Our first test was a very simple implementation, but worked like a champ.

I personally like to break my Ant scripts into multiple files, each with it own purpose. Some of my coworkers hate this approach and would prefer a single XML file. In the long run, the multi-file approach seems easier to support, especially when you factor in multiple maintainers. You would never put all of your Java code in a single class, why would you put all of your XML in a single file? I will ultimately create a single file to import; it will import all of the individual XML files. This should make the consumers of the templates a little happier!

<property name="ant.xml.location" value="http://svn:1234/Ivy/Ant/Common/trunk/main/1.0/xml" />
<import>
       <url url="${ant.xml.location}/ant.ivy.xml" />
       <url url="${ant.xml.location}/ant.checkstyle.xml" />
       <url url="${ant.xml.location}/ant.pmd.xml" />
       <url url="${ant.xml.location}/ant.clover.xml" />
       <url url="${ant.xml.location}/ant.findbugs.xml" />
       <url url="${ant.xml.location}/ant.testability.xml" />
</import>

I think the javaresource approach would be the perfect way to solve my problem. Unfortunately, I could not find any way to combine a remote URL with a classpath attribute. It is a neat little trick to remember, maybe this feature could be added to the next version of Ant.

<import>
    <javaresource name="common/targets.xml">
      <classpath location="common.jar">
<!--  <url url="http://somewhere/common.jar/> -->
    </classpath>
  </javaresource>
</import>

One final word about completeness. My personal goal is to make the build system completely self-contained, meaning no external dependencies (outside of Ivy), no custom Ant installations, no Ant-lib requirements, or any manual configuration. Ideally, a developer should be able to check out the project from the version control system and get to work. It should automatically build and execute from Eclipse, and as long as Ant is installed, the developer should just be able to type Ant <target> to accomplish where ever is required.

I have been changing all of my templates to make use of the ivy:cachepath and ivy:cachefileset tasks. In prior Ant implementations, I used Ivy retrieve to access all of the dependencies; my new goal is to only use files from the Ivy cache. Additionally, I think the Ant templates should do their own Ivy dependency management. Now applications teams only need to worry about their own dependencies, not what is required by the environment or build system. The follow example demonstrates how to add Clover to the build process, and ensure that the Clover license is added to your class path, all without really knowing where the files are… Pretty cool!

<ivy:cachepath pathid="classpath.CLOVER" conf="runtime" organisation="atlassian" module="clover" revision="3.0.2" inline="true" />
<ivy:cachefileset setid="fileset.CLOVER" conf="license" organisation="atlassian" module="clover" revision="3.0.2" inline="true" />
<pathconvert pathsep="${line.separator}-> " property="clover.license.path" refid="fileset.CLOVER" setonempty="" />
<dirname file="${clover.license.path}" property="clover.license.dir" />

<taskdef resource="cloverlib.xml">
     <classpath>
          <path refid="classpath.CLOVER" />
          <path location="${clover.license.dir}" />
      </classpath>
</taskdef>
https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png


Jul 02 2010

Must have Android Applications…

Category: AndroidPhil @ 12:01 am

My wife gave me a Motorola Droid last December for my birthday. I have been refraining from blogging about it, as there are about a million other phone blogs out there…  However, there have been so many good applications released in the last few weeks, that I thought I would share my “Best Apps List”. I think the Android platform has made some significant progress in the last few months. It finally seems like the phone platform does not really matter any more, Android or IPhone OS.  All of the “must have” applications are available on the relevant platforms. This is great for consumers, as we are not forced into a platform for specific applications, but rather the experience or philosophy. With the recent releases of Dropbox, SlingPlayer and Kindle applications,  combined with all of the other major applications, WordPress, Facebook, Twiiter, Ebay, Flixter, NFL, DirecTV, etc… Now, you have the option to burn hours of your life away on the phone, rather than sitting at your computer!

I use the following applications on a regular basis, applications that give me the most value from my phone and make the experience fun! Sorry if you were looking for some good games, I’m not much of a gamer and most of the games seem pretty weak, when compared to the IPod Touch games that my kids play. Maybe that will all change with the 3.0 release of Android that was just announced…

LauncherPro I have not tried many of the home screen replacement applications, but Launcher Pro is a keeper. It supports the multiple desktops, tons of customization, and performs very well. I really liked the concept of Slide Screen, but the RSS reader did not sync with Google, and I don’t think it worked with IMAP mail servers.
Swype This by far the coolest application/utility on Android. It takes a little getting used to, but once you build confidence and become fluid with your swipes,  you can really crank out those words. I’m still amazed how well it seems to determine the words I swipe. It is still beta, but seems to be included in many of the newest Android phones. I hope they don’t take advantage of us when it comes out of beta; I’m sure I would purchase it; it is that good.
NewsRob Until this week, this has been my RSS reader of choice. It works very well, so well that I did not bother to purchase the pro version. I read a lot of feeds on my phone and don’t have any complaints. However, I just installed Google Reader, and am considering the switch. It works equally as well, but I think I like the user interface a little better; plus it gives you a widget for your desktop!
Kindle Not too much to say about this application, as it was recently released. It works exactly like my DX. I installed the software, logged into my Amazon account, and it synced right up the to book I was reading.  I have been waiting for this application for a long time, as I did not want to carry both my Kindle and phone. Now, I can pick up my book, any time I want. Sweet.. If there were only more hours in the day to read!
Pandora Pandora is just the bests. I run two miles every morning and listen exclusively to Pandora.  If I want to listen to my own music, I use ^3 (cubed). It seems much nicer than the default Android application, but it is just hard to beat Pandora!
WP Stats I use WP Stats and Google Analytics on my website to track what you are reading, but seem to only watch the WP Stats numbers. Google is obviously cool, but too much for my little blog. This application gives me most of what I really care about, daily visitor information. It can also provides a simple graph of our visitor trends. Unfortunately, the application does not do weekly or monthly roll-ups like the WordPresss plug-ing does, but still worth it!
Craigslist Notification Unless you are into Craigslist, this application will not do you much good! I use it to watch Craigslist for items and alert me when they show up. It actually works really well, providing multiple queries and pictures of the items that I’m interested in.
Dropbox I was very happy when Dropbox finally came out for Android. I have been using Dropbox on my home computers for quite some time. It works great to share files between my Linux and Windows machines, and even store files that you want to access remotely.
RAMDroid Widget This is a little widget that looks like an old fashion memory chip. You just give it a tap and it will show you how much free memory (system) you have left. More of a geek thing, but I like it. Another nice little geek utility, is Uptime. Makes me feeling like I’m really running a little Linux box!
Handcent SMS Text Messaging application, much more visually appealing than the default application. Pictures, audio, treading, pop-up notifications, and many other configurable options. Highly recommended.
Keepass Droid This is one of my favorite, multi-platform applications. I use it on my Windows and Linux boxes, and now Android. It is a basic, open-source password vault / manger. The Windows version is really nice; you can just double click on an entry and it opens the default web browser and fills in the user id and password. I realize there are lots of other solutions for this problem, but this one is free and works on all major platforms.
DiskUsage Another geeky utility. Use this to keep an eye on your disk (SD) usage. I thought the user interface was very innovative. You can do the typical pinch, zoom, double tap actions to explore your file system.
Google Voice If you have a Android phone, then you have to get Google Voice! I don’t use the VOIP feature; I’m thinking it won’t be a reality until Verizon rolls out their 4G network. I like the the voice mail features and it as my primary voice mail system. The application gives you “Visual Voicemail” without any additional cost. The transcription service is a little iffy, but the experience and interaction with your voice mail is simplified and streamlined.
App Brain This is a simple little application and web site to watch what other users are downloading or looking for. I check it out every so often to find new and popular applications.
Timeriffic Simple application that lets you configure profiles which can be automatically activated at different times of the day. I have a profile to mute my phone at 9:00 each night and another which turns it back on when I get back to the office. You can also control the WiFi, brightness, and several other phone functions.
Fancy Widget This is an HTC clone widget, which makes your non-HTC phone look ‘cool’. Unfortunately it was so nice, HTC has forced the developer to discontinue development and remove it from the market. I would keep my eye on this developer, as he promises to release an ever better version; I guess we will have to wait and see.
https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png


Jul 01 2010

Eclipse 3.6 Helios: WTP + Ivy and Deployment Assembly… Working reliably at last?

Category: Eclipse,Software DevelopmentPhil @ 12:01 am
I have a love/hate relationship with the IvyDE plug-in when it comes to WTP project types. I have literally wasted hours of time trying to get IvyDE to publish into my web application. It is very strange, no pattern at all; it works fine for days at a time and then decides to stop working… I don’t believe that the team has actually developed a “reproducible pattern” of forcing WTP to publish the Ivy dependencies; it is a continual “trial and error” of the following steps, clean the project, Ivy resolve, clean Tomcat, un-deploy the application, redeploy the application; one of these will eventually cause WTP to push the dependencies over. Can you say, frustrating?

Some developers just give up and use Ivy retrieve to dump the dependencies into the WEB-INF/lib directory; this will work, but is not a very elegant solution. To make this work, you are required to refresh the workspace after the Ivy resolve/retrieve completes, which pushes the dependencies into the web application; nasty.

This problem may have been answered in Eclipse 3.6 (Helios). I tested some of the 3.6 milestone releases, but was fooled into thinking that the IvyDE and WTP integration was broken; the J2EE module dependency option was no longer visible on the project properties. I was too busy to investigate (or look at other menu options!), I just assumed that it was just broken. To make a long story short, there is a new menu option called Deployment Assembly. The new option is a much simpler concept to comprehend; it is more flexible and actually seems to work consistently with Ivy. The most obvious change is the presentation; it gives a better picture of how the web application will be assembled. The old J2EE Module Dependency manager allowed you to choose components, but might have been more tailored for building an EAR file, rather than a WAR. Another issues is the inability to exclude any of the “source” folders. If you manage your unit test classes in a separate source folder, they would also be included in the application (WAR). Not necessarily a problem, but not very clean and could possibly hide unintended dependencies between the two source trees.

When using IvyDE, the first step is to add an Ivy classpath container. This will add Ivy dependencies to the classpath of your Eclipse project. If you have a WTP project, you will also need these dependencies copied to the WEB-INF/lib directory. This is easily accomplished by clicking the Add button under Deployment Assembly (under project properties). You can add other projects and external jars as well. Here is where it gets interesting, there appears to be two different options. If you have already added the Ivy classpath container to your build path, click the “Referenced Classpath Entries” option and select the ivy.xml file from your project. There might be a little display problem, as the “Deploy Path” for the classpath container does not appear correct, but it does put the libraries in the correct directory location.

The second option is to remove the Ivy classpath container from the build path, and re-add it under the new Deployment Assembly properties. This option displays properly and copies the dependencies to the WEB-INF/lib directory. I actually prefer this option, but there is a significant problem; there is no way to re-resolve the dependencies; it appears you need to delete the container and re-add it, if you change your ivy.xml file. Rather annoying!

I started writing this blog a couple of weeks ago and was under the impression that the new version was working perfectly. However, I’m currently back to where I started, with inconsistent, unexplainable behavior. Time to submit a question to the Ivy support list. Hopefully, I will have something to report back soon.

https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png