Mar 25 2009

Mylyn – Task Focused Development

Category: Software DevelopmentPhil @ 8:30 pm

I recently wrote a post about how Eclipse helps to maintain the developer’s focus; what I really wanted to write about was the Mylyn plug-in.  I recently worked on a small project where we setup a completely open-source development environment, built on top of an Ubuntu Linux server. What made the environment interesting, was how well everything worked together. One of the more challenging issues working on a small project is the communication and task management; this is where Mylyn helps smooth out the process.

Mylyn is a simple Eclipse plug-in that integrates with a large number of commercial and open-source issue tracking systems. On the surface, it looks like any other tracking tool, but provides far more functionality than a simple web-based solution.  Building on my previous post, Emacs to Eclipse, the developer can conveniently manage their programming assignments without leaving Eclipse. It even has a notification system which alerts the developer to newly created or assigned tasks. I believe this approach is far more effective than relying on email or external applications to manage the development process.

What makes Mylyn unique, is that each task can be assigned a context. The context represents the state of the Eclipse session, maintaining a list of all of open files. The first step is for the developer to activate a task. Once activated, the developer has the ability to assign a context to that task.  From this point forward, as long as that task is active, the context will be kept in sync with all of the files that the developer opens.  If the developer chooses to activate a different task, the current task is deactivated and the new task is activated. The open files are automatically closed and the files linked with the new task are opened; even restoring the cursor position within the files.  The is extremely helpful when working on multiple, unrelated tasks which are being worked in parallel.

There are a variety of other subtle integrations that also improve the developer’s effectiveness:

  • One of the more convenient features is the ability to create tasks from any “warning” or “TODO” comment. The developer just right clicks on the marker to generate the new task.
  • Mylyn is also integrated with the Team Plug-in (SVN/CVS). It will automatically populate the commit message with the task information.
  • Using the plug-in’s file upload functionality,  developers can easily attach supporting documentation (stack traces or generated files). There is even a built-in screen capture utility which allows the developer to grab, crop, and annotate images, which can be quickly attached to the task.
  • Mylyn magically tracks the amount of time the developer spends working on a task. It appears to be the real time spent on the ticket (based on having Eclipse open and active, not simply based on when the ticket was opened and closed; pretty cool!). This could be very useful in Agile projects, comparing the estimated time to the actual time spent on the task. What a great way to help the developer to improve their future estimations.

The primary issue tracking tool where I work does not integrate with Mylyn. It would be nice if it did, but I don’t know how important that really is. I have been thinking about the integration of a secondary, fine grain task management tool into the environment that is more developer focused.  I actually think it would be interesting to have two issue tracking systems. The first system would be exclusively for the QA and production support teams, mainly to track defects discovered in testing or production related issues.  The second system would be used exclusively to manage the development process. Why create tens or hundreds of tickets in the real ticket tracking system, just for the development team to build the next release? The answer really depends on the scope or purpose of the ticket, but generally it is too much work to create task level assignments in real tracking system. I would really like the tickets in the development environment be more fluid and granular, actually representing the work that needs to be done, not an outdated, vague interpretation of what might happen during development cycle. If a create needs to be created for a developer to refactor a method or add a jUnit, no big deal.  If I’m using the real production ticketing system, I would need several levels of approvals, both from the business sponsor and test teams. Unfortunately, this impledes progress and the actual work being performed is of no interest to them anyway, it is just part of the normal development process. The bottom line is that I currently have no way to easily task or manage the development process or myself; it is currently done through email chains and some crude “TODO” comments in the code. Kind of a bummer when the technology exists, for free (minimal configuration time) and could actually be a huge benefit for the development teams. If you include the planning aspects of the tool, I could see this being a huge help for the project managers as well.

Unfortunately, because of the infrastructure requirements (mysql, tomcat, etc.) of the open source tools (bugzilla, trac, etc.), it is politically too difficult for teams to integrate in the corporate environment. I recently read about an interesting Mylyn connects that lets you store the tracking information in a relational database, rather than a full featured ticket tracking system. I have yet to explore this option, but it might open up the door for allowing me to utilize Mylyn, without the infrastructural overhead. I will keep you posted!

Mar 11 2009

Build, Package, Deploy

Category: Software DevelopmentPhil @ 8:08 pm

A recent manager of mine worked tirelessly to drill this concept into our team: Build, Package, Deploy. If you step back and think, it is a very simple, yet powerful paradigm. I work in an environment that has not fundamentally changed the way we deploy J2EE applications for at least, I am only guessing here, but could be close to 10 years. The group that managed the Internet defined a standard footprint, multi-site J2EE cluster, with all components (Servlet, EJB, etc.) deployed on the same instance. Everyone was given the same configuration and we ended up building a large collection of heavy, monolithic applications. The worst part was that each application was dependent on the specific J2EE container and hardwired to the deployment environment, making an extremely static, inflexible, unscalable solution. Many of us developer’s silently rebelled against the J2EE bandwagon and built simple Servlet-based applications; making sure to use as little as possible, from the provided J2EE container (no connection pools, or even JMS). We pushed to use a simple Servlet container, such as Tomcat or Jetty, but our pleas went unanswered. The default architecture was structured in such a way, that everyone created huge, highly coupled, monolithic web applications; a single WAR that did everything, from vending JSPs, to performing business functions, to accessing the database. Unfortunately, I too was brainwashed by the architecture and stopped thinking about how architecture could be done differently. Fortunately for me, that recent manager was new to the company and had completely different ideas about architecture. Instead of starting a revolution, we embarked on a evolutionary journey to change the way we build software, rather than focusing on how the systems were deployed. One of the first points of enlightenment, to change the way we thought about building system: build it, package it, and then deploy it.

I will tie this paradigm back to the Lean principle of postponing decisions (committing to something) until you actually have to make the decision (the commitment). How does this apply here? When we build applications today, the developers know up front, where and how the system (code) will be deployed. The developer will make design decisions, many of which are unconscious, based on their knowledge of the deployment environment; hardwiring their implementation for what the application will look like when it is deployment. Without giving this much thought, it almost sounds like the right process. Unfortunately, we probably just boxed ourselves right into a corner!

Would it not be better, if the developers simply built components, the true business functionality, without thinking about how the software would ultimately be packaged or deployed? Obviously, there would be some architectural patterns to be followed, but the developer’s would have no idea how their component will be assembled into the final application. Now let’s imagine that we have a Component Packaging Specialist; they would be responsible for bundling up components into deployable packages. This person determines the optimal combination of components, creating a virtual application that performs the required business functionality. This virtual application is bundled in such a way as to create minimal dependencies between bundles, which are independently migrate-able and allow high scalability. Finally, the bundles are given over to a Deployment Specialist, who figures out how best to deploy the virtual application into the environment. Depending on the operational requirements, the virtual application could be deployed in any J2EE or simple Servlet container. How many of each bundles are to be deployed would be determined by the predicted transaction volumes. If the actual volume exceeds the expectations, the Deployment Specialist can easily adjust the environment to accommodate the volumes, without involving the development staff.

The primary point is to allow as much deployment flexibility as we can afford, by not letting the code dictate how or where the application will be deployed. This approach provides less risk (no need to change code) and creates fewer dependencies (better portability). Developers should remember and separate these three tasks as they design and implement systems, even if one person is performing all three roles. By separating these three tasks (actual disciplines), we should end up with better systems, providing the operational flexibility to meet the ever changing business demands.

Mar 08 2009

Trying to save a few dollars…

Category: Go GreenPhil @ 7:33 pm

I always wanted one of those electric monitoring devices, but always found enough skeptical reviews that I never went through with the purchase.  After talking to a friend who was going green, he mentioned that he had purchased a Kill a Watt device. It will monitor the electric consumption of anything (110V) that is plugged into it. So, following his lead, I went ahead and eBay-ed myself a Kill A Watt EZ. The EZ version will actually compute the electricity consumption in dollars per day, month, or year; based on the cost of a killowatt-hour charged by the electric company.

I have only had it for a couple of days, so I can not report too much at this point. It is very interesting to watch (from a geek perspective) how much electricity a TV consumes in stand-by mode, verses when it is actually turned on. I need to let my wife know this, as she likes to listen to the DirecTV music stations, with the TV on! Another friend said I might turn into the “Electricity Police”, you never know, if I have some numbers to back up my arguments, the kids might actually listen!

Here are a few things that I have monitored thus far:

  • A wall mounted water fountain. I just has little water pump that runs 24×7.  This was not too bad, only about $5.37 a year.
  • The 12 year old refrigerator. Only monitored it for 3 hours, and it ran up to $108 per year.
  • My computer, 24″ LCD, and HD DirecTV DVR, on a UPS, all of which also run 24×7. This totaled up to about $124 a year!

It will take some time, but there are lots of little things I’m curious about; such as the cordless phone chargers,  two printers in stand-by mode, the freezer, the washing machine, etc… I still have some big things too look at.

So what can I do with these numbers? Well, I popped over to, picked out a new 21 cubic foot refrigerator, just to check out the Energy Guide information. The new appliance had annual operating expense of only $45 dollars, using 10.65 cents a kWh. I was using 6.3  cents a kWh! I find this way to hard to believe, if I recalculate the cost of the new refrigerator to the 6.3 cent rate, the annual operating cost drops to only $27, which would save me about $81 per year.  Quite a bit of money, but would still take quit a bit of time to recoup the cost of a new appliance, assuming that a new refrigerator would cost at least $1,000.

I honestly don’t think I can make that many changes to save money, without actually unplugging things or replacing the big consumers; time will tell! My family wants to put solar panels on the roof and sell the excess electricity back to the power company; I’m sure that is a small investment!

Mar 08 2009

Good Reading…

Category: Software DevelopmentPhil @ 11:50 am

I found the blog of Miško Hevery last week; he is an Agile Coach at Googe. Scanning his recent posts, he seemed to capture many of my own thoughts, saving me the time of actually having to put them into words! Please check out his posts, topics range from testability, to dependency injection, to Changing Developer Behavior(1)(2).

His blogroll lead me to some other great sites: The Agile Buddy and Agile Journal. I have yet to read these two posts, but they really grabbed my attention:

I’m sure that I will enjoy this read, as the second paragraph of the Requirements Come Second post states that  making the development environment more effective should be the first priority.  I have been a huge advocate of creating an effective development within teams for a long time. I typically feel that I’m on an island by myself, as most people don’t really care what or how something gets done, as long as the date is not missed.  Just getting it done has never been good enough for me.  Obviously, projects have to get done or none of us would have jobs. So the question is, why can’t we get the project and and do the right thing?  Hopefully, there will be some good wisdom in the post! Enjoy.

Mar 07 2009

Setup Hudson for Continuous Integration

Category: Continuous IntegrationPhil @ 8:24 pm

I have been experimenting with Hudson for several weeks now and I like the tool much better than Cruise Control. Not to say it does not have its issues, it generally works very really.  I have mostly had issues getting reports to show up in the project. For example, it appears that on a new project, jUnit test results will not show up until after the first completely successfully loop; kind of strange. One issue that is really frustrating me, the Testability Explorer plug-in;  I just can’t seem to get the report integrated, all of the other plug-ins have worked as expected.

There is one big difference between Cruise Control and Hudson, which I have not exactly come to grips with yet: the handling of build errors. Hudson introduces the the concept of Job Stability. Traditionally, I relied on Ant to determine the success or failure of the build, but with Hudson, it is also possible to determine the stability or health. Most plug-ins allow you to provide custom ranges to indicate the job’s stability, based on the number of errors or warnings generated by the component. On most of my projects, we did not allow any PMD or Checkstyle errors, so job stability was never much of an issue. However, there are many projects that allow Checkstyle, PMD, and even jUnit errors; this is where the stability metric can be really helpful. This would also be useful when introducing a new tool into a project and it takes time to cleanup the issues.

I found a very well written article for setting up a Hudson server.  If you are really ambitious, the article walks you through setting up a Tomcat or JBoss installation.  I originally setup Hudson using Tomcat, but most of my experimental work, I have used the built-in Servlet Container; it is a little easier and makes the frequent updates as simple as a mouse click.

Mar 03 2009

Evolution: Emacs to Eclipse

Category: Eclipse,JavaPhil @ 11:10 pm

Early in my software development career, I had the good fortune to be mentored by someone who was an Emacs guy.   After much complaining about the crazy, obscure key combinations, Emacs became my weapon of choice and I actually became pretty good at hacking LISP! I am still an Emacs guy, but only those crazy key bindings live with me today. We used to joke that Emacs was not an editor, but an operating system! I would start up Emacs in the morning; it was the only tool I needed all day. I could read my email, read news, compile code, debug, and even play games (which I never did, but thought it was cool!).

So, why is this history lesson important? It is all about focus and productivity. By allowing the developer to accomplish multiple activities within a single tool, helps them stay focused on the task which they are trying to complete. By minimizing the need to switch tools, the developer can easily toggle between the different activities, without interruption. Think about all of the different tasks that one does developing software; the most basic ones being managing problem tickets or programming assignments, interacting with the configuration management system,  writing code, and executing unit tests.  I have worked in environments where every tool operated in its own isolated world, with little or no integration to other tools. Each time I changed tasks, it was like a huge context switch: 1.) start the problem ticket tool 2.) Locate the ticket information 3.) Put the ticket in the correct state 4.) Check out the code 5.)  Edit the code 6.) Test the code  7.) Repeat selective steps.  Quite a process; it could actually involve using four different tools, just to change the code! The problem is even worse, when you are working on multiple tickets?  It can be a little more than challenging to keep the changes associated with the proper ticket.

Eclipse and other Integrated Development Environments (IDE) have evolved into the world that was once dominated by Emacs. These IDEs have literally hundreds of custom plug-ins that allow the developer to stay focused on the task at hand.  The IDE provides more than simple access to the supporting tool (via a plug-in), well designed plug-ins are able to integrate information into the current context of the developer’s activity. Plug-ins seem to take one of two integration approaches, task management or task support.  I consider task management a major activity such as ticket management or debugging. The plug-in developer would provide a big picture (multi panel) view of that task, a perspective in Eclipse terminology. From this perspective, the developer can focus on all of the activities that are relevant to that task. Task support is the alternate integration approach; this allows the developer to customize the task management perspective, by adding additional, supporting  panels; a view is Eclipse terminology. These panels show specific, subset information about different, potentially related task, such as coding problems, J2EE server status, or a selected class hierarchy.

It is this customization and integration with the task management perspective that highlights the true power of an IDE. Simple integrations include code quality plug-ins that highlight non-conforming or poorly structured code as the developer is type. One of my favorites is integrated code coverage tools; untested conditionals and statements are automatically highlighted in the editor after running a unit test.  This is the big win, focus plus productivity. Developers can now concentrate on their primary task and the output from supporting tools is seamlessly weaved into the process by the IDE. No need to switch windows or applications, everything they need is right there in front of them, exactly when they need it.

Just like many of my posts, they started off in one direction and end up in a completely different one! I really wanted to write about an interesting plug-in called Mylyn, which is a task focused interface for Eclipse.  The integration aspect of Mylyn with Eclipse is very  intriguing. Hopefully, my next post will explore how Mylyn could be used to make developers more focused and provide management with a more realistic perspective of the project’s progress.