Oct 19 2009

Spring Framework Anti-Patterns

Category: Java,Software Development,TestingPhil @ 4:40 pm

Not that I’m a big fan of the whole “Anti-Pattern” concept, but there is some subtle irony in the title of this post; some of my coworkers will actually understand. Sorry about the mystery for the rest of you, but I think this post is really relevant for those trying to use Spring for the first time. After building my first Spring-based application, I was on the fence concerning Spring’s actual value. Obviously, I made lots of mistakes in my first implementation; I think many of the following anti-patterns can typically happen on your first Spring project; my goal is to make you aware of these pitfalls, such that you can skip past the mistakes made by myself and others.

Interesting Thought Tangent?
While looking for the title to my post, I have found a little snippet that calling “Spring a direct attack on J2EE“. I thought this was a unique way of describing the Spring Framework. For sometime now, I have been on the bandwagon to simplify the development practice through Spring (Spring Core, Security and Remoting). I never really thought about Spring as another religious battle front; such as the classic emacs/vi or mac/pc wars. In essence, I was picking a side: No more J2EE! I guess it was not obvious to me before today, as I really can’t escape the J2EE container; I’m are required to deploy in a J2EE container, even though I make no use of its capabilities; a Servlet container is all most of us really need! Just an interesting thought! Agree or Not?

So where to begin, I guess the first question is: Why do you want to use Spring? What service(s) will it provide for you? I think many developers simply wanted to slam it on their resume! Let’s start with the the most obvious use of Spring, Dependency Injection (WIKI, Fowler). I think this is the hardest thing for most developers to actually understand! If you think you are just writing XML code to create your instances rather than writing Java code, you have already missed the boat. I really believe that most developers have not actually taken advantage of DI in their applications.  They just use Spring to create an alternative static (hard-wired) bean creation strategies. Just for fun, I thought I would highlight some anti-patterns I have seen used before and hopefully never again!

Create a Factory.
Factories are so 1990. This is one hurdle that some developers have a tough time understanding. They continue to think about creating objects via factories. There is NO need to ever create a factory when using Spring. Spring is essentially the a super factory, plus it is actually responsible for the life-cycle of those vended objects. There are scenarios where you need to specifically create objects that Spring manages, in a loop for example; but these are very uncommon. The proper way to solve this problem, is to create a bean that is ApplicationContextAware.  When this bean is created (by Spring itself), Spring will set the application context on this bean. Now, you have the ability to ask for any bean that Spring is managing.
Ask Spring for objects.
This is probably the toughest habit to break and is easiest pattern to get stuck in. This is especially true if you are trying to integrate Spring into an existing project and is your first attempt at Inversion of Control. You will have a much better chance understanding these constructs, if you can start from a clean slate (or at least create a bunch of samples). Spring done right, is truly is a different way of thinking. For example, when working with a context-aware Servlet, you fundamentally NEVER need to ask Spring for a single instance; this is almost a perfect example of IOC and DI.  The whole point of DI is the INJECTION. It is not about ASKING for an instance, it is about being GIVEN an instance;  throw in the IOC principle and your bean will be invoked at the RIGHT time to provide the appropriate behavior.
Domain Object DI.
This was an unusual one for me. I recently observed a project that used Spring to created multiple static Singleton factories to vend out a variety of instances, each factory with it’s own Spring context file. Needless to say, this application was far from easy to understand. To my surprise, there was even a factory specifically implemented for creating the business domain objects. Why this was done is still a mystery to me and seems to violate the true spirit of Spring. Why would I ever want to call DomainFactory.getNewWidget() or DomainFactory.getNewGadget()? The configuration had the domain beans defined in a separate Spring context file and oddly, used static methods to initialize specific properties of these new bean instances. There was no way which enabled the XML configuration to be overridden, allowing for different implementations. Worse yet, you were tied to the initialization behavior of the factory methods! Depending on your definition of a domain object, I see very little reason domain object dependency injection.
Use Ant to create Spring Configurations.
This is an interesting use of Ant in a Spring-based application. The idea here is to generate the Spring configuration and properties file from an Ant task, to specifically configure the context files to each deployment environment. They values are essentially baked into the application and unchangeable.  It is bad enough to substitute property file values, but some developers actually substitute bean name references. This might be a little hard to follow, but the Spring XML configuration would define two different implementations of the same basic function, each with its own name. When the application is built, the Ant tasks would substitute ref=”bean1” for some environments and ref=”bean2” for others; everywhere that implementation was referenced in the configuration files. Call me crazy, but I don’t think this what the Spring creators ever intended!

As I have developed a better understanding of Spring and how to take advantage of it features, we have evolved our Spring applications to a state of “environmental awareness”. Using this solution, all of the context and property files are packaged into the application, for all known environments at build time. This enables us to build the deployment image (jar/war file) one time, using simple Ant tasks, no more templates, no more substitutions, no complexity. At execution time, Spring is now intelligent and can determine it’s execution environment; we can use specific property files for specific environments, as well as use alternate implementations (via DI) for each environment. This is where DI really pays off, allowing us to use the appropriate objects, not just properties, but objects for each environment! No code changes, no crazy Ant tasks, you build once and deploy anywhere. I think this is actually pretty slick and what Spring (core functionality) is all about.

I do have to put in a plug in for testability. Taking advantage of jUnit 4.x annotations and Spring integration, makes unit testing a snap. Using the @Resouce annotation to inject everything (instances) you need into your unit test is amazingly elegant; unit testing could not be made any easier. This approach allows us to have highly controlled, customizable execution environments; we can seamlessly manage unique configurations for local development, continuous integration, as well as any actual deployment environment almost magically.

https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png


Sep 09 2009

Hudson Task Scanner Plug-in

Category: Continuous Integration,Software DevelopmentPhil @ 8:21 pm

Ok, that is just about enough Hudson for this month, unless I get fired up about Selenium and try out the Hudson integration! I did find one more useful plug-in this week, the Task Scanner.  Because I tend to check in code early and often, my code is typically in an incomplete state; you will tend see a lot of TODO or FIXME comments in the code. Managing these task inside of Eclipse is easy enough, but I thought it might be interesting to also monitor them on Hudson’s dashboard.

The plug-in is very easy to configure and allows you to specify thresholds on the number of open tasks. Like most other plug-ins, it generates configurable trend graphs and provides a convenient task browsing screen.

Here was an interesting blog post that shows how you could use the plug-in to determine how messy your SVN merges will be… kind of cool..

https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png


Sep 08 2009

Continuous Deployment with Hudson…

Since I was looking through the Hudson plug-in list this week, I decided to try a few more of them. We currently have Hudson continuously deploying a simple web application to a Tomcat instance. Since both Hudson and Tomcat are running on the same server, it was pretty simple to have Tomcat point to the WAR file generated during the build process. Tomcat notices the new WAR file and automatically redeploys the application. Pretty simple and pretty handy; this is about a 100 times easier and faster than dealing with our standard Weblogic deployment process!

So, how do you deploy your application if Hudson is running on a different machine than your Tomcat instance? Fortunately, Hudson provides a plug-in to solve this problem, simply named the Deploy Plug-in. It supports a variety of containers, taking advantage of the Cargo Framework. Can’t get much easier than this, just point to your WAR file and fill in the container information. That’s it! I tried it out today and it worked perfectly. This was a much cleaner approach than adding the logic to the build process.

Having the current code set continuously deployed is a great benefit all by itself. This helps ensure that your application is built correctly and is deployable; How many times have you forgot to add that new JAR file to the WAR, only to discover the failure several days later? Continual deployment also makes the application constantly available to the analysis and test teams. This allows everyone to see the current state of development and bounce through the application to see how it will ultimately work. This early exposure also gives developers a change to resolve issues much earlier in the development cycle. We can achieve more value by adding the integration or system testing to the process as well… Just think how much we could reduce our development and testing cycles with this level of testing?

I have been starting to read about Selenium on the Hudson mailing list and found this interesting blog by Ben Rometsch on “Easy Automated Testing with Hudson and Selenium“. I really liked this picture from the article (hopefully it will spark your interest too!) and truly believe this is what we should be working towards… Maybe this can be one of my next educational adventures!!!

https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png


Sep 07 2009

Why would anyone still be using IE6?

Category: Software DevelopmentPhil @ 7:54 pm

I have been working on a web application and was required to make the  work with IE6.  We were doing nothing fancy, but the application had numerous issues rendering in IE6. First, we implemented the presentation using  JSF and Facelets. All of our content was in XHTML documents, which IE6 apparently has no idea how to deal with! After doing some research,  we found a simple solution to the problem;  simply by removing the XML prologue, IE6 could now display the pages. We also had problems with some of the CommandLinks and CommandButtons not posting back to the server, by playing with different tag combinations, we finally got it working all of the browsers. What a pain in the behind!

I just cannot image how anyone could actually still be using IE6… Unfortunately, It is still the corporate standard where I work too. Somehow, I have no idea how, my Internet Explorer magically upgraded itself to IE7 a couple of years ago, and then again to IE8. It is amazing how that happened, and even more amazingly, everything still works fine! How do people surf without tabs? That is the question I really want answered! I really don’t use IE anyway, as the rich supply of plug-ins makes Firefox the best option out there, in my humble opinion.

Anyway, one of my friends forwarded me a couple of posts that I just had to share, the best one being “IE6 must die“! You just really have to love what gets people fired up… People are even spending their time building Javascript libraries and  WordPress plug-ins, to encourage people to upgrade. I found some of these URLs rather funny…

  • http://isie6dead.com/
  • http://idroppedie6.com/
  • http://www.ie6nomore.com/
  • http://ie6update.com/

The most interesting post was about “IE6 Offenders“. I checked out the site, ieoffenders.com, which highlighted organizations that were still using IE6. It reported the number of users and the reason why they had not upgraded. In searching for the site tonight, the guy actually took his site down! I guess the guy must have caught some flack? I actually thought it was quite interesting!

I also like some of the banners, example below, that people are adding to their sites… I found this post tonight, The browser that will not die. It says that Microsoft will support IE6 until at least, 2014! That is amazingly sad!

https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png


Sep 07 2009

Hudson Plug-in for Custom Documentation, integrating an Ivy Report

Category: Continuous Integration,Software DevelopmentPhil @ 7:14 am

One of the best things about the Hudson Continuous Integration platform, is the amount of activity. People are constantly fixing problems and adding new functionality. I’m sure that some would say this is a bad thing, but even with the frequent releases, it is amazingly stable.

I have been moving projects to the Ivy Dependency Manager for the last couple of years. We have established a multi-repository solution, that allows us to control access to the repositories without impeding the progress of the development teams.  Introducing multiple repositories and transitive dependencies takes a little getting used to, so I have been adding an ivy.report task to each of the project’s build scripts.  Here is an example of Ivy dependency report. I think it is quite handy on multiple levels, but the primary benefit is displaying the dependencies of each component. Side benefits include showing the different types licenses used by each component, kind of nice for corporate environments. The report also shows where the components are resolved from, such as the legacy or sandbox repositories (indicating that you might need to do some promotion or upgrade work before releasing the project).

My problem was, how to tie the Ivy dependency report into my continuous integration process. Building the report was easy, but how could I make it available on the main Hudson project page? Unfortunately, there was no plug-in to make this happen; I was actually thinking about writing my own plug-in this summer, but never quite got around to it (or blogging for that matter!) Fortunately, this appears to have been a common need and someone was nice enough to release a really nice plug-in this past July, called DocLinks. It allows you to add multiple links to items generated by the build process. The process could not be simpler.  The links will automatically show up on your main project page, but will not be active until the next build completes.

This also solved a problem that I have with the Testability Explorer plug-in. I have about a 50% success rate with this plug-in. I am able to generate both the XML and HTML files for all projects without issue, however, the Hudson Plug-in reports nothing for half of the projects; I have yet to figure out the pattern. Anyway, the DocLinks plug-in now lets me add a link to the Testability Explorer HTML report, so that everyone can always see the results, independent of the Hudson integration.

Suggestion!There is one area that the Hudson community could be a little more helpful. The Hudson tool is very good about showing you when updates are available for itself or installed plug-ins. However, there appears to be now good process for discovering new plug-ins. I even follow the Hudson mailing lists, and there are no notices about new or updated components; the only way I know about upgrades is to click the “configure” link every so often, when I actually remember! To find out about new plug-ins is even worse, I have to look at the Plug-in page, and notice if there is something new. There are so many plug-ins now, it is almost impossible to know if something has been added. Hopefully, they will add some kind of search for recently added or updated feature to their page; I’m sure that I’m not the only one that would appreciated it!

https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png


Jun 10 2009

Hudson Continuous Integration and Eclipse Update Sites

I was looking for a development build of the IvyDE plug-in for Eclipse tonight and I stumbled across the Apache Hudson server. You have to check this out. Pretty cool, Hudson running over 150 jobs!  Their build farm consist of both Solaris and Ubuntu servers. What I found most interesting was how they used Hudson for creating an Eclipse update site. I subscribe to the Hudson mailing list, summarized here, to follow what is going on with the project. People seem to use Hudson for a variety of different purposes, such as a cross platform testing tool and even as a deployment tool. I have only been able to get people interested in Hudson as a “Cruise Control” replacement…. but at least it is a good start!

This site also demonstrates how multiple projects can be linked together, triggering builds of dependent downstream projects.  I’m not exactly sure how these jobs were assembled, but at a high-level, Ivy and IvyDE each have its own build loop (job). Each job is responsible for creating build artifacts. After a successful build of either job, a third job is triggered which creates/updates and Eclipse update site.  Pretty slick. (As any FYI, the dev plug-in looks better, new presentation and several new new options)

Another small plug for Continuous Integration. We recently set up something kind of cool (trivial, but cool)! As we continue trying to convince the powers that be, to move away from Weblogic towards Tomcat (or even JBoss), we found a really nice little feature. Since Tomcat and JBoss can hot deploy a new version of the WAR file, just by noticing a difference on the file system, combined with the fact that we can spin up a Tomcat instance in about 2 minutes, we actually created an auto deployment mechanism.  Our Weblogic environment seems so cumbersome in comparison…. We could have built the WAR and deployed in in our build loop, but that would have minimally added another 25 minutes to the loop (let’s just say we have an interesting deployment process). With Tomcat, we did a one time deployment to the location of the WAR file inside the Hudson workspace. Now, with each series of check-ins, we build the WAR as normal and Tomcat instantly deploys it, in seconds! If we were a little more sophisticated, we would write some jUnits to bang against it, making sure we did not break anything or create any new dependencies.  Maybe I will throw that in during my next project!!! I know this integration was possible before (with Weblogic), but no way as simple!  Even better Continuous Integration… on my last project I had Ant spin up an embedded Jetty web server, ran my jUnits against the Clover instrumented code, and finally had Ant shut down Jetty…. Pretty slick, from a code coverage collection perspective…

https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png


May 24 2009

Still on the Jetty Bandwagon… Go Embedded?

Category: Eclipse,Software DevelopmentPhil @ 2:23 pm

image I was searching for some information on using the Eclipse IvyDE plug-in and found a post that reminded me of some work I did last summer. For the most part, I’m happy with the way the IvyDE plug-in works, but sometimes you have to be a little tolerant of its usability issues. Every so often, IvyDE seems to forget about my configuration and refuses to add libraries to the class path. My only option is to remove Ivy Dependency Manager and re-add it to the class path.  One problem that I’ve been unable to solve, is the integration of an Eclipse Web Tools Platform (WTP) project and Ivy; I cannot get the Ivy resolved libraries added into a “Dynamic Web Project”, making them available to the Tomcat. I have tried a dozen different configurations and settings with no success. Hopefully, I will get this working in the near future. If anyone has this working, please give me some tips!

The real thought behind this post was echoed in this post by Daniel Spiewak, So Long WTP, Embedded Jetty For Me. I was really fired up about Jetty last summer. When compared to my experience developing Weblogic-based applications, the utter simplicity and flexibility of Jetty was amazing. I was working on a custom Servlet-based web service framework and decided to do my development with Jetty. I created a very small Java program that configured Jetty to deploy my Servlet, in about 20 lines of code! The best part is there was no special Eclipse project type, no special environment, and zero deployment work; making debugging and testing especially easy.  I never used the word embedded when I talked about Jetty, but that was exactly what I wanted. I always deployed in the Weblogic container because of our corporate standards. I should have explored this a little more, further highlighting the point that Weblogic and all of its complexity is truly unnecessary.

Following the BEA Bible, you end up with specific machines, specific file systems, specific networking, and hardwired clusters. Combine that with an environment that is highly controlled, it can take a dozen change control tickets and multiple days to bring up a new instance. Not exactly what I call agile or responsive. How cool is would it be to just drop a WAR file on a machine (any available machine), launch java to bring up a web server, add it to the load balancer and instantly expand your processing capacity? Pretty powerful idea, I think.

Embedding a web server is a good idea when building a web-based application, both for development and operational reasons. A good example is the Hudson Continuous Integration tool. Hudson is distributed as a single WAR file that you can pop in any Servlet container; but it also includes an embedded web server. This approach is becoming more and more common, as several open-source and commercial products are shipping their products with embedded web servers. The real benefit is flexibility. It literally took me seconds to get Hudson up and running; no messing around with Tomcat, no configuring security realms, no anything! It allowed me to focus on Hudson, rather than the environment to make it work. Down the road, if I want to deploy Hudson in the standard application server environment, it should be an easy switch, This is quite different from my recent experience installing a few open-source code review tools; installing multiple Python packages, configure an Apache web server, setting up database servers, etc. I don’t think I have to say much more; the better choice is pretty obvious.

image It is pretty hard to beat the Eclipse WTP – Apache Tomcat integration as a development environment, but directly embedding Jetty into your environment (no WTP) would make the environment even simpler. I have been trying to encourage teammates to develop from a container independent perspective, demonstrating higher productivity with no additional risk to the project. If we can move from large, monolithic applications to smaller, self contained services, the attractiveness of Jetty goes way up. The flexibility of running a moderately sized collection of small, independent services, verses a small tightly coupled clusters of monoliths, has to be an attractive architectural vision.

Unfortunately, many corporations don’t eagerly embrace open-source software solutions, preferring to pay for name recognition and guaranteed support. I personally feel that Jetty is a viable option, but it seems to be relatively unknown to most developers. Apache Tomcat and JBoss have a much higher exposure level, giving them more perceived viability. Jetty is no slouch, check out the Powered by Jetty page. Hopefully, this and Daniel’s post demonstrate there are alternatives to the J2EE Silver Bullet, and that we don’t really need to do everything by the J2EE book. There are a lot of simple alternatives out there for us to take advantage of, all we have to do is keep an open mind!

https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png


May 21 2009

Unbelievably Simple Java Zip Example… And What About Reuse?

Category: Java,Software DevelopmentPhil @ 5:15 pm

I was working on a typical web portal application today and needed to support a multi-file download. Luckily, I had some code which I had used several times before. I started to tweak it for my specific requirements, realizing it was such a bad fit, I quickly tossed it out. My previous implementation worked with physical files and was tightly coupled to a legacy framework. This time, the files were already in memory. I was hoping to find some code that allowed me to build the zip file in memory, rather than dealing with the file system and temporary files. Since the portal user could only download a limited number of files at one time and the files were very small, I didn’t have to worry about memory constraints.

Bottom line, I found the perfect piece of code on Trip over IT. Within 15 minutes, my application was downloading a sweet little zip file! The code was so simple and concise, I had to share it. I will definitely put the code in my tool box of handy snippets. 

Here is a provocative thought: What is the point of a reusable library? Do we focus on reuse because we continually over-design and over-build our software, thinking that we are saving time by building these reusable libraries? Are there alternatives? What about simple, small, and disposable code? Don’t be so attached to your masterpiece. I promise, it will not look so elegant tomorrow! All you really need is Google to find some code to start with. I found this little snippet of code, changed it to fit my problem, and I was done within a few minutes, no support and minimal overhead. Maybe the real reuse is my memory and ability to relocate this snippet in the future… Or am I just crazy?

https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png


May 05 2009

Problem Detection, Recovery, and Prevention…

Category: Software DevelopmentPhil @ 7:51 pm

I have spent a lot of time discussing software principals and architecture over that last couple of years, and the subject of error handling and recovery always seems to always rear its ugly head. Most recently, in talking about moving from monolithic application deployments to a distributed, web service orientation, many developers seem to immediately go into paranoia mode. They only see two choices for working with distributed systems: the two phase commit or store everything approach, such that it can be resubmitted when the service is available. There is one other option which we cannot forget about, used by many applications regardless of their deployment, the traditional “logging, debug, retry” logic. Unfortunately, we developers like to add this code at the end of the development cycle, and usually as an after thought; but more on that later!

Let’s talk reality… When was the last time one of your systems actually failed due to some type of hardware or service failure? I honestly don’t recall a single failure. Considering today’s redundant hardware technologies, I believe it would be pretty hard to have a total system failure; provided we move away from those monolithic application deployments! I agree that some kind of isolation layer (protection) is required for systems outside of our operational control, such as remote web service. But, our internal network and applications should be a different discussion, one that assumes 100% availability. I realize that some systems can experience performance degradations, but that is also a blog for another day, not something that we should be “protecting” ourselves from. I believe that most of the bugs we encounter in the production environment are actually caused by incorrect or misunderstood requirements, and the the occasional developer contribution  (bad code, insufficient testing, etc.), not by a system failure. Another interesting finding, many of the bugs are actually found in the logging and monitoring (prevention) code, rather than the true business logic. What or who are we actually protecting ourself from?

One of our top goals should be to produce the smallest, simplest amount of code possible to implement the solution. I feel there are two (2) fundamental problem types:

  • Show Stoppers. A show stopper is a dependency in which your system cannot perform its primary function. For example if you created a purchasing system that relied on an external pricing service, your system cannot function without live prices. Assuming that this pricing service is internal to your company, there is no need to protect your service/application from this scenario. The actual risk of this type of failure is not worth the amount of effort required to implement and maintain a possible fall-back solution.
  • Recoverable errors. Recoverable errors are software problems that can be captured and dealt with later, at an appropriate time. Examples might include a failed database update or the failure to update a logging service. If the transaction can be still be completed, without the end user being impacted, this is an acceptable approach. The assumption is that we don’t have tens or hundreds of this issues to deal with, as they can and will happen over time. We just need to ensure that we have a method for recovering from the problems, ensuring that the system can be put back in the proper state of integrity.

I think the  relevance of this post is to ensure that “software issues” can be detected. If we can assume that collaborating internal systems are always available, we can eliminate most of the code to manage these dependencies. We should focus on a  common design/implementation/strategy that can ensure the detection of issues. Looking across existing applications, it seems that each one has its own unique way of capturing and monitoring problems; with each team spending significant design and implementation effort attempting to create a robust method. Ironically, with all of the work developers do, many of these systems have problems that go undetected for hours,days and even weeks, without anyone ever knowing that an “issue” happened. It appears that we should spend more effort on a common monitoring strategy to detect problems; one that makes us aware of problems within minutes, not hours of the problem occurrence.

The second part of this approach is the ability to recover. There are probably some other important concepts that I have in my head, not part of this blog, which actually makes this all possible. I think for a system to be recoverable, it should  be compartmentalized; such as smaller databases, each with its own functional domain areas; rather than a single, gigantic database with full referential integrity. By breaking the system (services and database) into smaller, self-contained “mini” environments, each detected failure has a smaller overall impact on the system, and ultimately makes it easier to recover, as the impact is now isolated to a distinct area of the system.

Hopefully, with ability to detect problems earlier, minimizing their impact, and simplifying the recovery process, should make the prevention of the problem easier to address. With smaller, simpler systems (services), it will be easier to locate and correct the offending code (because there is less code and data coupling), creating less risky changes which are easier to test and validate. Sounds like there is an overall win-win architecture in here somewhere… I sure hope I get to build a system with these principals and goals some day!!!!!

https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png


Apr 12 2009

Angry Coder Mode

Category: Software DevelopmentPhil @ 1:46 pm

When you start researching the code review process, you run across a lot of interesting blog postings, from the process perspective to the emotional aspects.  I found this one, Working with your angry coder, and thought it was worth sharing. I stole the picture from his site too; both the kids and I thought it was funny!

Over that last year, I have definitely tried to be more aware of the comments I make. However, I do remember saying this phrase numerous times in the past “Whoever wrote this code should be shot!”.  I too apologize for to all of the people I’m made cry over the years! I believe that most of these comments were directed at the code and never the original author; I don’t think I am that angry!  I do have a passion and strive for perfection (at least the best job that I can do, given the scenario), which makes it very easy for me to be perceived as an angry coder. I used to think that I was the only one who cared about developing  software quality. Someone told me that most developers are generally passionate about something in the process, usually just not from the same perceptive as me. So in reality, everyone is trying to do the best they can, we are all just coming at it from different directions. We just have to find a way to work more smoothly as a team. As one of the comments said, it is all about understanding each other. To bring this back to the code review process, I think it is all about finding bugs and learning from each other.  If we truly practiced collective code ownership, maybe none of us would take the comments so personally and we would have less anger!

https://www.beilers.com/wp-content/plugins/sociofluid/images/digg_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/reddit_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/dzone_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/delicious_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blinklist_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/blogmarks_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/google_48.png https://www.beilers.com/wp-content/plugins/sociofluid/images/facebook_48.png


« Previous PageNext Page »