At Agile and Development conferences this year some sessions have been looking at how to improve Agile techniques using failed projects as examples. Discussing some of these issues and reading blogs and articles it appears that a significant percentage of Scrum projects have failed and it is not quite the silver bullet many expected it to be.In this weeks video blog I asked some of the leading Scrum experts (Dan Rawsthorne, Alan Shalloway, Sanjiv Augustine and Ken Pugh) what they thought the most common reasons for failing Scrum projects is and what advice they can give to resolve this. In their opinion, it appears management and managing the Product Owner are currently areas to improve upon.I believe that many organizations that practice Agile techniques do not strictly follow Scrum, XP, Crystal or any of the other flavors of Agile directly, but instead adopt a hybrid ‘Agile-esque’ practice using parts of these methodologies that they feel work best for their projects. This was reiterated at Bob Martin’s keynote at Agile 2008 where he also stated that ‘Agile’ will be the term used as a practice rather than the umbrella for the individual practices such as Scrum, XP etc. Will this see an increase in Project success? Only time will tell.One opposing theory is; teams will adopt whatever practices they can implement easily and quickly to comply with senior management edicts so they can claim their organizations are ‘Agile’ (whatever that means to them?!?). The reality is, underneath they will really be working in a Waterfall manner with little change to project success.Such proponents of this theory will state that only by following the practices strictly, can you achieve project success repeatedly (as demonstrated by Menlo Innovations).

Fred Brooks’s law of ‘adding manpower to a late software project makes it later‘ is one most of us have tried to prove wrong…….and failed!I was at Agile 2008 and saw an interesting session, “Breaking Brooks’s Law” from Menlo Innovations, a Michigan based Java development company. They claimed to disprove this law and demonstrated their working environment and techniques that allowed them to do so.Although the presentation was only 45 minutes, we were in the room for almost 2 hours asking questions to determine how robust their techniques were, and to gain more insight into the conditions developers work under.Menlo’s results are based on a 3 year project that the customer had a deadline to demonstrate at a show. More features were required for the show than currently in the plan. So rather than re-prioritize, Menlo decided to add more developers to attempt to complete the work. They managed to complete the Project on time with all added functionality.The environment at Menlo is quite unique. All developers are co-located in the same large room (no offices or cubes) and pair program 100% of the time - they follow strict XP practices. A scheduling team determines which projects developers work on and who they pair with on a weekly basis. So developers work with different team members and possibly different projects every week.Also, as part of the contract, the customer comes to Menlo every week to prioritize the work for the next sprint.These techniques may appear somewhat draconian (100% paring for example). I managed to catch up with the team and interview them to discuss this project further, bug rates, staff attrition rates and how Project Managers can push the message of pairing to Senior Managements/Directors (see video).I thoroughly enjoyed talking with the team from Menlo and they invite anyone passing by to stop in and take a look at how they operate. They also have an interview process which involves a large number of candidates performing a number of tasks including Pair Programming, with an appointment you can observe this too. A detailed paper about their techniques and contact details are here.

Chuck Allison’s interesting article “Software: Use At Your Own Risk”, (Better Software Magazine, July/August 2008, p. 9) discusses the risk inherent in software licenses compared to other warranties, reasons for the lack of quality in software and lists a sample of the code of ethics of the Association of Computer Machinery.Here are some reasons why I believe that the Government will not take effective measures to enforce more liability on software vendors in the short or long term (within the next 10 years).1. Consumers are instant gratification junkies. Consumers want things ‘now’ and are only mildly disappointed when they have small failings, so they do not officially complain so much as in other industries. We only have to look at the lines outside Apple Stores for the new iPhone to realize this. Many people were not even concerned that some phones had activation issues, as they understood that Apple would eventually fix this.2. No physical boundaries. License issues will differ in different countries and trying to enforce this will be an administrative and legal nightmare. Also, software can be downloaded from the Internet from anywhere. Even the attempted Chinese firewall was worked around within a day3. It is understood that software is never defect free. Maybe this is not the right attitude, however it is one we have ingrained into ourselves over the last 20 years as both users and developers. Microsoft’s famous Blue Screen of Death was frustrating for developers, but after a quick reboot, work continued – it was accepted (I say “accepted” as it was the most common OS used and there was no mass exodus to UNIX, or later Linux). Would the same attitude apply if your car broke down several times on your way to work?4. Software organizations do not learn. Many articles quote from references such as Brook’s ‘Mythical Man Month’. This book was written 30 years ago and many of the failings in Software Development back then still exist today. For example adding people to late running projects only makes them later.5. Perceived insignificant importance. We know the cost of failed software is in the billions, but other things in the public eye have happened to constantly keep this from being a primary concern. For example, The Gulf War, 9/11, Presidential elections, rising fuel prices and the current Fannie & Freddie crisis. Although software is involved everywhere the quality is not considered a public priority.One of the ethics Allison lists is “Contribute to society and human well-being”. Unfortunately, for Governments to pass legislation of the sort suggested by Allison, a monumental catastrophe will have to happen. We saw something along these lines with the introduction of Sarbanes-Oxley compliance following the Enron scandal. However, even if a plane were to crash due to software error, it is unlikely that even loss of life would be enough for the Government to pursue measures. In my opinion, the outcome would be that the airline and the manufacturer would have lawsuits filed against them, while the FAA and software producer would, at best, experience some bad press.

Two of our recent blogs; The Misnomer of Best Practices and The Dreyfus Model are tied together nicely in this episode of Enerjy.tv by way of a conversation I had with Dan North of Thoughtworks.

Dan regularly speaks and blogs about these topics and gives his views on where so called Best Practices should be in an organization and how they are used depending on your position on the Dreyfus Model. Dan gives examples of how developers use Best Practices when starting a new skill or position within a company and how these practices and rules diminish in importance and become a hindrance once skills levels rise towards the Expert level.

A large part of improving the quality of software is knowledge sharing (indeed it is one of the basic tenets of Lean). In most organizations when a bug is submitted it is usually delegated to the developer who wrote the original code.

Paul Pagel’s blog discusses how his team tackles bugs in development and production.

I especially liked:

“[The Production Support Resource] seems a lot like a silo to me. Everyone should be able to do production support on any system. I should have to, because it is a perspective of the system that is important to have. In response to this, we came up with a system of triage. Each day of the week is assigned to a specific developer. If a support item comes up, it is the job of the triage developer to respond to the client/customer we are working on it.”

This is such a simple way of sharing knowledge and keeping tedium away from a designated ‘support engineer’. However, managers may originally question this for several reasons including:

1) A faster response time will occur if the original author fixes the bug. – This is short-term instant gratification. After a while the difference in time for a bug fix between developers (of similar skill level) will most likely be negligible.

2) The author is an expert in that area. – This allows others to increase their skills knowledge, benefiting the entire team. What happens if the original author is out sick or even leaves the company?

3) We’ve always had a designated Support Engineer, everyone knows who to go to. – This is a Command and Control’ management style and in most cases leads to lower team results. Managing by Facilitation and allowing the team to assign work themselves has proved more motivational to the team and resulted in higher quality artifacts.

I strongly believe that this method is more beneficial to the team, evolving individual’s skills and ultimately benefits the customer.

The Dreyfus Model shows the 5 stages of skill acquisition possible through experience of a subject.

Created by Hubert and Stuart Dreyfus in 1980 while researching AI, the model was popularized by Dr Patricia Benner in the mid Eighties in her work on the Nursing Crisis in the US. Over the last few years this model has been resurrected in the Software Industry, primarily by Andy Hunt of Pragmatic Programmer fame.

The graph shows that in the Software Industry (as with all subjects), most people achieve a level of ‘Absolute Beginner’ or ‘Competent’, with few moving to ‘Proficient’ and fewer to ‘Expert’. The graph also reflects that as you move up the model the need for rules and best practices diminishes to the point where they actually become a hindrance.

With the huge shortfall in supply of Software Developers combined with the quality issues our industry faces, surely it is better to keep the small percentage of people at the top of this model rather than move them on to management? (As so often happens.)

I caught up with Andy Hunt at the recent Better Software Conference to ask him to discuss this and explain further how this model applies to mentoring team members and where ‘best practices’ fit.

After reading several blogs and comments on the demise of Java, I felt compelled to post some interesting facts.

Only a small percentage of developers go to conferences like JavaOne. These people constitute the majority of those who read, write and comment on blog sites and they are the first-adopters of new technologies and languages. These people are enthusiastic evangelists of new and cool technologies fueling, for example, the rise of Dynamic languages (which I applaud). But these same folks can be vocal complainers, ignoring possible work-arounds of existing features in languages such as Java, and predicting the death of Java. For many developers, these dire predictions can inspire fear for their jobs, compelling some to prematurely drop Java, further perpetuating the rumor that Java is indeed dying.

Maybe there are too many frameworks? Maybe Generics were an overly complex feature to introduce and maybe Closures will be too? It is hard for Sun to sustain the velocity of new features every year, and maybe this is a good thing that they do not. – It gives people breathing space.

It is impressive that Java has been consistently popular for over 12 years (Ruby was created in 1995, but only really became popular once David Heinemeier Hansson released Ruby on Rails in 2004). In our industry it’s impossible to predict how something will be used in a decade and maybe we shouldn’t try. Have Sun got carried away with the popularity of Java and tried to make it into something it wasn’t designed to be, creating an overly complex beast? Are these proponents of Java-Death simply Java gurus that who are just seeking perfection?

Do I think Java is dying? – No way! Each programming language goes through changes: it’s all about evolution. [Most] people do not program in Assembler anymore. The next generation of languages will build on the limitations (mistakes?) of the past and be more suited to the hardware and designs of the future. For example, concurrent programming is very difficult to manage in Java so maybe laguages such as Fortress will be indicative of things to come. With multi-core processor machines the norm, the industry will demand that future software development take better advantage of this hardware advance.

So here are a few facts:

• The JavaOne Program Office announced that there were around 15,000 attendees at JavaOne this year. (There was also a lot of negative discussion regarding JavaOne, but this criticism was really about the show rather than Java–see my comment on Jason Lee’s: Postmortem on JavaOne 2008)

• One of the most impressive announcements at JavaOne was that Sun have made a 68% performance increase running Java 5.0 against Java 6 update 10. On the same hardware (showing their commitment, at least, to the JRE)

• Released only last month, Effective Java 2^nd Ed by Josh Bloch is ranked #291 of TOTAL book sales at amazon.com and #8 in the Computers & Internet sub-section

• Without too much analysis (i.e. just typing in the flowing keywords in Dice.com) current vacancies for the following are:

Java     15,337

Ruby    818

C++      7,633

C#        7728

Python  1,395

Perl     5,463

Cobol  1,131

Assembler 196

- I was just interested in these last two

Note that I haven’t even commented on the millions (billions?) of dollars organizations have spent on applications written in Java. Are they about to drop and rewrite these applications in the next generation of languages?

I’ll be speaking at Sys-Con’s Real World Java one-day seminar in New York City next Monday, August 13. Despite the fact that it’s August, it looks like they have lined up an interesting panel of speakers for the day, including sessions by Yakov Fain on using Flex with Java (that’ll be interesting to me) as well as sessions by Sun on Java 6.0 and a panel session on Java 7.0.

The title of my session is “Code Quality: Pay Now or Pay Later”, and I will be talking about why code quality is an issue that many sweep under the rug, why we should care deeply about it, as well as some practical steps to start down that path. Our CTO Mark Dixon will be there with me, so if you’re in the NYC area, come say hello!

Registration is discounted by $500 through the end of this week, and you can register here.

Heard this before?

A couple of weeks or so after launch, I thought it might be fun to scout around and find out whether iPhone users are experiencing significant software problems with the device or not. I haven’t traded my Blackberry for an iPhone (yet), but a couple of folks in the office have them and are all glassy eyed and in love.

It’s a slick device, there’s no question about it, and the software appears to be well executed. Of course there have been some early glitches, but these all seem relatively minor (and shrink even more in comparison to some of the problems experienced with the software on the Nokia N95 - probably the iPhone’s closest competitor).

The iPhone story that interested me the most though, was Bubba Murarka’s tale of his service experience with Apple. Here’s someone who clearly likes the product, but the whole experience is let down by the support model. I had a similar experience when I returned my malfunctioning Macbook 17 days after I purchased it. If the problem had arisen within 14 days, the Apple “Genius” happily told me, they would replace the device with no questions asked. But because it was now 17 days old, they would have to repair it. I won’t bore you with details of the story, other than to say that ultimately I was left with no laptop for more than 2 weeks.

Ultimately, it sounds like the iPhone launch has been a success. Apple deserves to do well, in my opinion, on the basis of their products, which are well designed and well executed. But I hope that the Genius Bar support model, along with some lame warranty policies, is not their undoing.