It’s Friday, so it must be Glitch Watch day. In another example of users not reporting software problems that they encounter, an elderly man in Melbourne, Australia, has been ordered to repay $12m that found its way into his business banking account. Seems that Victor Ollis had a facility to transfer funds from his personal account into his business account if it reached a certain threshold. Ollis discovered that he could continue to draw funds from the business account, and it would continue to be topped up, even although he had insufficient funds in his personal account to honor the transfers. Three years and $12m later, Westpac bank discovered the problem.

Ollis, who apparently has an unidentified disease and has been given 2 years to live, has told the bank they will not see a cent of their money…

Last week, I had the opportunity to have lunch with Zach Gemignani, co-founder of Juice Analytics. Juice is mainly focused on business analytics, but their interest (and, clearly, talent) in the area of data visualization had me add their corporate blog to my Google Reader a while ago.

If you’ve had any interaction with Enerjy in the past, you’ll know that we are big on metrics. And, after sitting with Zach and eating lunch while looking out the window at Motif Number 1, I thought about his post back in July about the importance of choosing the right metric. This is a great post that really lays out a process for thinking about what metrics to track, something that we don’t spend nearly enough time on, in my opinion.

We often find ourselves with an uphill battle when it comes to code quality metrics, simply because people’s experience of tracking the wrong metrics has given the whole subject a bad name. The most often quoted example in our world is measuring lines of code per unit time. But there are others. I once worked in a development shop where we were actually bonused on the number of bugs we fixed in unit time. On our own code. So, in a sense, the more bugs we introduced - then found and fixed - the more money we made.

Central to Zach’s piece about metrics are what he calls the four dimensions of a good metric:

1. People should be able to easily recognize what the metric means;
2. The method of calculating the metric should be shared and easy to understand;
3. The data should be derived relatively easily from a trusted source;
4. It should be actionable, i.e. when the metric goes off target, it should be understood what actions are needed to correct it.

Good, well thought out stuff, and to my mind, the very act of thinking through the implementation of a metric is where many organizations will get the most value.

Any readers drive the Maserati Quattroporte (base list price $122,400)? If so, you may be aware of the recall that Maserati issued for that car this week, owing to a software problem that causes the traction and stability control system to shut down when it thinks the battery is low on power. The recall only affected 718 cars, although we’re not sure what that represents in terms of a percentage of the number of Quattroportes on the road…

Some Tennesseeans may have planned to spend their lottery winnings on a Maserati, when unbeknown to them, they had zero chance of winning at all. Just 23 days after switching from a good old ping pong ball-based system to a shiny new software-based system, one of the lottery players noticed that the same digit never showed up more than once in any winning number. So, if your ticket had duplicate digits in the number, it had zero chance of winning. The problem’s now been fixed, and the lottery will pay refunds - as long as you can produce a losing ticket with duplicate numbers. Personally, I always keep all my losing lottery tickets, just in case I ever need them again.

Finally, we now have proof that you can’t rely on users to point out software errors - even when they detect the fault. The MBTA (or the “T” as it’s known round these parts) is a Greater Boston institution. But it seems like some people have been riding the T for free after a software problem mistakenly caused monthly accounts to be marked as “paid” when they weren’t. An unidentified passenger said she had been using her “golden card” for six months, without paying a dime. For some reason, she never got around to reporting the problem.

There are many tools available to provide developers with metrics to help determine the quality of the code they are writing. However, there are situations where using tools in isolation, or not understanding the subtleties of the data they are giving you can result in indications that your code quality is higher than it actually is. This is especially true when tools are run over large bodies of legacy code.

This can be best demonstrated using an example:

JDepend is a free tool that ‘traverses Java class file directories and generates design quality metrics for each Java package’ (there is also a plug-in for Eclipse users). It is a good tool that helps detect smells such as high coupling and circular dependencies. No matter how good the tool is though, its results can be misinterpreted.

One of the features of JDepend is to show a graph that represents each package’s ‘Distance from the main sequence’. This metric is an indicator of the package’s balance between abstractness and stability and can help you get a sense of the maintainability of the package in the future. The values range from 0-1, with 0 representing highly unstable packages and 1 representing stable packages. (For more explanation of these types of metrics, see Andy Glover’s article on coupling metrics here.)

The screenshot below shows a selection of packages from the Jedit project (used for example purposes only). The packages with green dots on the graph show a close balance between abstractness and stability, while the one black dot represents a more unstable package that I may want to investigate further.

Narrowing it down, I quickly find the offending package and to appease my boss (who wants all code to have green dots) I can game the results of JDepend by simply adding an empty Interface to the package, thereby increasing its abstractness value which results in the package moving closer to the Main Sequence:

Adding an empty interface to a package just to satisfy a metric adds redundant code, looks ugly and is, well, plain stupid. I don’t believe anyone would actually do this to satisfy a green dot on a graph.

The reason I use this example is this. Imagine running a tool like JDepend over some legacy code to determine whether the code needs refactoring or a redesign for quality purposes (e.g. near the end of the project). If all the packages show a green result on the graph, can you be satisfied that your code is of high quality? Not really, as you are probably only looking for code that fails to satisfy these thresholds, rather than looking more closely for reasons why the ones that pass do pass.

Can we trust these tools to give us reliable metrics on the quality of our code? Is the practice of using these tools a waste of time? The answer to both these questions is a resounding ‘No!’

When using a tool, any tool, you must truly understand the details of what the tool actually does, look behind the raw information it gives and put the results in context. Using JDepend, does a Coupling ratio of 3 mean your code is great quality or full of bugs? How do you know, unless you understand the context of the code and package? Also, this tool is not actually telling you whether what it is reporting is good or bad, but giving you the information to investigate and decide for yourself. This is an important point as sometimes these types of tools are misinterpreted (or marketed) as providing a high/low quality mark, when really they are not. Therein lies the danger of using colored icons such as black, red or green dots.

In this example, it would be better to run JDepend after a static code analyzer (again, properly understood and configured). That way, the analyzer can detect issues such as empty code blocks or interfaces and unused imports, and eliminate them before running JDepend.

Last week I was in New York, speaking at the Real World Java seminar. My presentation was about code quality, and as part of the presentation, I wanted to talk about how, as an industry, we have a tendency to try and adopt best practice from other, more mature industries. It’s always bugged me that we try to adopt lessons from manufacturing industry when, to my mind, that industry is so fundamentally different from what we do.

So in preparation for the presentation, I was researching the origins of the software industry, along with a typical manufacturing industry, the auto industry. And I was rather shocked to learn that you could argue that the software industry is actually older than the auto industry.

Let me explain. Henry Ford introduced the Model T in 1908, although the four stroke engine had been invented by Nicolaus Otto in 1872 (that’s it on the right). But hang on a second, remember Charles Babbage? He invented his analytical engine - generally considered to be the first programmable computer - in 1835. That was about 37 years before Otto’s engine. Even although Babbage’s engine was never actually built, in 1842, Ada Lovelace (for whom the language Ada was named) described an algorithm for the analytical engine to compute the Bernoulli numbers, and, as such is considered by many to be the first computer programmer.

So, if our industry has been around for longer than the auto industry, why does it seem so young in comparison? I think it’s because the pace of technological change in the computer industry has slowed advancement in the development of processes. The engine that’s under the hood of my car is based largely on the same technology that Otto invented in 1872. But computers have moved from Babbage’s original conception of a mechanical device through analog machines in the 1930s to our current digital technology - and we are constantly coming up with new programming languages just because - well, we can.

And so it is that we keep telling ourselves that our industry is young, when really what we mean is that it’s immature. For it to mature, we need to come up with - and adopt - best practice in a number of areas. But that’s not going to happen while we try and adapt practices from other industries that have very different characteristics to our own.

I like Jeff Atwood’s Coding Horror blog. Atwood believes that discipline is the key to benefiting from the conventions that we do already have, and that it has to start with the individual developer. I think he’s right - a mature industry means process improvement, and as developers, it’s up to us to drive that process improvement - and that sure takes discipline.

The outage at LAX that delayed 17,000 travelers last weekend turned out to be caused by a network card failure, so, in its capacity as a hardware glitch, we won’t even mention that here.

Amongst all the hubbub about that, you may have missed the rather curious case of the Y2K bug that has only just surfaced in the past week. Apparently the bug caused errors in NASA climate data that, now that they have been corrected, have opened a whole debate about whether global warming is really happening or not. Nothing important, then.

Talking of important, anyone who ever doubted the financial impact of software quality should take a look at eBay’s stock price, which, according to the Wall Street Journal (subscription required for full article) shed 89 cents yesterday owing to the Skype outage. “It’s a software issue” according to the Skype blog, and they expect the problem to be fixed within 12 to 24 hours. As of now, my Skype service is still down…

One of the most difficult questions I am asked by customers is: “As a manager, what incentives can be used to get developers to pay attention to metrics and hit thresholds I’ve set?” This question was discussed at a round table by a few attendees here at Agile 2007 this week.

One company represented here had offered cash bonus incentives for individuals and teams that hit thresholds for desired metrics. Inevitably, people started to ‘game’ the system and, since there was only one prize offered per month, after the first month the developers found reaching these numbers mundane and de-motivating. The company ended up paying the bonus in a round robin format to try to keep up morale. – Not exactly the desired outcome!

The best answer I can offer to this problem is to appeal to developers’ sense of professionalism, education and pride, in that order.

1. Professionalism: As long as the development team understands why the metrics are being used, i.e. the business value they provide, or the quality process that they are a part of, then I believe developers will support the effort. As professionals, they should understand that their work needs to be measured against organizational or industry standards (especially in a regulated industry), but oftentimes managers do not spend enough time explaining why quality metrics are being put in place.
2. Education: Integrating metrics into a feedback loop provides a great educational tool. Visibility of common mistakes help developers self-improve, and steps can be taken to fill skill gaps. In the long term, that will lead to more job satisfaction. Too often, metrics are not shared widely enough or discussed with the team on a regular basis.
3. Pride: Software Development is something that people want to do! I have never met anyone who was forced into the job because they could not find anything else to do. Most developers want to produce the highest quality work possible, and they are proud of their output. If somebody/something can easily point out how they can improve their craft, the more mature developers will pay attention. As long as the metrics have some credibility.

If a metrics program is put in place, care should be taken to fully explain the reasons why they are being used and what, if any, feedback loop is going to accompany them. If developers are told how these metrics are going to help them in their job, and the metrics are based around proven quality measures, rather than being productivity related, then the metrics are more likely to be embraced. Of all these issues, the feedback loop is the most important. After all, if people feel that the metrics are not being used, why bother collecting them in the first place?

I’m at Agile 2007 this week having a grand old time. On Monday, Bob Martin discussed some benefits of pair programming in his ‘Introduction to XP’. Martin believes too much can happen between when code is written and when a code review occurs, and that it is more efficient to have the review done as the code is being written. Also, he suggests that a daily change of pairs is essential, so that more developers gain more knowledge of the system. This is doubly beneficial because (a) if one person is absent, no knowledge of the system is lost and (b) individuals gain more experience of the different technologies involved, which is motivating and would not occur if a traditional ‘specialist’ methodology was adopted.

In my experience, pair programming is the least practiced tenet of Agile methodologies. I have heard several reasons for not adopting this technique, including:

• Management not willing to pay two people for what they perceive as one person’s job
• Developers do not want to be “criticized” by their colleagues
• Pairs spend too much time arguing about trivial issues regarding coding techniques.

It put me in mind of Alistair Cockburn and Laurie Williams’ paper (PDF) on the subject, in which they concluded that:

• The additional overhead of pair programming was only 15%
• Pair programming resulted in a 15% reduction in defects
• Although that may sound like break even, the time that would have been required to find and fix the defects was 15 times more than the time represented by the loss in initial productivity.

In other words, in their study, they observed a 1,500% ROI.

My take? As professionals, developers will want the team to produce the best quality work possible, and pair programming is a step toward this. Unfortunately, despite the potential ROI benefits, pair programming represents such a radical change to current development practices that I believe that it will remain a minority practice for at least several years.

From the many reports of software glitches this week, (including an outage at all-the-rage social networking site Facebook), I decided to focus on a couple of interesting (to me, anyway) stories that have one thing in common: speed.

The world of Formula One racing is not familiar to many Americans, and yet it is a wildly popular sport in other parts of the world. It also is seen as a crucial testing ground for many new automotive technologies that eventually find their way into the cars that we drive. Being an ex-European, I like to keep an eye on that sport, and so it was that this story caught my eye.

Last Sunday was the date of the Hungarian Grand Prix, held in Budapest. The day of the race, crash.net ran a story indicating that Polish driver Robert Kubica was complaining about a software problem with his BMW car that was causing problems with acceleration.

By the time the problem was detected, the car was in “parc ferme“, which is the period right before the race in which teams are not allowed to do any maintenance to their cars. In the event, the glitch couldn’t have caused that many problems for Kubica, since he finished in 5th place, only two places behind his BMW teammate who, by the way, did not report any such software problems.

In other speed-related news, the Newport News Daily Press reported on a presentation made at Langley AFB on maintenance procedures for the Air Force’s newest jet, the F22A Raptor.

In the presentation, Air Force Lt. Col. Dane West said that the plane’s extensive reliance on software has pros and cons. On the one hand, it makes diagnosis of mechanical problems easy: “Most of the time, all that a mechanic or plane crew chief has to do to diagnose a problem is hook up a laptop computer to the plane, and the aircraft will spit out a code detailing what went wrong inside.” On the other hand, software can “go on the fritz” too.

Apparently, the first time the plane flew over the international date line, the aircraft’s GPS system got confused, and the planes had to follow refueling aircraft back to Hawaii.

I’ll be speaking at Sys-Con’s Real World Java one-day seminar in New York City next Monday, August 13. Despite the fact that it’s August, it looks like they have lined up an interesting panel of speakers for the day, including sessions by Yakov Fain on using Flex with Java (that’ll be interesting to me) as well as sessions by Sun on Java 6.0 and a panel session on Java 7.0.

The title of my session is “Code Quality: Pay Now or Pay Later”, and I will be talking about why code quality is an issue that many sweep under the rug, why we should care deeply about it, as well as some practical steps to start down that path. Our CTO Mark Dixon will be there with me, so if you’re in the NYC area, come say hello!

Registration is discounted by $500 through the end of this week, and you can register here.