To follow from the successful launch of our free Eclipse plugin, today sees the official launch of our Service Offerings for Java Development Teams.Over the last 5 years we have been helping development teams increase the quality of their applications.From these experiences and our own experiences we have formalized a collection of offerings that can help improve code quality further.In summary our Services consist of:- An Internal Quality Code Report- Continuous Integration Services- Context Specific Standards for the free Enerjy Software plugin.More details can be found on our Services page of our website.

After much thought, we have decided that selling Java development tools is not a viable business model for us. Several recent events have helped us reach this conclusion:

• At JavaOne this year, free tools completely dominated the sessions. The exhibit hall was sparsely filled, and while we saw lots of interest in free t-shirts and spin-the-wheel games, there wasn’t much interest in the vendors’ products themselves.
• Our friends at Agitar have begun to wind up operations. It was interesting to watch them go through several iterations of their business model but ultimately they were unable to find anything that gave them a reasonable return.
• Sales of our own product have been slower than we had anticipated.

Ironically, I was putting together my own budget for next year and realized that there just aren’t any software purchases on there. We’ve bought a few specialized pieces of software - Mathematica for the index work; JProfiler for performance tuning and Atlassian’s Jira and Bamboo - but we’ve moved to free tools for everything else. Atlassian has insane maintenance prices, so if anything happened to Jira or Bamboo, we’d almost certainly replace them with Trac and something like Hudson.The good news is that we are absolutely committed to the Enerjy product and, therefore, with immediate effect, the Enerjy plugin for Eclipse is available at no charge. Our goal is to become the most widely used static code analysis tool in the Eclipse world by providing a wide but pragmatic ruleset and the same level of seamless integration that you’ve come to expect from Enerjy. We have great plans for widening the ruleset and making the configuration wizard smarter to help manage all those extra rules.So, how are we going to stay in business if the product is free? Well, we run a very tight ship here, and we have sufficient funding to continue for the foreseeable future. Moving forward, we have had success in the past providing expert advice to organizations on code quality. With the additional data and experience we have gained from the Enerjy Index, the plugin and Bugpedia we believe that we can offer unique, technology-based consulting services to companies seeking help with code quality issues.To those users who purchased the product after April 1, we will be contacting you to arrange a refund of your purchase price.If you have any questions please post them here or, if you prefer, contact me directly at mark_dixon at enerjy.com.

 After three months of beta testing Enerjy, it’s April 1st and that’s the date we always said we would go live with the commercial version of the product.  So, as of today, we’re switching from the free beta version to a 30-day trial model.  If, 30 days after you download and install Enerjy, you decide to keep it, you’ll need to buy a license at $195.We have discussed the variables at length.  The sub-$200 level feels right to us, in terms of the value that the product brings.  Software pricing is a tricky subject and it’s amazing how almost everyone has an opinion on it.  Feel free to comment if that includes you.  We’ve also discussed the relative merits of a 30-day trial versus having a “free” and a “professional” version of the product.  The problem with the latter approach is that different people will pay for different features, so it becomes hard to draw the line between the “free” and the “paid for” feature sets.  So, in the end, we decided to go with a pretty traditional model, a $195 perpetual license with a 30-day trial.During the beta period, we had about 1,200 people download and try out the product.  We have also been pretty pleased with the level of engagement on the forum.  In fact, we decided to give free licenses to anyone who posted on the forum during the beta period.  Just our way of saying thanks for participating.  Also, students and other academic uses should be free, we think.  If that includes you, drop us a line.So, to everyone who participated in the beta, we thank you.  If you have thoughts or suggestions about the commercial model, please send them our way.  We’re all ears.

It’s Thursday morning, and yesterday we pressed the button on launching the New Enerjy. That means a shiny new Web site, not to mention a brand new (free beta) product. I won’t spend too much time here talking about the new product, there’s plenty to read on the new Web site. Plus, we had our talented friends at Common Craft create a short video describing what it is and how to use it.

Suffice it to say that Enerjy is based on roughly 12 months of research that we have been embroiled in leading up to yesterday’s launch. Our previous product, CQ2, was great for tracking code quality metrics. But often people would ask us: “where is the proof that improving these metrics will reduce defects?” To be honest, there wasn’t any. So we set out to answer that question, by analyzing hundreds of thousands of Java source code files, and correlating more than 200 metrics to defect rates. We found that some metrics, and even some combinations of metrics, turn out to be good predictors - to within about 85% accuracy - of bugginess.

So, if you’re interested, I invite you to try out the new Enerjy. It’s an Eclipse plug-in, and can be downloaded directly from within Eclipse. Most importantly, let us know what you think.

There’s more than one reason things have been quiet on this blog over the holidays. We’re getting ready to release the public beta of our new product, and things are a little, er, hectic around here. Personally, I’m not wild about corporate blogs that talk exclusively about the internal workings of the companies they belong to, but as we get ready for this new release, I thought some people may be interested in a little perspective.

The new product itself is an Eclipse plug-in. Our previous products have all supported multiple different IDEs, which we have always upheld as a key design goal. Unfortunately, that means that there are many compromises we’ve had to make in terms of what the product could do, and how well we could integrate with the IDE. Finally, we decided that Eclipse is ubiquitous enough that the cons of cross platform support were outweighing the pros. The good news is that there is a ton of neat stuff in the new product that we just couldn’t have done before.

The official launch date for the new product was going to be Monday 7th. However, we need to finalize some patent applications before we go live, and that’s taking a bit longer than expected, so we probably won’t actually go live until the middle of next week.

The new product, which replaces the current CQ2 and, by the way, will simply be called “Enerjy”, is best used by individual developers, as opposed to the team-centric approach of CQ2. Our experience with CQ2 was that many people found it over-complicated and almost overkill for their needs. By removing per-developer attribution of metrics, for example, we can do away with the need to integrate with 12 or 13 different source code management systems, which significantly reduces product complexity and, in particular, makes for a self-installable product. All of that means we’ll be moving to a more familiar self-service, try-before-you-buy sales model.

But by far the most significant piece of work we’ve been buried in for the past 12 months or so is the research behind the new metrics engine. Millions and millions of source code metrics have been analyzed and correlated to defect rates to find statistically significant relationships that tell us which combinations of source code metrics are good predictors of bugginess. That allows the new Enerjy to finger the areas of your code that you should pay special attention to, assuming you have finite resources available for quality improvement initiatives. It also monitors new code development, indicating immediately when something is not quite up to snuff.

Hand in hand with the new product release is a new version of the enerjy.com web site. That will have some very cool new stuff on it, but to find out what that is, you’ll have to come back next week!

If you’ve been reading here for a while, you will know by now that we have been doing a lot of number-crunching here at Enerjy lately. We’re on a quest to find a correlation between metrics and defect rates, and to answer the question: “what metrics are good predictors of bugginess?”

There is a different, but in a way, very similar quest going on at Netflix, where, for the past 13 months or so, they have been running a competition to try and improve their recommendation engine. That’s the software that figures out that, if I gave Bambi five stars, I would probably also enjoy watching Reservoir Dogs. Clearly, improving the recommendation engine is important to Netflix: back in October 2006 they offered up $1m as a prize to anyone who could improve their current algorithm by 10%. Despite the best efforts of the nearly 24,000 teams working on the problem, no-one has achieved the 10% improvement yet, and as of now (December 2007), the best percentage improvement stands at 8.5%.

But the job of a recommendation engine is very similar to what we are doing here. We’re analyzing tens of thousands of source code files, collecting data on a couple of hundred metrics on each one, and then statistically correlating those to the number of defects that were found in each one of those files. The good news is that we’ve found some strong correlations between certain combinations of metrics and defect rates, that allows us to make a pretty accurate prediction of whether the code we are looking at is going to be bug-prone or not. We’ll be launching a product based on that analysis in the new year. Once we’re done with that, maybe we’ll take a crack at the Netflix Prize…

By the way, if you’re interested in reading more about number crunching, and some of its applications in everyday life, I can highly recommend Super Crunchers by Ian Ayres. Fascinating stuff.

Bob Charette pointed me to an article in today’s Wall Street Journal about the Microsoft team that processes all those “Report this problem to Microsoft” messages that get sent any time something crashes in Windows. It’s certainly comforting to me that at least someone actually looks at those things. The article goes on to talk about the trials and tribulations of trying to track down bugs in complex software, and in particular how users are often asked to help with the process:

The experience of dealing with a software bug can involve an emotional journey that starts with helpless blind rage and ends with, if not a Kübler-Ross style of acceptance, then at least a little empathy.

The encouraging thing to me about this article is that it brings us another step toward a more general awareness of software quality issues. There are a few things that can happen to make the industry sit up and take notice of this issue: legislation, the attention of the legal profession, and the raising of awareness.

Last week’s announcement of the formation of SAFECode is interpreted by some as an attempt to head off the threat of legislation governing software quality. We have already seen examples of lawyers picking apart software quality as part of their case. And every time software quality is written about in a major non-trade publication, the caused is advanced.

I’ll be speaking at Sys-Con’s Real World Java one-day seminar in New York City next Monday, August 13. Despite the fact that it’s August, it looks like they have lined up an interesting panel of speakers for the day, including sessions by Yakov Fain on using Flex with Java (that’ll be interesting to me) as well as sessions by Sun on Java 6.0 and a panel session on Java 7.0.

The title of my session is “Code Quality: Pay Now or Pay Later”, and I will be talking about why code quality is an issue that many sweep under the rug, why we should care deeply about it, as well as some practical steps to start down that path. Our CTO Mark Dixon will be there with me, so if you’re in the NYC area, come say hello!

Registration is discounted by $500 through the end of this week, and you can register here.

In previous postings, I’ve been talking about code quality and coding standards – writing code right. This month I’d like to talk about the other side of successful product development: writing the right code.

Pretty much my entire career has been in the area of developing shrink-wrapped software and I’ve been behind the creation of around twenty new products. For reasons that I’ll talk more about in coming months, data mining is an area of research that I’m very interested in right now, so I thought it would be interesting to see if data mining is useful as a means of predicting the success of a new product. Obviously, twenty products is a small dataset and I’m not sure that we’ll be significantly redesigning our innovation process based on these results, but I think there are some general conclusions we can draw.

This analysis looks only at products that were offered for sale. I haven’t included products that never made it out of R&D. For each product, I recorded three key characteristics and the ultimate outcome. The outcomes are one of Successful, Marginal and Failed. A Successful product is one that you can build a business on. It is profitable even after including all the overheads of running an (appropriately sized) business. A Marginal product is still profitable and would make a valuable addition to a suite, but it isn’t strong enough to stand on its own. Failed products were not able to cover the costs of developing and selling them.

The characteristics I measured for each product were the source of the product idea, whether we used an external design partner, and whether the product was in a new area for us. The source was either Internal or External. Internal ideas are those that were generated purely within the company. External ideas originally came from a customer or prospect. A design partner is a customer or prospect who worked with us throughout the product development. Finally, I defined a new area to be an area in which we did not yet have a successful product: it may be a totally new area or an area in which we had introduced a marginal or even a failed product.

A golden rule of data mining is to start with the simplest model you can, since often adding additional complexity to the model has very little impact on the accuracy. A good starting point is the 1R (for 1-Rule) algorithm that attempts to predict the outcome based on a single characteristic. That didn’t work too well for our data, with all characteristics having an error rate of just below 50%. For the record, though, if forced to make a prediction based on a single characteristic, the best guess is that products in a new area will be successful whereas products in an existing area will be marginal.

The next step up in complexity is the Naïve Bayes algorithm. This is similar to the technique used by some spam filters to identify spam email based on the words in the message. For each combination of characteristics, the Bayesian model generates probabilities for the outcome of the resulting product. While I can’t share the raw data (many of the products are still being sold), here are the numbers from the Bayesian analysis.

For products in a new area:

For products in an existing area:

The Bayesian model has one immediate result: for any combination of the other product characteristics, using a design partner roughly doubles the likelihood that the product will be successful. It also significantly reduces, sometimes as much as halving, the likelihood that the product will be marginal or a failure.

There is less difference than you might expect between internal and external ideas. In a new area, both internal and external ideas have an equal chance of success. External ideas are, however, 10-15% more likely to be successful. In an existing area, internal ideas are more likely to succeed or to fail, whereas external ideas tend to be marginal. That makes a lot of sense, since customer ideas are often variations on what we already have, and thus likely to be successful but not outstandingly so. Internal ideas in an area we know well are likely to be more innovative and trade a slightly higher chance of failing for a higher chance of success.

So what do we make of the influence of design partners? My feeling is that the actual contribution of the design partner is not the cause of the product success: we certainly build better products when a design partner helps us, but I’m not sure that the difference is enough to account for a different outcome for the product. No, my view is that the fact that we are able to find a design partner willing to work with us on the product shows that the product is addressing a problem that companies are prepared to invest (time) in to solve.

That means that f we aren’t able to find a design partner for a new product then we know we’re taking a gamble: internal ideas in a new area have a 45% chance of failure with no design partner. However, we can’t solve that by, for example, offering money to a potential design partner, since it’s their very investment that validates the product.

On the subject of failure, our overall failure rate is around 25%. Not surprisingly, it varies significantly between products in new areas (35%) and existing areas (10%). I admit to being a little embarrassed by that number: it’s way too low. I remember one talk where Seth Godin, the former VP of Permission Marketing for Yahoo, revealed his failure rate was up in the 80% range. Nowadays it’s so cheap to bring a product to market (see Guy Kawasaki’s post where he creates and launches a Web 2.0 site for $12,107.09) that there’s no reason not to try more things. Sure there’ll be more failures, but there will be successes too.

One of my favorite business quotes of all time is from Herb Kelleher, former CEO of Southwest Airlines. He said “We have a ‘strategic plan.’ It’s called doing things.” (Herb Kelleher, quoted by Tom Peters – (PowerPoint link)). Given that Southwest is the only major airline to remain profitable after 9/11, that plan seems to be working out very nicely for them.

If you’re interested in becoming a design partner for code quality tools and you’re doing Java development using Eclipse, I’m interested in hearing from you. Drop me a line (mark underscore dixon at enerjy dot com) and I’ll follow up with you with more information on what we’re looking for. But, like I said, I’m afraid we won’t be paying you!

I was reading the Google Reader blog today and noticed a sense of openness there about a number of outages they have suffered recently. That got me to thinking about the difference between the sugar-coated brochure-ware of most corporate web sites, versus a more open, blog-centric face that companies can present to the world. Whatever else you say about Google, you couldn’t say that their web site(s) fall into the “sugar-coated brochure-ware” category.

And so I cast a critical eye over our own web site. Hmm, definitely some sweetness there, blog notwithstanding. Of course we all want to put our best foot forward, but I think that there’s something coming over the horizon as a result of all the hoopla around Web 2.0 and social networking, that means that in the future, customers and prospects will expect a more open, honest relationship with companies - and the people behind them. That doesn’t mean to say that your web site shouldn’t look nice, of course. But I think that people expect to be able to connect with some of the personalities behind a company these days, in ways that would have been unthinkable only a few years ago. It’s fascinating to me to see how many business people are opening Facebook accounts, uploading a bunch of personal information, and then finding that their business contacts find them there. That, to me, is symptomatic of a breakdown between our personal and business lives that seems like it’s inevitable. (If you want to friend me on Facebook, go ahead - you can find me here.)

We’ll be reworking our web site over the coming months as we lead up to a major product release we have coming up in January. As we work through that, I’ll be keeping some of these ideas in mind, and trying to strike the same delicate balance between professionalism and openness that works so well at Google.