The Cost of Poor Performance

Phil Dixon recently gave a great talk at Velocity 2009 talking about the cost of poor performance. Through a series of rigorous tests, measurements and reports, Shopzilla was able to quantify the cost of performance to the bottom line of the business. Performance monitoring is often thought of as insurance, you pay a small premium every month so that you are notified when the site down. But it is increasingly becoming much more than this. With the variety of tools and APIs available now on multiple platforms, it’s possible (though this does require a good amount of effort) to tie performance and profitability directly together. There was another talk at Velocity that discussed the future of performance monitoring is really about being able to translate the performance and scalability costs into the direct impact to the business. In this model, the idea is to create a curve that plots costs of scalability and performance (hardware, engineering, etc.) against the impact to the bottom line (revenue, direct sales, etc.). Allistar Croll and Sean Power’s book talks a lot more about these topics in depth.

Evolution of the Monitoring Client

I’ve been exposed to the monitoring industry for almost 5 years now.  In that time the industry has actually made some pretty major changes to how monitoring is performed.  The ultimate goal of monitoring is to provide the most accurate availability and performance metrics.  For now availability metrics are only made more accurate by monitoring more frequently.  Frequency of monitoring is not a major hurdle and availability really only has one consistent definition (either you’re up or you’re down) so the collection/reporting of availability isn’t very dynamic and will not be subject to a series of evolutionary steps (though you could argue that availability could be different depending on the browser type you use – an argument I make below).  However, performance metrics will evolve as the industry strives to provide more accurate performance metrics and also, provide those metrics for any number of browsers currently on the market.  Here’s the evolution so far:

1. Internal web monitoring – Provides monitoring of systems applications, and network connections.  Because there are no major players out there the resources do not exist to allow for monitoring from an end users perspective.  Performance monitoring is a very subjective process where an end user has to describe how the perceive the performance of a website/application.
2. External web monitoring – Starts to provide information about the performance of a website/application but it is rudimentary and usually the performance data is based off of a program that only hopes to emulate the performance of a browser against the site.  No actual browser is used for monitoring.
3. External RIA monitoring – Applications are becoming more dynamic and rely on the capabilities of the browser to display the page more like a desktop application.  This requires that external monitoring take these pieces of behavior into account when considering the performance of a web application.

That’s where we’re at right now.  I think a logical next step would be:

• Generic External RIA monitoring – This would be monitoring that would consider any (and every) platform possible for viewing the web application.  For simplicity just assume that it’s every browser on the market (Chrome, Safari, IE6, IE7, IEX, Firefox 2, etc.).

The browsers mentioned seem to be diverging and are trying to grab more market share by adding new functionality to their offerings.  So how does having performance data based on Chrome help you determine what your performance would be in IE7?    For the time being there really is no need for such distinctions…the important factor is that you have an actual browser that is generating your performance metrics.  Because of price and implementation I would suspect that monitoring solutions are going to become more decentralized.  Also, with all this new functionality there is going to be an increase in the different methods that are used for delivering functionality.