Importance of Site Speed in Google’s own words

I recently came across a video by Google, stressing the importance of site speed. Google has sent strong signals that the download speed of web pages matters, not only for the best conversion rates but perhaps as a factor for ranking in the SERPs in the future. In addition, speed can have an impact on how many of your pages get indexed.

So why does Google think speed is of importance? Through the following tests:

Side by side testing of optimized vs Original versions of the sites:

Only difference is that the optimized version was faster but the content remained the same in both. Firefox and Shopzilla had some interesting findings.
Shopzilla had 7 – 12% increase in conversions and a 50% decrease in operating costs.
Firefox reported a 15.4% increase in downloads. Firefox estimates that this is about 60 million extra downloads.

User satisfaction test:

Google and MicroSoft conducted a user satisfaction test by returning delayed results to some users. Results showed that just about half a second delay caused 1% of the users to feel dissatisfied. As the delay increased to one second, the dissatisfaction rose to 4%. As the test continued, users were feeling so dissatisfied that Microsoft ended the test fairly quickly.

Google & Microsoft conducted another experiment. They found that a 400 ms delay in the rendering the search results reduced the query volume drastically and this trend continued for 7 weeks. In fact the query volume didn’t get back to normal until 11 weeks after the experiment was discontinued. So this speaks for itself how much a site’s speed plays a role in determining its conversions.

While content and relevance are still primary in determining a site’s rankings in SERPS, site speed plays a major role too now.

According to Steve Souders, a Google Employee and the author of ‘High performance Web sites blog’, the Performance Golden rule is that “80 – 90% of the end-user response time is spent on the front-end. Start there”.

How is that? Look at a Waterfall chart from webpagetest.org’s performance analysis of a web page.

It took under one second for the content to be returned by the web server. Close to one second, the browser started to render the content. But then it had to make all these requests to all the associated page components like the javascript, css, images. So the final page didn’t load until after 7 seconds. This proves conclusively that the front-end of a page needs a lot of attention and optimization.

Google’s webmaster tools’ site performance dashboard not only gives the average load time of a site’s pages but also provides a comprehensive comparison of the speed with other web sites on the web. So if your site performs better than 95% of the sites on the web, then turn your attention to the content. If not, make speed your priority.

‘Page Speed’ is yet another powerful tool to analyze a page’s bottlenecks. Page Speed is a firefox plugin that provides a score for each page based on its analysis and also provides its recommendations. People have found to decrease their load times by 5 seconds on an average following these recommendations.

What is a good response time to aim for?
According to Google, Following your competition is the best answer to this question. If your competition provides a better user experience then you would want to aim for a better and a reliable user experience for your audience. Akamais’ own case studies show that 2 seconds is the threshold for e-commerce acceptability. Google’s aim is however under half a second.

What other tips did Google provide?
Implement Progressive Rendering – Progressive rendering is when a browser can display content as it’s available incrementally rather than waiting for all the content to display at once. This provides users faster visual feedback and helps them feel more in control.
Bing found a 0.7% increase in customer satisfaction with progressive rendering.

How to implement it? Simple. Put stylesheets at the top of the page. This allows a browser to start displaying content ASAP.

5 Key Graphs

Over the past week I’ve been developing a presentation/demonstration for training people on performance monitoring using the Webmetrics GlobalWatch platform for monitoring.  As part of that training I identified 5 key graphs that prove to be invaluable when analyzing performance data.  To present these graphs I developed a slide that would allow my audience to draw by hand the 5 graphs and list the value that each graph provides.  So here is a copy of the slide filled out by me:

5 Key Graphs

To clarify, these graphs are specific to a monitoring service that monitors a multiple step transaction (ex: purchase on Amazon.com, user log-in, or user registration) however similar graphs should be available for other types of monitoring (stream performance, web service monitor, URL monitor, DNS monitoring).  Basically, I recommend that whenever you look to implement performance monitoring you should make sure that the presentation layer of that monitoring allows you (at least) to view your data in fashions similar to these 5 graphs:

1. Transaction Average Load Time – A graph that shows you the high level view of the data you are collecting, in this case a view of the average load time throughout the day of the performing the transaction (or viewing a single URL).  This acts as an executive summary as well that helps to show basic trends in the performance of the transaction over time.  Optionally it is of value to display any errors that occurred in the graph.  Also, it helps if the graph can easily be drilled-down on so that it does not lock you into only a high level view (the drill-down will allow you to see the individual sample values that make up the displayed average).  Again, the primary benefit of this graph is a high level view that allows you to look for trends in performance.
2. Transaction Step Averages – Another view at average data, but this time we’re drilling down to the individual steps that make up the transaction.  The example drawing shows a 6 step transaction with errors on steps 1 and 4 (and a performance bottleneck at step 4 as well).  The benefit of this graph is that you can now breakdown the performance of the steps that make up the transaction being monitored.  However, it’s still an average.  So it’s going to give us a high level view that allows us to identify what steps in a transaction can use improvements in performance as well as breakdown a complex set of data.  While errors on a per step basis should be an option to the graph drill-down capabilities would probably be overkill and presentation would be clunky at best.
3. Transaction Steps Over Time – A graph that shows the average performance over time for each step in the transaction relative to the other steps.  This graph is similar to the first graph discussed but it breaks down the data so that we can look at trending for each individual step (as well as see how performance degradation affects individual steps in the transaction – as opposed to the transaction as a whole).  Errors should again be an optional parameter to the graph but errors should be distinguished by what step it occurred on since the primary data plotted is per step performance.  This graph would only add value for a service that monitors multiple steps (either a transaction or a number of URLs).
4. Full-Page Breakdown – Arguably the most important graph that external performance monitoring can generate.  If you’re looking at a monitor solution that doesn’t provide it (or…gasp: doesn’t even collect full-page data) then you are not getting the true value of external performance monitoring.  The full-page breakdown is a waterfall style graph that shows the download/rendering characteristic of a web page (or pages).  The full-page displays performance data for every item that makes up a web page (images, CSS, JavaScript files, etc.).  Whenever you record performance of these items you should consider at least the basic performance metrics: HTML download, redirection, network latency, and transfer time).  The full-page breakdown is a great reflection of browser fidelity (which is a representation of how accurately a monitoring solution emulates an actual client – such as a browser).  This is generally the lowest level of granularity you can get with external monitoring and it allows you to see what impact components (JavaScript, images, etc.) have on the overall performance of your web applications and pages.
5. Uptime & Average Load Time -  This graph is central to external monitoring solutions (and would only exist on massively deployed internal solutions).  The focus of this graph is on providing performance metrics on a per location basis.  Since external monitoring is done from global locations outside your firewall you will see different performance for different regions (samples originating further away from your servers will take longer to traverse the Internet).  Monitoring solutions that are deployed in house suffer from proximity between the resource being monitored and the tool that is monitoring…this graph will show you what is the performance from different locations (the line drawn across the graph) as well as the uptime from each location (the bars in the background of the graph).  A common usage of this graph is to evaluate the benefits of a CDN.  If a site is not using a CDN you would expect to see a rightward trend in performance improvement (that is the line representing performance would descend to the right for locations that are closer to the server being monitored).  When a CDN is used you would expect to see a very consistent performance line because accessing a site/resources provided by a CDN will reduce overall performance no matter where the client is accessing the site/resource from (CDN = content delivery/distribution network.  This is a network of servers that push content to the edges of the networks around the globe so that requests for the content don’t have to travel far…thus reducing latency times).

One important note on the last graph.  I mentioned that the Uptime & Average Load Time graph is good for evaluating CDNs, this is true if the external monitoring solution is a impartial third party.  Some monitoring services may be partnered with CDNs such that the CDN refers customers to the monitoring company in exchange for performance metrics that are skewed to show better results then what are actually achieved.  There isn’t anything tricky going on here…it simply has to do with where the monitoring agent is located in comparison to a CDN server that is providing content.  If they’re in the same data center then the performance data collected is somewhat biased and will show better performance improvements than most people will experience in using the CDN.  Definitely check and find out what the context is behind data that you’re using to evaluate CDNs or any other web technology that promises to improve your performance.

PerfMon PerfOrmance

Hi everyone,

I thought I’d post some graphs on the performance of the PerfMon Blog over the last couple months.  There are a few reasons for doing this:

1. I want to see how consistent performance of a WordPress blog is.
2. The graphs added to the page will mean more objects being downloaded which will affect the weight of the page, which in turn will allow me to look at how adding images will affect the performance of a page.

This first graph shows the average load time of the PerfMon Blog over time.  We can see from this graph that the performance of the blog is pretty consistent over a 30 day period (around 1.84 seconds) and rarely do errors occur (the errors on the graph are timeout errors that occurred when the perfmon.wordpress.com page took longer than 3 seconds to download):

Average load time performance of the PerfMon Blog.

So far we see that the performance is pretty consistent from a high level.  The timeout errors when the page takes longer than 3 seconds to download does not concern me too much (it’s bound to happen) and a very consistent performance (that is a non-spikey graph) is always a good sign, especially when monitoring from geographically dispersed locations.  Now let’s drill down a bit and see what the performance of the page looks like from the object level (images, CSS, JavaScript files).  Here we have a waterfall style graph called a Full-Page Breakdown:

Full-Page breakdown for PerfMon Blog

The graph shows us the performance of the items on the page and breaks down that performance into three key values: DNS lookup time, latency time (i.e. 1st packet time), and transfer time.  Since monitoring is being performed from an IE browser we know that we’re seeing the actual performance of the page (JavaScript execution, rendering and layout, as well as any objects requested asynchronously…if any).  The object level performance looks pretty good the only thing I’d really like to comment on is the DNS lookup time (the yellow values of the graph).  It seems that because of a number of third party items (analytics and advertising code) there is a bit more DNS lookup time then I’d like.  The way IE works it will perform a DNS lookup for a domain only once and then cache that information for the remainder of the session.  So every time a new domain is introduced a DNS lookup needs to be performed.  We spent about 0.5 seconds doing DNS queries!

This final graph is here to show how the PerfMon Blog performs for viewers coming from different locations.  It looks fairly good:

Uptime and average load time for PerfMon Blog

This is fairly good performance.  The average load time (per monitoring location) is represented by the green line.  This line is rather straight meaning that the performance is consistent regardless of where in the US you are viewing the PerfMon Blog from.  That’s good news!  Often this line will start out low on the right (meaning the performance is good from those locations) and will increase as the line moves to the left of the graph indicating that users further from the server hosting the web-site see poor performance.  The background of the graph is broken down into 3 values:

1. Green – This is the percentage of successful samples taken from that location.
2. Yellow – This is the percentage of unvalidated errors.  An error is unvalidated if another monitoring location is unable to duplicate the error that was reported.
3. Red – This is the percentage of validated errors from the location.  An error is validated if a number of monitoring locations reported the same error.

We can see that certain regions generally see more errors (validated and unvalidated): Salt Lake City, Boston, San Jose, Newark, Scranton, and Los Angeles.  This coeincides with their higher load times as well.  Overall, the performance of the PerfMon Blog page is pretty good.  The question is; is that because it has minimal content, is hosted on a solid infrastructure, or has very few people viewing it .

I left my heart in San Francisco (Web 2.0 that is)

Earlier this year (March/April) Webmetrics exhibited at the O’Reilly Web 2.0 conference in San Francisco and we found that there were quite a few unanswered questions on the mind of fellow exhibitors and attendees. The most prominent questions were:

1. Service Level Agreements (SLA) – Just about everyone who came to the Webmetrics booth had some sort of requirement for SLA reporting. Mostly we saw that the requirement was to provide an SLA to users of a service (since many of the exhibitors were companies that have a SaaS model/platform or at least were providing web services that could be used by their clients to extend functionality of existing products. There were some cases where tracking SLA values was more geared towards keeping tabs on SLAs that are offered by third parties but the overwhelming majority were looking to provide their clients with the SLAs that had been agreed upon. This indicates that many companies are becoming proactive in sharing information with their clients (in the form of SLAs). Which leads to…
2. Collaboration – Everyone understood. Very rarely did someone not get the idea of collaborating with third parties or partners. One of the main ideas behind the Web 2.0 movement is to develop software using a service model. Just about everyone in attendance of the conference was entrenched with some sort of third party. People are naturally suspicious which makes for a bad situation when a third party offers up some metrics that were collected in house. Often reports are generated by the provider of a service and then handed over to the user without any explanation of what errors are, where data was collected from, or even…god forbid, incomplete data sets.
3. Problems – Finally, the majority of people who stopped by the booth had experienced some sort of performance issue. In most cases it was uptime, that is, the service being provided was not available for extended periods of time (or unavailable for short periods of time very frequently). Although users of web services are becoming more sophisticated with their consumption they need really need to buckle down and pay attention to SLAs and demand (or track) SLA information (see the first point).

I will be at the Web 2.0 conference in New York City this September. Please feel free to stop by the Webmetrics booth and share your thoughts and opinions with me on the performance issues that surround web eco-systems and Web 2.0 applications. September.

The performance for this blog (today’s average load time and uptime):

• Average load time is 1.78seconds.
• Availability (uptime) is 100%.