Search and Best Results Analytics

Prefetching content is a clever way to speed up your browsing experience. Simply put, given certain conditions, your browser can prefetch web pages, so when you do visit those pages, they are already in your cache, thus speeding the time to display them in your browser.

Google started prefetching content for Firefox users in March of '05. Note that this doesn't work in IE (although StumbleUpon does), nor even in Chrome (although Chrome does prefetch/pre-resolve DNS information). They don't prefetch for all searches...only those where there's one result that has a high likelyhood of being selected...the "best result".

Here's how Google's search prefetching works. You perform a search, say for "apple". You get back something like these results.

Behind the scenes, google has also added an extra tag to one of the links instructing the browser to prefetch the Apple site (again, only Firefox does it) of http://www.apple.com/

For those who want more details, the tag which generates the prefetch includes: link rel="prefetch"

And to tell you maybe even a little more than you want to know (but highlights an important fact), when the browser does prefetch this information, it doesn't fetch any additional objects referenced on the page. Only the HTML content of the link. (yep...if you're using a javascript tagging solution for your analytics, you are not capturing that prefetch visit...)

Now, how does Google determine what link to prefetch? What if I really wanted info on the actual fruit? Google doesn't divulge how they actually determine when to insert a link, I think it's an interesting metric that is worth investigating. I think of it this way. Google is basically saying, "I have a pretty good idea what you're going to do next, so I'm going to make that next site you're going to visit a little faster for you".

We all pay pretty good money to GYM (Google, Yahoo and Microsoft) and others to send search traffic to us. We also spend money on tools to help us better understand what search terms folks use to get to us even when we're not paying GYM and others. And of course, we all track this important information by paying WebTrends and others for excellent tools to make sense of all of our efforts.

However, unless you've made plans to intercept this prefetch information, you are not collecting this important data. Why is it important? Only Google knows how many times your site is returned in search results. You only know:

1. How many times it was returned in paid results
2. How many times a visitor clicked through to your site in search results (paid or organic)

What's missing is how many times your site was returned in organic searching (does anyone know how to get absolute search info from GYM?). And what would also be excellent to know is how many times Google thought that the visitor was going to select your site - based on the best result. Again, here's what I'm thinking:

Google thinks a visitor is interested in your site. This tells you:

1. The visitor searched for something that likely would have led them to you.
2. Google thought it was relevant enough to even prefetch the content for the visitor.
3. The visitor may or may not actually make it to your site (why or why not?).
4. Which search terms were actually deemed relevant enough to make Google think your site was the right destination?

Ok, so how can you capture this information? If you're using web logs, you need to make sure you're capturing request headers so you can pull out a special header that is sent with each request and put it in your logs. The header is: X-Moz: prefetch

If you're using tagging, here's an approach that I put together a couple of years ago so we could watch this traffic over time: intercept the request and generate a request to your Web Analytics vendor data collection environment. I'd be happy to show you how to do this if you're interested.

Now you can track this traffic over time, and per visitor (depending on your WA solution you're using). What are these "best results" that these visitors are searching for? Did they actually select your site? Assuming you want them to visit your site, how might you change things so they do reach you? If they do make it to your site, which paths did they follow? Which scenarios did they fall into?

Questions? Thoughts? Comments?

Back at WebTrends...

This past week was my first week back at WebTrends. First, let me say thanks to Alex, Bruce, James, Mike S, Shane, John D, John R, Leo and Barry who all made time for me to bounce ideas around and find the right fit for me. Everyone has made me feel very welcome back.

So, why jump back inside analytics? And why back to WebTrends?

#1: The WebTrends Team
You can't underestimate the power of a strong, smart executive team. I had the privilege of working with an incredibly bright, innovative, forward-thinking management team at Jive, and I can tell you it makes all the difference to have a quality team with whom you can trust, engage, and collaborate.

Alex is building something special here at WebTrends. I had heard it from others, and I can already see it. The team rallies around him, and the exec team isn't wasting any time to make real change within the company.

#2: The Analytics Industry
I like this industry. The technology is dynamic, our customers are creative, and there's a large number of smart people out there solving interesting problems. I see too that the WA blogging world continues to grow, and there's plenty of twittering from a plethora of sources as well. This level of sharing and collaboration is very healthy for us all.

Many thanks to the many voices out there who are keeping us all enlightened, challenged, converted, and otherwise demystified. I'm looking forward to getting ramped back up and connected with you all again.