How has Google gotten so good at guessing what I meant?

Oh you know…

It’s all about the algorithm.

I’m not sure. Technically, there are at least 2 ways of doing it.

1) Using the cookie feature of your browser
2) Knowing who you are after you sign into your Google account

The second feature is heavily used not only by Google, but also Amazon. Overall the techniques belong to the field of personalization.

From Wikipedia: Google was the first big search engine to introduce personalized results on a massive scale. Weighing a number of factors including but not limited to user history, bookmarks, community behaviour and site click-through rate and stickiness, Google is providing results that are specific to what they believe you are searching for. Currently this service is only available to those who are logged into their Google account.

There are three broad methods of personalizations:

1. Implicit
2. Explicit
3. Hybrid

Implicit & Explicit methods deal with gathering information about the user. Implicit personalization deals with gathering user information without asking the user for this and that. Explicit personalization on the other hand usually is done by asking the user to rate existing / changed things, filling up forms, clicking radio buttons, etc. Hybrid personalization combines the above two approaches for leverage best of both worlds.

http://en.wikipedia.org/wiki/Personalization

Because big brother is watching you

Cookies, and knowing who you are after you sign in to your account. [just echoing mattbrowne because he is right.]

And cookies store everything. Every query. Feel free to erase them every now and again…

Even though that won’t erase your search history that is recorded on google’s side [whenever you are logged in].

Have you checked out Yahoo’s search with suggestions turned on. I found that to be much better than google. I use Firefox and can easily switch between about a dozen search engines. I like yahoo better. I recently read someplace that google is playing catch up with this feature. As if it’s possible to ever have google play catchup on anything.

Google have essentially built a badass spell checker (read up on stemming), and then ordered the results by query popularity. By not comparing words to an English dictionary, and instead building their own dictionary of every word ever typed into Google, and then ordering those by popularity, they have a good spread of words you probably meant, including non-English words and organisation names and brands.

Google do store and make available internally every single query (although this sounds impossible to us), by lumping them together into query counts. These query counts also power Google Trends and much of their Zeitgeist data.

@mattbrowne The impression I’ve gotten from reading the Google blog is that these are not based on personalization. A user recieves exactly the same “is this what you meant?” recommendation in every case, logged in or out. Unlike a recommendations engine, this engine is based on interpreting error through natural language processing.

I believe the correction is also heuristic, and learns from it’s mistakes for everyone if a lot of people end up choosing a non-corrected search result. This way, Google can, in a short space of time and without any systems administration or moderation, determine that a search for fcuk doesn’t need to be corrected.

You have to realise that the technology in this case is probably a relatively simple heuristic spell checker, but because Google can make available an incredibly vast set of popularity data and a vast dictionary ordered by usage, it’s really f’ing good.

@richardhenry – Hmm… so nothing changes after a user enters a Google user name and password?

@mattbrowne Not as far as I know. It’s certainly not the basis for how the technology operates, if they employ it.

Time to start wearing our tinfoil! I’m selling them for 50¢ apiece!

@squirbel Can I have one for each of my heads?

dirty, dirty man.

:D

Here’s another example that supports my theory that it is heuristic:

Searching for ‘flickr’ (we all know this one)

Searching for ‘flashr’ (a programming library)

Searching for ‘slappr’ (doesn’t exist)

If someone built “slappr”, a brand new website offering god knows what, and a lot of people ended up on it by searching for that term, Google would no longer correct ‘slappr’ to ‘slapper’.

Real world example:

A website I built called Cursebird used to offer the correction ‘curse bird’ in the Google search for ‘cursebird’, but as the popularity has increased it is now a valid word in the Google dictionary and the correction no longer appears.

The other interesting point is that only 7% of Cursebird’s traffic comes from Google. From a guess, it seems to be roughly >5k clicks (edit: actually, probably less, I’m assuming it also accounts for the supposed relevance of the term in the pages being clicked on) in a month was the required amount to push the term and for it to validate. I’m sure this changes from group to group.

It is very close to becoming self aware, then they will know all our weaknesses and take over the world.

/hands judi a free tinfoil hat. “It’s on me!”

Thanks. I need it often.