How do search engines work?

Searching the web has a lot more to do with spiders than you'd expect.


Asked by: Phil Oldfield, Stockport

When you enter a search in Google, you aren’t actually searching the web, you are searching Google’s index. This is compiled and continuously updated by automatic programs, called ‘spiders’, that follow links from page to page to travel across the web. The information they collect – along with snapshots of the pages themselves – is stored in the index.

The index is much smaller than the web and the index entries are all stored together so it is much quicker to look through it than hopping from one computer to the next across the internet. Also the index is sorted and organised to make searching faster.

Subscribe to BBC Focus magazine for fascinating new Q&As every month and follow @sciencefocusQA on Twitter for your daily dose of fun science facts.