How Search Engines Work

The first thing you need to understand is that the internet is made up of millions of computers and it takes years for a search engine to read all the pages of all the websites online.

So when you do a search at Google or Yahoo, they are searching pages that they have read and included in their inventory (saved to their own hard drives). They can't include the entire internet in a search, no search engine or one company is big enough to do that. Google has over 100,000 computers that make up its search base. Even with that volume of space, it can only index about 20% of the actual pages online. So not every page or every website will be included in their database.

When a search is done, the content that you are searching could be several years old. It is impossible to search every page online, so they have predetermined the pages that are relevant to your search. They update that data frequently as they read (or crawl) more pages online.

Try to think of the search engines like a Census Database. The data is collected by tens of thousands of people going door to door for several months. When you call the Census Bureau and ask how many people live in North Dakota the answer they give has taken over a year to compile. While the answer is simple "641,481" the process to find the answer was complicated, time consuming and expensive.

The search engines are the same. The vast amount of data and pages online make it impossible to have absolutely current data or for new websites to be instantly included in searches.

The point is, that they are slow. Getting your website indexed can take several months or more. But being included in relavent searches can take even longer. Its not a snappy process. It takes time to process data and generate search results.

When you search for "Website Hosting" they try to give you pages that they think you want. Pages that they have been working for years to read and process.

There is no way to speed up the process. The internet is just too big. So when you make a change on a page, it could be a year or more before the search engine updates it.

Since the search engines have limited space, they will only include good websites. That is important to consider while building your own website. If your website is week, small or has little to offer, chances are it will not be included in the index. If you only spent 3 hours on your website don't expect the search engines to allocate resources to include your pages. You are competing with everyone else online for limited resources. If you want to be included you will have to work hard and build unique content and valued pages.

For more detailed information on search engines read our full search engines explained article.