Search engine optimization of your web site (SEO) is, of course, a good and hot idea. But it helps in that effort to know just how search engines function, so that you can tailor your efforts appropriately.
SEO Basics
Search engines fall into two broad categories, although nowadays most search engines use both of these methods. A search engine may be crawler-based, using automated programs to find web sites and incorporate them into a database according to search algorithms, or human-powered, relying on descriptions of web sites submitted to a directory service by the web managers or written by the service’s own editors.
Most SEO efforts are aimed at crawler-based search engines. That’s because only these can respond to changes in the web site itself, such as incorporation or revision of keywords or the acquisition of internal or external links. Human-powered search engines or directories can only be influenced by sending a revised description in to the service. In keeping with that, and also recognizing that the way human-powered search engines work is pretty obvious, let’s explore crawler-driven search engines in a little more depth.
How it Works
A crawler-based search engine has three main elements. The first is the crawler or spider. This is a program that visits a web page, reads its content, and follows links from it to other pages. When a page has been visited by a crawler, we say that it has been crawled or spidered. Most search engines dispatch the spiders back to review sites every so often to update information.
The second element is the index or database. Everything the spider crawls over gets incorporated into the index for review by the search engine’s software. When the web page is incorporated into the search engine’s index, we say that it has been indexed. Until a web page has been indexed, it can’t show up in a search using that search engine.
The third element is the software the search engine uses to generate search results, and this is where search engines get cagey and secretive. This software includes the algorithms that the engine uses to rank matches to searches in order of importance or relevance to the search. Details of this software are proprietary and secret partly because search engines don’t want their competitors getting hold of them, but also because they don’t want web site administrators to be able to game the system by using the algorithms to generate traffic artificially.
You can count on your web site being crawled before long after it becomes available to the public, and indexed quickly thereafter. The rest, though, is up to you, and you can increase your visibility in searches by good web design, use of keywords, and getting quality links inbound to your site.