Parts of Search Engines

Parts Of A Search Engine

A search engine consists of three major components.

These are:

The Crawler or SpiderFrequently called a spider, the crawler is a piece of software that analyzes the contents of web page and compares it to the keywords or search term used by the searcher. It follows any links provided on the page and assesses the relevance of these links to the content of the page. You can enter html text to ensure that spiders do not follow any link that you do not want them to do. To do this enter the following after the URL statement in the HTML: rel=nofollow. If you add this inside the HTML tag – just before the last chevron ‘>’ is a good place, the spider will ignore the link.

The crawler will visit your site on a regular basis and it is wise to keep your content fresh and updated. You will be listed higher if your content is regularly refreshed or added to so that the spider can see that it is not stagnant between visits.

The Index:The results of the spiders search are put into the index. It is simply an index of the web pages found relating to the search term used. It is possible for your web page to have been visited by the spider, but is not yet included in the index. It will not be included if the spider software doe not consider your site sufficiently relevant to the search term, or keyword, used and is unlikely to be included if you site is a copy of one already indexed, or its content is similar.

The best chance your page has of being included, especially in a high position, is if you have unique content to offer anyone any searching for information on a specific keyword.Your content should also be up to date and regularly refreshed. If you have few pages indexed in a search engine such as Google, you will be unlikely to be included in free directories such as Dmoz since your site will not be different enough from all the other sites on the same subject to warrant inclusion.

The Search Engine Software:This is the coding that searches all the pages in the index and list them in order of decreasing relevance to the search term being used by the user. Google, like most other search engines, wants to satisfy its customers, and not present them with a gaggle of web pages irrelevant to or all with the same information on what they are seeking.

The more specific a searcher is with the keywords used, the more relevant will be the information provided by the search engines.

So, the more specific you make a search, the more specific will be the information, and if you have a website, specific pages on your site giving good information on certain aspects of the subject are liable to be listed higher than your home page.