spoton corporation home clients contact upload download artwork resources A-Z FAQs site map terms
solutions

products

exhibition productsprint litho
print digital

services

web designexhibition design
graphic designproduct design
advertisingwebsite hosting

solutions

marketingdesign concepts
brandingweb marketing::
e-commerce

 

What are search engine robots and spiders?

A robot is a program that visits web sites to retrieve information from them.

Search engines use software programs called robots or spiders to gather web pages for indexing. Robots and spiders will then follow links on the pages they find and continue to gather information.

The majority of robots provide a valuable service by automatically gathering vast amounts of information from new websites or pages of older sites that have been updated. It then stores this information in the search engine database. Search engines are then able to respond to a search query immediately as they already have the neccessary information stored.

Robots will generally index your home page first and then follow links to other pages. The more popular your site the more likely a robot is to index it. 

 

The factors that robots look for.

Most robots will look at the information contained on your site and then insert it into its database. Different robots will look at different factors on a site. Some will index the HTML titles, the first few paragraphs or index the entire HTML code, including the META tags. Robots will then assess how focused this site is and store the information in its database.

Register my site with a robot

 

The most popular robots and spiders

Google: GoogleBot and Freshbot

Yahoo: Slurp

Inktomi: Slurp/2.0 (Inktomi is owned by Yahoo)

AltaVista: Scooter (AltaVista is owned by Yahoo)

EuroSeek: Arachnoidea

Lycos: LyCos_Spider

It is obvioulsy very important to design a search friendly website if you wish to be indexed by the robots and spiders. If your site is difficult to navigate it will not be included in their database and therefore will not be selected for any queries by a browser searching for you on the web. 

 

 

 

 

I

SEO tips for robots and spiders

All your important pages must contain good quality text that can be indexed by search engine robots and spiders.

Any in-bedded links should also have duplicate text links so that robots and spiders can find them and index these pages.

A site map is also an advantage and if connected to your home page is an effective way of ensuring that all your pages can be indexed.

Search engine optimization will ensure your business competes as effectively as possible in your online market and therefore your profits will increase.

[edit]