The Yahoo! Search engines like Yahoo! crawl through websites to gather information, returning a list of relevant websites to the user who types search terms into them. Bots return up-to-date information on currently active websites but have been known to return defunct websites.
Yahoo! Slurp
Yahoo! Slurp is the recognized name of the bot that the Yahoo! search engine uses. It's a different bot than Googlebot for Google and Bingbot for Bing. Yahoo! Slurp is based on Web crawler architecture developed for Inktomi. The Yahoo! Slurp bot crawls through websites and creates a virtual copy of them to be used later for the search engine.
Bot Tasks
The Yahoo! Slurp bot performs three basic actions for the Yahoo! search engine. First, it retrieves Web server website information. Then the bot analyzes the website content for relevant information. Finally, it files the information to be used in site indexing. The bot itself is little more than an automated script performing these three tasks over and over. However, it can do it at incredibly fast speeds. A Yahoo! search engine results page takes mere seconds to load. It wouldn't be possible without the bot.
Site Indexing
The process of ordering the websites that the Yahoo! Slurp bot has found is called site indexing. Websites are ranked according to their relevancy to search terms and their individual ranks within the Yahoo! search engine. The more popular and integral to the Internet the site is, the more rank it obtains and the higher it is when Yahoo! returns its search results. The Yahoo! Slurp bot helps index the sites for Yahoo!
How the Bot Searches
The Yahoo! Slurp bot uses four parameters when accomplishing its website information retrieval process. It operates using a selection policy that determines which Web pages to download; it uses a re-visit policy that determines when the pages are checked for updated information. The bot also operates by a politeness policy that prevents certain websites from being overloaded with searching. It also operates by a parallelization policy, which coordinates the Yahoo! Slurp bot with other Web crawlers.
Reference
Meg North,What Is a Bot on Yahoo?, Retrieve in 06 september 2011
URL:
No comments:
Post a Comment