Search engine bots – site crawlers


A bot, it is also an Internet bot, a robot, www-bot is an automatically working program, the purpose of which is to perform any given actions using the same interfaces as a regular live user.

The term comes from the English bot – the abbreviation of the word robot. Most often, the term is applied to the Internet, less often to other computer programs. Bots make the work monotonous and monotonous, and at such a high speed that it is definitely inaccessible to the average user. Their actions are based on the restrictions prescribed in the robots.txt file, which is hosted on the server.

Another important ability of bots is to function in conditions when the human reaction is not the best action (for example, they find their successful application at online auctions, in games). Sometimes they are used and, conversely, to imitate human actions (chat bots, etc.).