Host in robots.txt context – SEO WIKI
The robots.txt host is a directive that defines the main mirror of the site. Even in the absence of different domain names, it is recommended to specify the main URL so that the pages always appear in the search with www or without www. In robots.txt, the host is a member of the User-Agent: * and User-Agent: Yandex groups.
Typically, the host directive in robots is located immediately after Allow and Disallow. Specifying a port is optional. If you do not know exactly how to set the host directive, after the colon with a space, it is enough to specify the domain name (Host: site.ru). Keep in mind that search engines can determine the main mirror of the site themselves, so if you have not previously selected and installed it in the robots file, be sure to ensure that only one spelling option is used everywhere.
When using a secure SSL certificate, the protocol should be specified (Host: https://site.ru). If the host in robots contains an IP address instead of a domain name or is spelled incorrectly, the directive is not processed by the mirroring robot.