Robots.txt is a file that exists on the basis directory of every Web-site and can be utilized to instruct search engines like google and yahoo on which directories/information of the web site they are able to crawl and involve within their index. Technological setup, content material, and inbound links are https://news.rafeeg.ae