Robots.txt is a file that exists on the foundation directory of every website and can be utilized to instruct engines like google on which directories/documents of the website they might crawl and include things like within their index. be to utilize this aspect to help make your content extra accessible https://news.rafeeg.ae