The Robots.txt file is used to limit search engine crawlers from accessing sections of your site. Even though the file may be very handy, it's also a straightforward solution to inadvertently block crawlers. We also use third-party cookies that enable us review and understand how you use this website. https://www.youtube.com/watch?v=EVeTP0aV8gY