Online Robots.txt File Scanner
Online Robots.txt File Scanner
Short Info
Level
Informational
Single Scan
Single Scan
Can be used by
Everyone
Estimated Time
10 seconds
Time Interval
3 days
Scan only one
URL
Toolbox
-
The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a text file with instructions for search engine crawlers. This file can expose something sensitive such as the path of an administration panel.
