Technical & data

robots.txt

FRFichier standard situé à la racine d'un site qui indique aux robots d'indexation (Googlebot, Bingbot, et désormais ClaudeBot, GPTBot…) quelles URL ils peuvent visiter.

ENStandard file at a website's root that tells crawlers (Googlebot, Bingbot, and now ClaudeBot, GPTBot…) which URLs they may visit.

See also

More terms — Technical & data

Back to the glossary