Apps tagged with 'Robots Txt'

All apps in Apps tagged with 'Robots Txt' category. Use the filters below to narrow down your search. 
Copy a direct link to this comment to your clipboard
  1. CrawlTide icon
     Like

    SEO platform combining technical audits, Google rank tracking, competitor research, local grids, and AI search visibility monitoring.

    Cost / License

    • Freemium
    • Proprietary

    Application type

    Since

    • New

    Platforms

    • Online
    • US flagUnited States
    AI Answer Readiness check: grades how easily AI engines can pull a clean answer from a page.
    Competitors: keyword overlap and how often AI engines cite each site.
    16 free SEO tools that need no account.
    8 alternatives
  2. QueryRev AI Readiness checks how a website's public files treat AI crawlers. It reports which of 11 AI crawlers (from OpenAI, Perplexity, Google, Apple, Common Crawl, Meta, Amazon and ByteDance) robots.txt lets in, whether the site's llms.txt is a real llms.

    Cost / License

    • Free
    • Open Source (MIT)

    Application type

    Since

    • New

    Platforms

    • Windows
    • Mac
    • Linux
    • Node.JS
    • US flagUnited States
    QueryRev AI Readiness screenshot 1
    2 alternatives
  3. URL Verifier Pro is a native Windows application built for SEO professionals, developers, agencies and website owners who need to verify large numbers of URLs quickly and accurately.

    Cost / License

    • Paid
    • Proprietary

    Application type

    Since

    • Since 2026

    Platforms

    • Windows
    • GB flagUnited Kingdom
    URL Verifier Pro screenshot 1
    URL Verifier Pro screenshot 1
    URL Verifier Pro screenshot 2
    +2
    URL Verifier Pro screenshot 3
    10 alternatives
  4. Generates verified robots.txt, sitemap.xml, llms.txt and llms-full.txt from a website crawl, checking crawler rules, links, hostnames and hidden text.

    Cost / License

    • Freemium
    • Proprietary

    Application type

    Since

    • New

    Platforms

    • Online
    • CA flagCanada
    Lab451 Generate verify monitor. robots.txt , sitemap.xml, llms.txt
    29 alternatives