User-agent: * Disallow: */cdn-cgi/ Disallow: */internal-redirect User-agent: * Allow: /