User-agent: * Disallow: /config/ Disallow: /template/ Disallow: /WEB-INF/ Disallow: /META-INF/ Disallow: /rss/feed/sol/ Disallow: /rss/feed/video/ Disallow: /feed/externalheader/ Disallow: /feed/externalfooter/ Disallow: /adtest/ Disallow: /henvisninger/div/ Disallow: /sok/ Disallow: /widgets/ Disallow: /api/ # start AI crawler block User-agent: CCBot Disallow: / User-agent: GPTBot Disallow: / User-agent: anthropic-ai Disallow: / # end AI crawler block Sitemap: https://www.aftenbladet.no/sitemap.xml