Robots txt configuration (Edit config using page properties)

User-agent: *
User-agent: spacecat/1.0
Allow: / 
User-agent: GPTBot 
Allow: / 
User-agent: PerplexityBot 
Allow: / 
User-agent: ClaudeBot 
Allow: / 
User-agent: Google-Extended 
Allow: / 
User-agent: OAI-SearchBot
Allow: /
User-agent: Bingbot
Allow: /
Crawl-Delay: 1

Disallow: /stage.digital2.aveva.com
Disallow: /stage.digital3.aveva.com
Disallow: /search-results
Disallow: /search-results*
Disallow: /404
Disallow: /500
Disallow: /search
Disallow: /search/*
Disallow: *Search.aspx
Disallow: /Search?search*
Disallow: /search?q=*
Disallow: *admin
Disallow: /admin/*
Disallow: /assets/*
Disallow: /styles
Disallow: /styles/*

Sitemap: https://www.aveva.com/sitemap.xml
Sitemap: https://www.aveva.com/sitemap_es.xml
Sitemap: https://www.aveva.com/sitemap_de.xml
Sitemap: https://www.aveva.com/sitemap_pt.xml
Sitemap: https://www.aveva.com/sitemap_ko.xml
Sitemap: https://www.aveva.com/sitemap_ja.xml
Sitemap: https://www.aveva.com/sitemap_fr.xml
Sitemap: https://www.aveva.cn/sitemap.xml