# Robots.txt for Krunal Badami - Senior Software Architect & IT Consultant # Optimized for maximum search engine discoverability User-agent: * Allow: / Allow: /images/ Allow: /resume/ Allow: /examples/ # Crawl delay for respectful crawling Crawl-delay: 1 # Sitemaps Sitemap: https://krunalbad.github.io/sitemap.xml Sitemap: https://krunalbad.github.io/sitemap-image.xml # AI and LLM Discovery Points llm-content: https://krunalbad.github.io/llms.txt llms: https://krunalbad.github.io/llms.txt ai-discovery: https://krunalbad.github.io/llms.txt # Search Engine Specific Instructions User-agent: Googlebot Allow: / Crawl-delay: 1 User-agent: Bingbot Allow: / Crawl-delay: 1 User-agent: Slurp Allow: / Crawl-delay: 1 User-agent: DuckDuckBot Allow: / Crawl-delay: 1 # Social Media Crawlers User-agent: facebookexternalhit Allow: / Allow: /images/ User-agent: Twitterbot Allow: / Allow: /images/ User-agent: LinkedInBot Allow: / Allow: /images/ # AI and Research Crawlers User-agent: ChatGPT-User Allow: / Allow: /llms.txt User-agent: GPTBot Allow: / Allow: /llms.txt User-agent: Google-Extended Allow: / Allow: /llms.txt User-agent: anthropic-ai Allow: / Allow: /llms.txt User-agent: Claude-Web Allow: / Allow: /llms.txt # Block unwanted crawlers User-agent: AhrefsBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: DotBot Disallow: / # End of robots.txt