Advertisement
paddylandau

Barebones robots.txt for a basic WordPress site

Sep 20th, 2024
195
0
321 days
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Robots 0.26 KB | Source Code | 0 0
  1. ########    All user agents
  2.  
  3. User-agent: *
  4.  
  5. # Default WordPress entries.
  6. Disallow: /wp-admin/
  7. Allow: /wp-admin/admin-ajax.php
  8.  
  9. Disallow: /*?   # Disallow all parameterised URLs, because they end up on canonical pages.
  10.  
  11. Sitemap: https://example.com/sitemap.xml
Tags: robots.txt
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement