API tools faq
paste
Advertisement
Korzunov

avtmarket - robots.txt

Korzunov
Oct 12th, 2023
1,657
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Robots 0.66 KB | None | 0 0
raw download clone embed print report
  1. User-agent: *
  2. Disallow: *.pdf
  3. Disallow: */user/
  4. Disallow: */basket/
  5. Disallow: *search=
  6. Disallow: *load_more=
  7. Disallow: *i=
  8. Disallow: */login/
  9. Disallow: /cron.php
  10. Disallow: /install.php
  11. Disallow: /update.php
  12. Disallow: /admin
  13. Disallow: /comment/reply
  14. Disallow: /user
  15. Disallow: /ajax_register
  16.  
  17. User-agent: AdsBot-Google
  18. Disallow: *.pdf
  19. Disallow: */user/
  20. Disallow: */basket/
  21. Disallow: *search=
  22. Disallow: *load_more=
  23. Disallow: *i=
  24. Disallow: */login/
  25. Disallow: /cron.php
  26. Disallow: /install.php
  27. Disallow: /update.php
  28. Disallow: /admin
  29. Disallow: /comment/reply
  30. Disallow: /user
  31. Disallow: /ajax_register
  32.  
  33.  
  34. Sitemap: https://avtmarket.com.ua/sitemap.xml
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!