User-agent: SemrushBot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: Adsbot Disallow: / User-agent: MJ12Bot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: MauiBot Disallow: / User-agent: DotBot Disallow: / User-agent: MegaIndex.ru Disallow: / User-agent: BLEXBot Disallow: / # Advanced Wordpress User-agent: * Allow: /wp-admin/admin-ajax.php Allow: /*/*.css Allow: /*/*.js Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /readme.html Disallow: /license.txt Disallow: /xmlrpc.php Disallow: /wp-login.php Disallow: /wp-register.php Disallow: *?attachment_id= # Prevent Crawling of WordPress JSON API Endpoints User-agent: * Disallow: /wp-json/ Disallow: /?rest_route= #Block Parameters User-agent: * Disallow: *?s=* Disallow: *?p=* Disallow: *&p=* Disallow: *&preview=* #Block Feed User-agent: * Disallow: /feed/ Disallow: /feed/$ Disallow: /comments/feed Disallow: */feed Disallow: */feed$ Disallow: /?feed= Disallow: /wp-feed #Block Spam Directories User-agent: * Disallow: /trackback/ Disallow: */comments$ Disallow: */trackback Disallow: */trackback$ Disallow: /wp-comments Disallow: /wp-trackback Disallow: */replytocom= # Block archive.org bots User-agent: ia_archiver Disallow: / User-agent: archive.org_bot Disallow: / User-agent: ia_archiver-web.archive.org Disallow: / # Block Chatgpt User-agent: CCBot Disallow: / # Block Cart Page User-agent: * Disallow: /cart/ # Block Checkout Page User-agent: * Disallow: /checkout/ # Block Login Page User-agent: * Disallow: /login/ # Block Woocommerce Parameters User-agent: * Disallow: /*?orderby=price Disallow: /*?orderby=rating Disallow: /*?orderby=date Disallow: /*?orderby=price-desc Disallow: /*?orderby=popularity Disallow: /*?filter Disallow: /*?orderby=title Disallow: /*?orderby=desc Disallow: /*add-to-cart=* Disallow: /*add_to_wishlist=* Disallow: /*?paged=&count=* Disallow: /*?count=* # Block Ahrefs Crawler User-agent: AhrefsBot Disallow: / # Block Semrush Crawler User-agent: SemrushBot Disallow: / User-agent: SemrushBot-SA Disallow: / User-agent: SemrushBot-BA Disallow: / User-agent: SemrushBot-SI Disallow: / User-agent: SemrushBot-SWA Disallow: / User-agent: SemrushBot-CT Disallow: / User-agent: SemrushBot-BM Disallow: / # Block Moz Crawler User-agent: rogerbot Disallow: / # Block Majestic Crawler User-agent: MJ12bot Disallow: / # Block Xenu Crawler User-agent: Xenu Disallow: / # Block Scrapper Bots User-agent: DotBot Disallow: / User-agent: GiftGhostBot Disallow: / User-agent: Seznam Disallow: / User-agent: PaperLiBot Disallow: / User-agent: Genieo Disallow: / User-agent: Dataprovider/6.101 Disallow: / User-agent: DataproviderSiteExplorer Disallow: / User-agent: Dazoobot/1.0 Disallow: / User-agent: Diffbot Disallow: / User-agent: DomainStatsBot/1.0 Disallow: / User-agent: DotBot/1.1 Disallow: / User-agent: dubaiindex Disallow: / User-agent: eCommerceBot Disallow: / User-agent: ExpertSearchSpider Disallow: / User-agent: Feedbin Disallow: / User-agent: Fetch/2.0a Disallow: / User-agent: FFbot/1.0 Disallow: / User-agent: focusbot/1.1 Disallow: / User-agent: HuaweiSymantecSpider Disallow: / User-agent: HuaweiSymantecSpider/1.0 Disallow: / User-agent: JobdiggerSpider Disallow: / User-agent: LemurWebCrawler Disallow: / User-agent: LipperheyLinkExplorer Disallow: / User-agent: LSSRocketCrawler/1.0 Disallow: / User-agent: LYT.SRv1.5 Disallow: / User-agent: MiaDev/0.0.1 Disallow: / User-agent: Najdi.si/3.1 Disallow: / User-agent: BountiiBot Disallow: / User-agent: Experibot_v1 Disallow: / User-agent: bixocrawler Disallow: / User-agent: bixocrawler TestCrawler Disallow: / User-agent: Crawler4j Disallow: / User-agent: Crowsnest/0.5 Disallow: / User-agent: CukBot Disallow: / User-agent: Dataprovider/6.92 Disallow: / User-agent: DBLBot/1.0 Disallow: / User-agent: Diffbot/0.1 Disallow: / User-agent: Digg Deeper/v1 Disallow: / User-agent: discobot/1.0 Disallow: / User-agent: discobot/1.1 Disallow: / User-agent: discobot/2.0 Disallow: / User-agent: discoverybot/2.0 Disallow: / User-agent: Dlvr.it/1.0 Disallow: / User-agent: DomainStatsBot/1.0 Disallow: / User-agent: drupact/0.7 Disallow: / User-agent: Ezooms/1.0 Disallow: / User-agent: fastbot crawler beta 2.0 Disallow: / User-agent: fastbot crawler beta 4.0 Disallow: / User-agent: feedly social Disallow: / User-agent: Feedly/1.0 Disallow: / User-agent: FeedlyBot/1.0 Disallow: / User-agent: Feedspot Disallow: / User-agent: Feedspotbot/1.0 Disallow: / User-agent: Clickagy Intelligence Bot v2 Disallow: / User-agent: classbot Disallow: / User-agent: CISPA Vulnerability Notification Disallow: / User-agent: CirrusExplorer/1.1 Disallow: / User-agent: Checksem/Nutch-1.10 Disallow: / User-agent: CatchBot/5.0 Disallow: / User-agent: CatchBot/3.0 Disallow: / User-agent: CatchBot/2.0 Disallow: / User-agent: CatchBot/1.0 Disallow: / User-agent: CamontSpider/1.0 Disallow: / User-agent: Buzzbot/1.0 Disallow: / User-agent: Buzzbot Disallow: / User-agent: BusinessSeek.biz_Spider Disallow: / User-agent: BUbiNG Disallow: / User-agent: 008/0.85 Disallow: / User-agent: 008/0.83 Disallow: / User-agent: 008/0.71 Disallow: / User-agent: ^Nail Disallow: / User-agent: FyberSpider/1.3 Disallow: / User-agent: findlinks/1.1.6-beta5 Disallow: / User-agent: g2reader-bot/1.0 Disallow: / User-agent: findlinks/1.1.6-beta6 Disallow: / User-agent: findlinks/2.0 Disallow: / User-agent: findlinks/2.0.1 Disallow: / User-agent: findlinks/2.0.2 Disallow: / User-agent: findlinks/2.0.4 Disallow: / User-agent: findlinks/2.0.5 Disallow: / User-agent: findlinks/2.0.9 Disallow: / User-agent: findlinks/2.1 Disallow: / User-agent: findlinks/2.1.5 Disallow: / User-agent: findlinks/2.1.3 Disallow: / User-agent: findlinks/2.2 Disallow: / User-agent: findlinks/2.5 Disallow: / User-agent: findlinks/2.6 Disallow: / User-agent: FFbot/1.0 Disallow: / User-agent: findlinks/1.0 Disallow: / User-agent: findlinks/1.1.3-beta8 Disallow: / User-agent: findlinks/1.1.3-beta9 Disallow: / User-agent: findlinks/1.1.4-beta7 Disallow: / User-agent: findlinks/1.1.6-beta1 Disallow: / User-agent: findlinks/1.1.6-beta1 Yacy Disallow: / User-agent: findlinks/1.1.6-beta2 Disallow: / User-agent: findlinks/1.1.6-beta3 Disallow: / User-agent: findlinks/1.1.6-beta4 Disallow: / User-agent: bixo Disallow: / User-agent: bixolabs/1.0 Disallow: / User-agent: Crawlera/1.10.2 Disallow: / User-agent: Dataprovider Site Explorer Disallow: / User-agent: Rogerbot User-agent: Exabot User-agent: MJ12bot User-agent: Dotbot User-agent: Gigabot User-agent: AhrefsBot User-agent: BlackWidow User-agent: ChinaClaw User-agent: Custo User-agent: DISCo User-agent: Download\ Demon User-agent: eCatch User-agent: EirGrabber User-agent: EmailSiphon User-agent: EmailWolf User-agent: Express\ WebPictures User-agent: ExtractorPro User-agent: EyeNetIE User-agent: FlashGet User-agent: GetRight User-agent: GetWeb! User-agent: Go!Zilla User-agent: Go-Ahead-Got-It User-agent: GrabNet User-agent: Grafula User-agent: HMView User-agent: HTTrack User-agent: Image\ Stripper User-agent: Image\ Sucker User-agent: Indy\ Library User-agent: InterGET User-agent: Internet\ Ninja User-agent: JetCar User-agent: JOC\ Web\ Spider User-agent: larbin User-agent: LeechFTP User-agent: Mass\ Downloader User-agent: MIDown\ tool User-agent: Mister\ PiX User-agent: Navroad User-agent: NearSite User-agent: NetAnts User-agent: NetSpider User-agent: Net\ Vampire User-agent: NetZIP User-agent: Octopus User-agent: Offline\ Explorer User-agent: Offline\ Navigator User-agent: PageGrabber User-agent: Papa\ Foto User-agent: pavuk User-agent: pcBrowser User-agent: RealDownload User-agent: ReGet User-agent: SiteSnagger User-agent: SmartDownload User-agent: SuperBot User-agent: SuperHTTP User-agent: Surfbot User-agent: tAkeOut User-agent: Teleport\ Pro User-agent: VoidEYE User-agent: Web\ Image\ Collector User-agent: Web\ Sucker User-agent: WebAuto User-agent: WebCopier User-agent: WebFetch User-agent: WebGo\ IS User-agent: WebLeacher User-agent: WebReaper User-agent: WebSauger User-agent: Website\ eXtractor User-agent: Website\ Quester User-agent: WebStripper User-agent: WebWhacker User-agent: WebZIP User-agent: Wget User-agent: Widow User-agent: WWWOFFLE User-agent: Xaldon\ WebSpider User-agent: Zeus Disallow: / User-agent: Rogerbot Disallow: / User-agent: Exabot Disallow: / User-agent: MJ12bot Disallow: / User-agent: Dotbot Disallow: / User-agent: Gigabot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: BlackWidow Disallow: / User-agent: ChinaClaw Disallow: / User-agent: Custo Disallow: / User-agent: DISCo Disallow: / User-agent: Download\ Demon Disallow: / User-agent: eCatch Disallow: / User-agent: EirGrabber Disallow: / User-agent: EmailSiphon Disallow: / User-agent: EmailWolf Disallow: / User-agent: Express\ WebPictures Disallow: / User-agent: ExtractorPro Disallow: / User-agent: EyeNetIE Disallow: / User-agent: FlashGet Disallow: / User-agent: GetRight Disallow: / User-agent: GetWeb! Disallow: / User-agent: Go!Zilla Disallow: / User-agent: Go-Ahead-Got-It Disallow: / User-agent: GrabNet Disallow: / User-agent: Grafula Disallow: / User-agent: HMView Disallow: / User-agent: HTTrack Disallow: / User-agent: Image\ Stripper Disallow: / User-agent: Image\ Sucker Disallow: / User-agent: Indy\ Library Disallow: / User-agent: InterGET Disallow: / User-agent: Internet\ Ninja Disallow: / User-agent: JetCar Disallow: / User-agent: JOC\ Web\ Spider Disallow: / User-agent: larbin Disallow: / User-agent: LeechFTP Disallow: / User-agent: Mass\ Downloader Disallow: / User-agent: MIDown\ tool Disallow: / User-agent: Mister\ PiX Disallow: / User-agent: Navroad Disallow: / User-agent: NearSite Disallow: / User-agent: NetAnts Disallow: / User-agent: NetSpider Disallow: / User-agent: Net\ Vampire Disallow: / User-agent: NetZIP Disallow: / User-agent: Octopus Disallow: / User-agent: Offline\ Explorer Disallow: / User-agent: Offline\ Navigator Disallow: / User-agent: PageGrabber Disallow: / User-agent: Papa\ Foto Disallow: / User-agent: pavuk Disallow: / User-agent: pcBrowser Disallow: / User-agent: RealDownload Disallow: / User-agent: ReGet Disallow: / User-agent: SiteSnagger Disallow: / User-agent: SmartDownload Disallow: / User-agent: SuperBot Disallow: / User-agent: SuperHTTP Disallow: / User-agent: Surfbot Disallow: / User-agent: tAkeOut Disallow: / User-agent: Teleport\ Pro Disallow: / User-agent: VoidEYE Disallow: / User-agent: Web\ Image\ Collector Disallow: / User-agent: Web\ Sucker Disallow: / User-agent: WebAuto Disallow: / User-agent: WebCopier Disallow: / User-agent: WebFetch Disallow: / User-agent: WebGo\ IS Disallow: / User-agent: WebLeacher Disallow: / User-agent: WebReaper Disallow: / User-agent: WebSauger Disallow: / User-agent: Website\ eXtractor Disallow: / User-agent: Website\ Quester Disallow: / User-agent: WebStripper Disallow: / User-agent: WebWhacker Disallow: / User-agent: WebZIP Disallow: / User-agent: Wget Disallow: / User-agent: Widow Disallow: / User-agent: WWWOFFLE Disallow: / User-agent: Xaldon\ WebSpider Disallow: / User-agent: Zeus Disallow: / User-agent: Rogerbot Disallow: / User-agent: Exabot Disallow: / User-agent: MJ12bot Disallow: / User-agent: Dotbot Disallow: / User-agent: Gigabot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: BlackWidow Disallow: / User-agent: ChinaClaw Disallow: / User-agent: Custo Disallow: / User-agent: DISCo Disallow: / User-agent: Download\ Demon Disallow: / User-agent: eCatch Disallow: / User-agent: EirGrabber Disallow: / User-agent: EmailSiphon Disallow: / User-agent: EmailWolf Disallow: / User-agent: Express\ WebPictures Disallow: / User-agent: ExtractorPro Disallow: / User-agent: EyeNetIE Disallow: / User-agent: FlashGet Disallow: / User-agent: GetRight Disallow: / User-agent: GetWeb! Disallow: / User-agent: Go!Zilla Disallow: / User-agent: Go-Ahead-Got-It Disallow: / User-agent: GrabNet Disallow: / User-agent: Grafula Disallow: / User-agent: HMView Disallow: / User-agent: HTTrack Disallow: / User-agent: Image\ Stripper Disallow: / User-agent: Image\ Sucker Disallow: / User-agent: Indy\ Library Disallow: / User-agent: InterGET Disallow: / User-agent: Internet\ Ninja Disallow: / User-agent: JetCar Disallow: / User-agent: JOC\ Web\ Spider Disallow: / User-agent: larbin Disallow: / User-agent: LeechFTP Disallow: / User-agent: Mass\ Downloader Disallow: / User-agent: MIDown\ tool Disallow: / User-agent: Mister\ PiX Disallow: / User-agent: Navroad Disallow: / User-agent: NearSite Disallow: / User-agent: NetAnts Disallow: / User-agent: NetSpider Disallow: / User-agent: Net\ Vampire Disallow: / User-agent: NetZIP Disallow: / User-agent: Octopus Disallow: / User-agent: Offline\ Explorer Disallow: / User-agent: Offline\ Navigator Disallow: / User-agent: PageGrabber Disallow: / User-agent: Papa\ Foto Disallow: / User-agent: pavuk Disallow: / User-agent: pcBrowser Disallow: / User-agent: RealDownload Disallow: / User-agent: ReGet Disallow: / User-agent: SiteSnagger Disallow: / User-agent: SmartDownload Disallow: / User-agent: SuperBot Disallow: / User-agent: SuperHTTP Disallow: / User-agent: Surfbot Disallow: / User-agent: tAkeOut Disallow: / User-agent: Teleport\ Pro Disallow: / User-agent: VoidEYE Disallow: / User-agent: Web\ Image\ Collector Disallow: / User-agent: Web\ Sucker Disallow: / User-agent: WebAuto Disallow: / User-agent: WebCopier Disallow: / User-agent: WebFetch Disallow: / User-agent: WebGo\ IS Disallow: / User-agent: WebLeacher Disallow: / User-agent: WebReaper Disallow: / User-agent: WebSauger Disallow: / User-agent: Website\ eXtractor Disallow: / User-agent: Website\ Quester Disallow: / User-agent: WebStripper Disallow: / User-agent: WebWhacker Disallow: / User-agent: WebZIP Disallow: / User-agent: Wget Disallow: / User-agent: Widow Disallow: / User-agent: WWWOFFLE Disallow: / User-agent: Xaldon\ WebSpider Disallow: / User-agent: Zeus Disallow: /