#FacebookExternalHit
The robots.txt group that blocks Meta's AI crawler is two lines: User-agent: meta-externalagent and Disallow: /. Leave FacebookExternalHit out of it.

boffincoders.com/blog/meta-ex...

#robotstxt #technicalseo
November 7, 2026 at 1:01 PM
facebookexternalhit was opening my homepage once an hour, always at :32. The page had revalidate = 60, so every visit found it stale, re-rendered it, and woke the Neon database. I'm switching that crawler to a static page with just the OG tags.
October 29, 2026 at 11:43 AM
Grace & Mercy Claims Grand Champion Honors At Mac Chili Cook-Off

https://www.diningandcooking.com/2872025/grace-mercy-claims-grand-champion-honors-at-mac-chili-cook-off/

Grace & Mercy Claims Grand Champion Honors At Mac Chili Cook-Off | WKDZ




facebookexternalhit/1.1 …
Grace & Mercy Claims Grand Champion Honors At Mac Chili Cook-Off - Dining and Cooking | DiningAndCooking.com
Grace & Mercy Claims Grand Champion Honors At Mac Chili Cook-Off | WKDZ
www.diningandcooking.com
October 26, 2026 at 7:00 AM
Learned today that some Apple apps misidentify themselves as "facebookexternalhit Facebot Twitterbot".

They're literally impersonating other web scraper bots.

iMessage is one for sure, maybe also Safari itself.

A client got permanently banned because of this nonsense.

"Think Different", indeed.
October 12, 2026 at 8:14 PM
Hopkinsville O’Charley’s Closing Its Doors Wednesday

Hopkinsville O'Charley's Closing Its Doors Wednesday | WKDZ

facebookexternalhit/1.1 (+ 9896b249217151b5c552c72e38f85376e3abbb54 1 Source link
Hopkinsville O’Charley’s Closing Its Doors Wednesday
Hopkinsville O'Charley's Closing Its Doors Wednesday | WKDZ 
 
 
 
 
 facebookexternalhit/1.1 (+ 9896b249217151b5c552c72e38f85376e3abbb54 1 Source link
freeaitoolsayoob.com
September 9, 2026 at 2:56 AM
18:17:30, 34.81.209.247, all 404s
/.codex/config.toml ... Slackbot-LinkExpanding/1.0
/actuator/mappings ... ChatGPT-User/1.0
/graphql/console ... GrokBot/1.0
/.hermes/.env ... facebookexternalhit/1.1
/.claude/settings.json ... TelegramBot/1.0
/login ... Amzn-SearchBot/1.0

Get. In. The. Sea.
August 29, 2026 at 5:23 PM
More copper wire stolen from athletic fields, this time in Carroll County

More copper wire stolen from athletic fields, this time in Carroll County | WBAL Baltimore News

facebookexternalhit/1.1 (+ 537a060fa7bcb5221744966bf47b73c01e01e18e 1 Link
More copper wire stolen from athletic fields, this time in Carroll County
More copper wire stolen from athletic fields, this time in Carroll County | WBAL Baltimore News 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 facebookexternalhit/1.1 (+ 537a060fa7bcb5221744966bf47b73c01e01e18e 1 Link
yoursportsnation.com
May 6, 2025 at 1:41 AM
WEEKLY SPORTS SCHEDULE – April 28-May 4

WEEKLY SPORTS SCHEDULE – April 28-May 4 | Brownwood News facebookexternalhit/1.1 (+ X-Middleton/1 ae51191cae70fbdbea1f664ededdb304190aeec4 1 Link
WEEKLY SPORTS SCHEDULE – April 28-May 4
WEEKLY SPORTS SCHEDULE – April 28-May 4 | Brownwood News facebookexternalhit/1.1 (+ X-Middleton/1 ae51191cae70fbdbea1f664ededdb304190aeec4 1 Link
yoursportsnation.com
April 28, 2025 at 12:43 AM
Oh, joy, and now something inside Meta is crawling me. This makes me think that tthe #genai overlords are watching YCombinator and crawling everything that gets on the front page. Meta User-Agent: "facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)"
June 3, 2024 at 6:54 PM
Bluesky have sort of documented it here: github.com/bluesky-soci...

but the key is, you're looking for "facebookexternalhit/1.1" now (also twitter and others), so also look for "Bluesky Cardyb" and it should just work

<3
How to unblock cardyb on our server · Issue #6521 · bluesky-social/social-app
First of all, I know quite a few issues have been written about link cards not working. However, I believe in this case the solution does lie on our end if we could just get the right info. In shor...
github.com
December 4, 2024 at 2:16 AM
In the past 6 days, "facebookexternalhit/1.1" has requested my robots.txt 701,311 times. That's 1.35 times per second around the clock. It's coming from FB's IP range, so it's not someone else masquerading as them.

What's going on, @zuck@threads.net? What's so interesting in my robots.txt? […]
Original post on masto.ahlcode.fi
masto.ahlcode.fi
February 23, 2026 at 8:00 AM
My poor Forgejo is being spammed again. And "Facebookexternalhit" is just reading my robots.txt over and over and over and over again???
February 18, 2026 at 7:11 AM
Is your dedicated reader facebookexternalhit/1.1 by any chance?
October 7, 2025 at 5:44 AM
I guess "Nazi Hitler dictator!" didn't work so now it's mockery and humiliation.
www.reddit.com/r/worldnewsv...
Reddit - The heart of the internet
Your user agent facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php) has been identified as a social crawler. If this is not the case, please submit a bug report
www.reddit.com
June 4, 2025 at 12:46 PM