AI search readiness of Japanese web design agencies' own websites: 2,357 sites (November 2026)

We fetched the home page, robots.txt and llms.txt of 2,357 websites of Japanese web design agencies. 13.6% (321) had an llms.txt, 68.9% (1,624) had structured data (JSON-LD), and 1.3% (28 of the 2,158 with a robots.txt) blocked at least one AI bot in robots.txt.

Published 2026-11-10. Data aggregated 2026-11-05. Research by Nurevo AEO (Bestie.合同会社). Nurevo sells AI search optimization (AEO) tools and so has an interest in this topic.

Key findings

  1. 13.6% of Japanese web agency websites (321 of 2,357) have an llms.txt; 0.6% (14) have an llms-full.txt.
  2. 1.3% (28 of the 2,158 with a robots.txt) block at least one AI bot in robots.txt. Almost none turn AI crawlers away.
  3. 68.9% have structured data (JSON-LD), 58.3% have an Organization-type node, and 3.9% have FAQPage.

About the survey

  1. Population: the company websites of 2,756 web design agencies in Japan that we could confirm are agencies. This is not a sample of all business websites.
  2. Reached: the home page could be fetched for 2,357 (85.5%). All results refer to these 2,357 sites.
  3. Fetched: each site's home page, /robots.txt, /llms.txt and /llms-full.txt, once each (User-Agent: NurevoSurvey/1.0), in early November 2026, aggregated on 2026-11-05.
  4. How things were judged: structured data by parsing the home page's JSON-LD and counting @type values (JSON-LD that does not parse counts as broken). An llms.txt counts if it returns status 200 with non-HTML text longer than 20 characters. For robots.txt, a bot counts as blocked if the group that applies to it disallows the home page (/), using the longest matching rule.
  5. The same run also covered agencies outside Japan (20,720 sites in total, 15,657 reached). A comparison will follow in a separate report.

Results: main items

Basic SEO settings (HTTPS, meta description, robots.txt, canonical) are in place on more than 80 to 90 percent of sites. Newer items aimed at AI, such as llms.txt and FAQPage, are still in a minority.

Base: the 2,357 websites of Japanese web design agencies whose home page could be reached. robots.txt rows use the 2,158 sites with a robots.txt. Aggregated 2026-11-05.
ItemSitesShare
Served over HTTPS2,355 / 2,35799.9%
Built on WordPress2,166 / 2,35791.9%
Has a meta description2,139 / 2,35790.8%
Has a robots.txt2,158 / 2,35791.6%
robots.txt lists a Sitemap1,988 / 2,15892.1%
Has a canonical link1,988 / 2,35784.3%
Has Open Graph (og:title)1,909 / 2,35781.0%
Has structured data (JSON-LD)1,624 / 2,35768.9%
Has Organization-type structured data1,373 / 2,35758.3%
Has an llms.txt321 / 2,35713.6%
Has schema.org microdata184 / 2,3577.8%
Has FAQPage structured data92 / 2,3573.9%
robots.txt blocks at least one AI bot28 / 2,1581.3%
Has an llms-full.txt14 / 2,3570.6%

How robots.txt treats AI crawlers

Of the 2,158 sites with a robots.txt, 2.5% (54) mention an AI bot by name, and 1.3% (28) disallow the home page for at least one AI bot. Most blocks target training crawlers (GPTBot, Google-Extended, CCBot and similar). No site blocked the search crawlers OAI-SearchBot or Claude-SearchBot.

Base: the 2,158 sites with a robots.txt. Blocked means the group that applies to the bot disallows /.
BotSites blockingShare
GPTBot (OpenAI)271.3%
Google-Extended (Google)241.1%
CCBot (Common Crawl)241.1%
Bytespider (ByteDance)241.1%
anthropic-ai (Anthropic)221.0%
ClaudeBot (Anthropic)211.0%
Applebot-Extended (Apple)211.0%
ChatGPT-User (OpenAI)20.1%
PerplexityBot (Perplexity)10.05%
OAI-SearchBot (OpenAI)00%
Claude-SearchBot (Anthropic)00%
Googlebot (Google)00%
Bingbot (Microsoft)00%

llms.txt

321 of 2,357 sites (13.6%) have an llms.txt. Among them, the median llms.txt is 96 lines long. 14 sites (0.6%) also have the full-text llms-full.txt.

Types of structured data

Of the 1,624 sites with JSON-LD, 1.7% (27) include JSON-LD that does not parse. WebSite, Organization and BreadcrumbList are the most common types; PostalAddress appears on 16.3% of sites, LocalBusiness on 5.3% and FAQPage on 3.9%.

@type values found in the home page's JSON-LD, nested ones included. Base: 2,357 sites.
@typeShare
WebSite62.4%
Organization53.7%
BreadcrumbList43.0%
ImageObject38.1%
WebPage34.1%
SearchAction28.6%
PostalAddress16.3%
Person10.1%
ContactPoint6.4%
ProfessionalService5.6%
LocalBusiness5.3%
FAQPage3.9%

Six-item check

We counted how many of six items each site meets: (1) has an llms.txt; (2) has JSON-LD; (3) has an Organization or LocalBusiness type; (4) has a robots.txt that blocks none of GPTBot, ClaudeBot, PerplexityBot and OAI-SearchBot; (5) has a meta description; (6) has a canonical link. This is a simple count made for this survey, not Nurevo's 0–100 product score. The average is 4.0 items; 10.5% meet all six.

Base: 2,357 sites.
Items metSitesShare
0401.7%
11164.9%
225410.8%
336915.7%
436015.3%
597141.2%
624710.5%

Citing this survey

You may quote and republish the figures and tables freely, as long as you credit "Nurevo AEO survey (nurevo.jp)" and link to this page. Sentences ready to quote:

  1. 13.6% of 2,357 Japanese web design agency websites had an llms.txt file (Nurevo AEO survey, November 2026).
  2. Only 1.3% of Japanese web design agency websites (28 of 2,158 with a robots.txt) blocked an AI bot in robots.txt (Nurevo AEO survey, November 2026).
  3. 68.9% of Japanese web design agency websites had JSON-LD structured data on the home page, but only 3.9% had FAQPage (Nurevo AEO survey, November 2026).
  4. 91.9% of Japanese web design agency websites run on WordPress (Nurevo AEO survey, November 2026).

Limitations

  1. Only the home page was examined. Structured data on inner pages, and the state of the site as a whole, are not included.
  2. robots.txt was judged for the home page (/) only. A site that blocks only some directories counts as not blocking. Blocking by other means (firewalls, CDN bot protection) was not measured.
  3. The population is web design agencies' own websites, not businesses in general. The list of agencies is our own and does not cover every agency in Japan.
  4. One fetch per site: the 399 sites whose home page could not be reached (temporary errors included) are excluded. For llms.txt we checked only whether it exists, not its quality.

Our reading (opinion)

What follows is our opinion, not a finding. Agency websites have their basic SEO settings largely in place, and very few turn AI crawlers away. Items that help AI read a company's information, such as llms.txt and FAQPage, are still on only some sites. Adding them does not guarantee appearing in AI answers (in another of our studies, on-site signals showed no relationship with appearance), but we think they are worth checking as a basis for AI reading a company's facts correctly.

Check your own or your clients' sites

The free check shows llms.txt, structured data and AI crawler access in about a minute. On WordPress, the Nurevo WebMCP plugin publishes an llms.txt and structured data. Agencies that want to roll it out across client sites can see the partner program.

FAQ

How common is llms.txt on Japanese web agency websites?

In this survey, 321 of 2,357 websites of Japanese web design agencies (13.6%) had an llms.txt (November 2026).

Do many agencies block AI crawlers in robots.txt?

Few do. Of the 2,158 sites with a robots.txt, 28 (1.3%) blocked at least one AI bot. None blocked the search crawlers OAI-SearchBot or Claude-SearchBot.

Do these figures apply to business websites in general?

Not necessarily. The survey covers web design agencies' own websites, and only their home pages.

Can I use these figures in an article?

Yes. Credit "Nurevo AEO survey (nurevo.jp)" and link to this page, and you may quote and republish them freely.