# Serving Markdown to AI agents from a cached Laravel site

An agent that sends Accept: text/markdown gets the HTML your CDN keeps for browsers. How Brewless routes it past the cache, and how a .md page avoids that.

## In short

- A CDN keeps one answer per address, not per Accept header, so an agent asking a cached page for Markdown gets the HTML that was kept for browsers.
- Every pull zone Brewless manages answers a GET or HEAD with Accept text/markdown with a 302 before the cache is looked in, so the agent reaches an address the edge treats differently.
- Without a Markdown address of your own, the agent is sent to the same page with brewless-markdown=1, which is never kept, so your application answers it and can negotiate on Accept.
- When your application answers the .md address of the homepage with Markdown, every deploy notices and the agent is sent to /page.md instead, which the edge keeps like any other page.
- A browser never asks for text/markdown, so the HTML your visitors get and the way it is cached do not change.

AI agents read websites for people, and HTML is a costly way to read a page: navigation, scripts and markup around a few paragraphs of text. So a site can offer the same page as Markdown, and an agent can ask for it with `Accept: text/markdown`.

On a Laravel app behind a CDN that keeps pages, that request usually gets the HTML anyway. This post explains why, what every pull zone [Brewless](/) manages does about it, and how one route in your application lets the edge answer agents without waking your container.

## Why does an agent get HTML when it asks for Markdown?

Because the edge keeps one answer per hostname and address, not per `Accept` header. Once the HTML of `/pricing` is kept, the next request for `/pricing` is answered from the cache, whatever it asked for, and never reaches Laravel to negotiate.

Content negotiation is something your application does. It reads `Accept`, picks HTML or Markdown, and answers. That works as long as every request reaches it. Behind an edge that keeps public pages it does not: the point of the cache is that most requests stop there.

The textbook fix is a `Vary: Accept` header, which tells a cache to keep a copy per value of `Accept`. The [edge cache documentation](https://docs.brewless.eu/running/edge-cache) is explicit about what happens with it here: the edge keeps one answer per hostname and address and does not look at `Vary`. And browsers send many different `Accept` headers, so a cache that did split on it would keep many copies of the same HTML.

There is a second problem. bunny.net has no edge rule that skips a page it already keeps: a cache time of 0 only helps on a miss. Brewless warms the edge after every release, so for a public page there is no miss to wait for.

## Sending the request to another address

If the request cannot be told apart by its header, it has to be given another address. That is what every pull zone Brewless manages does, with two edge rules that are there from the moment the zone is made:

| Rule | What it does |
| --- | --- |
| `brewless: markdown past the edge cache` | A GET or HEAD whose `Accept` names `text/markdown` gets a `302` before the cache is looked in |
| `brewless: markdown address never kept` | Anything with `brewless-markdown=1` in the query string gets a cache time of 0 at the edge |

By default the `302` goes to the same address with `brewless-markdown=1` added to the query string. The redirect keeps the query string the request already had, and the first rule never fires on a request that already carries the marker, so there is no loop.

That address is never kept, so the request reaches your application, with its `Accept` header. What it answers is up to you: Markdown when you have it, the HTML otherwise. `Accept` names a preference, not a requirement. Your application can ignore the `brewless-markdown` parameter.

You can see it from the command line. The agent has to follow the redirect, which `curl` does with `-L`:

```bash
curl -sL -H 'Accept: text/markdown' https://example.com/pricing
```

> [!NOTE]
> A browser never asks for `text/markdown`, so nothing changes for your visitors: their HTML is kept and answered by the edge as before. A browser that follows a link with `brewless-markdown=1` gets your HTML, and that is not kept either, so it never reaches the next agent.

You do not configure any of this in `brewless.yml`. A new pull zone gets both rules when it is made, and every deploy adds them to a pull zone made earlier, or puts one back when it was changed or switched off.

## The cost of the marker address

The marker solves the wrong answer, but it has a price. An address that is never kept is an address that always reaches the container. Every agent that asks for Markdown wakes it, and on Scaleway a container that is woken stays up for another 15 minutes.

That is not a bug in the rule. The marker address answers HTML to one client and Markdown to another, depending on who asks, so keeping it would serve the wrong one sooner or later. The only way to let the edge keep a Markdown answer is to give it an address that always answers Markdown.

## A Markdown address of its own

So the better answer is in your application: serve each page's Markdown on an address of its own, such as `/pricing.md`. That address answers the same for everybody, so the edge keeps it like any other page, and the next agent is answered without your container.

Every deploy asks your site once whether it answers the `.md` address of the homepage with Markdown: a `200` whose `Content-Type` says `text/markdown`. When it does, the redirect is set to go there instead: `/pricing` becomes `/pricing.md`. When it does not, the marker stays. For the homepage, `/`, that address is `/.md`, because the rule appends `.md` to the path.

In Laravel that is a route per page, or one for a whole group of pages:

```php routes/web.php
use App\Http\Controllers\HomeController;
use App\Http\Controllers\PageController;

Route::withoutSession(function () {
    Route::get('/', [HomeController::class, 'show']);
    Route::get('/.md', [HomeController::class, 'markdown']);

    Route::get('/{page}.md', [PageController::class, 'markdown']);
    Route::get('/{page}', [PageController::class, 'show']);
});
```

Register the `.md` route first: `/{page}` alone would also match `pricing.md`. Constrain `{page}` the way your own routes do. `Route::withoutSession()` keeps the session cookie off these answers: a page that sets a cookie is never kept at the edge, and that would undo the point of the separate address.

The controller method only has to say what it sends:

```php app/Http/Controllers/PageController.php
public function markdown(string $page): Response
{
    $markdown = $this->pages->markdown($page) ?? abort(404);

    return response($markdown, 200, [
        'Content-Type' => 'text/markdown; charset=UTF-8',
    ]);
}
```

Where the Markdown comes from is up to you. A site whose pages start as Markdown files already has it. A page built from Blade components needs a second, plain version, and that is a choice about what an agent should read, not only a conversion.

> [!TIP]
> Point to the Markdown from the HTML as well, with `<link rel="alternate" type="text/markdown" href="/pricing.md">` in the head. An agent that fetched the HTML first can then find the Markdown without guessing.

The check is made of the homepage only, once per deploy. So the `.md` address of the homepage is what switches the redirect for the whole pull zone. If only some pages have a Markdown twin, an agent asking for one of the others is sent to a `.md` address that does not exist. Give every public page its twin, or leave the homepage's out and keep the marker.

## Which one to pick

| | Marker address | `.md` address |
| --- | --- | --- |
| What you change | Nothing; negotiate on `Accept` if you want to | A `.md` route per page, the homepage included |
| Kept at the edge | Never | Like any other page |
| Wakes the container | On every agent's request | Only when the edge does not have it yet |
| Who picks it | The default | Every deploy, when `/.md` answers Markdown |

If agents rarely read your site, the marker is enough: the agent gets the right answer and you changed nothing. If they read it often, or your container is supposed to sleep most of the day, the `.md` routes are worth the work.

Brewless runs on Brewless. Its own marketing pages each have a `.md` twin, and the application sends a request that asked for Markdown on to it.

## What this does not cover

The rules only send a request on. They do not turn HTML into Markdown, and they do not make an application answer Markdown that never did. A site without any Markdown still gives the agent its HTML, now from the container instead of the edge.

They also say nothing about which agents send `Accept: text/markdown`. That is up to the agent. A site that wants to be found by agents that do not ask can still name its Markdown in the HTML, in `/llms.txt`, or offer tools with [WebMCP](/blog/webmcp-tools-in-laravel).

For the rest of what keeps a public Laravel site cached, from sessions on guest pages to [links with tracking parameters](/blog/laravel-tracking-parameters-edge-cache), see the [edge cache documentation](https://docs.brewless.eu/running/edge-cache#markdown-for-agents) and [why a container never scales to zero](/blog/laravel-container-never-sleeps).

## Frequently asked questions

### Why does an AI agent get HTML when it asks my Laravel app for Markdown?

Because a CDN keeps one answer per hostname and address. Once the HTML of a page is kept, the next request for that address gets it, whatever its Accept header says, and the request never reaches Laravel to negotiate.

### Does Laravel's Vary header not solve this?

Not at bunny.net as Brewless sets it up. The edge keeps one answer per hostname and address and does not look at Vary, so a page that answers differently per Accept header must not be kept under one address. Brewless gives the Markdown request an address of its own instead.

### What do I have to change in brewless.yml?

Nothing. Every pull zone Brewless manages gets the two Markdown rules when it is made, and every deploy adds them to an older one or puts them back when they were changed. Every deploy also checks once whether your application serves a .md address and picks the redirect to match.

### Will browsers or search engines see the Markdown?

Not by asking for the page. A browser never sends Accept text/markdown, so the redirect never fires for it, and its HTML is cached as before. A browser that follows a brewless-markdown link gets your HTML, which is not kept either.

### Does an agent have to follow the redirect?

Yes. The answer is a 302 to another address, so a client that does not follow redirects sees only that. With curl you add -L.

Source: https://brewless.eu/blog/markdown-for-ai-agents-laravel
