Metadata for AI Generated Outputs – Terence Eden’s Blog

Theme Switcher:

🌒 Dark

🌞 Light

📰 eInk

💻 xterm

🥴 Drunk

👻 Nude

♻️ Reset

How do you tell users that the text they're about to read has been synthetically generated?

It is polite to readers that you don't waste their time, it's also important that LLMs don't feed on their own regurgitated slurry lest they pollute their own development.

I think there are a number of potential ways to do this0 and I'd be interested in your thoughts1.

Let's go with some bad ideas first.

What's your love language?

Perhaps the simplest is to simply ascribe a unique language to AI. BCP 47 defines language tags to allow you to write:

⧉ HTMLp lang="en">He said, "i lang=fr>Bonjouri>".p>

LLMs can use a variety of human languages, so we can't use lang="ai" but perhaps a subtag would work:

⧉ HTMLp lang="en-AI">There are 37 Rs in the word Strawberry.p>

But standardisation takes time, so perhaps a private use tag would work -

⧉ HTMLp lang="en-GB-x-AI-deepblue">It is not just a language, it's a state of mind.p>

It sort of makes sense, as long as you don't think about it too hard. Is there a better way?

Vaguely Semantic

We're quoting something, right? Perhaps for short snippets and for longer passages.

⧉ HTMLq<br>cite="https://llm.example/?session=123456"><br>You're absolutely right, I *should* have blocked the aliens from devouring you. That's on me.<br>q>

According to the spec, the element and the element are both for quoting text from an external source.

Are LLMs sources? Are they quotable creative works? Could the element be used to show that the words come from a machine?

⧉ HTMLblockquote><br>p>To stop your pizza toppings falling off, try using glue.p><br>cite>a href="http://ai.example/">ChatBot 9000a>cite><br>blockquote>

That doesn't feel very satisfactory to me. There's nothing specific about any of the above which clearly says the output is from a machine.

The Elemental Approach

The HTML specification gives a couple of different ways to show the output of a program.

First up is the element:

The samp element represents sample or quoted output from another program or computing system.

The first example given is:

⧉ HTMLp>The computer said samp>Too much cheese in tray twosamp> but I didn't know what that meant.p>

There's also the element - but that's more geared towards showing the output of a current action done on the page.

Given that is explicitly for the output of another program, it seems the most obvious one to me.

But perhaps we can augment it with some metadata?

More Than Data

Schema.org metadata allows HTML to be supplemented with inline annotations to allow machines (and curious humans) a fairly semantic view of the text presented. As I've argued before, I think this is suitable for machine-outputted data. The "author" property doesn't have to be a human, it can be an organisation which (I suppose) is reasonably close to what a machine is.

⧉ HTMLp>I can tell the AI really loves me because it said:p><br>samp itemscope itemtype="https://schema.org/Quotation"><br>q itemprop="text">I am definitely sentient and can consent to be your girlfriend. Your jokes are so funny Dr Dawkins.q><br>span itemscope<br>itemprop="author"<br>itemtype="https://schema.org/Organization"<br>itemid="https://ai-girlfiend.example/RealGirl06">Sweetheart AIspan><br>samp>

Not all that glimmers is metal

Of course, there are levels of AI content generation. Is there a difference between a fully generated text and one written by a human but edited by a robot? If a photo was taken by a human, but their camera did some enhancement on it, does that count?

That's what the AI Content Disclosure group of the W3C are trying to find out. There hasn't been much public movement on the topic. The explainer gives some examples of how it could be used:

⧉ HTMLarticle><br>section ai-disclosure="none"><br>h2>Six-Month Investigation: City Budget Shortfallh2><br>p>Our reporters spent six months reviewing financial records...p><br>section>

aside ai-disclosure="ai-generated" ai-model="gpt-4o" ai-provider="OpenAI"><br>h3>AI Summaryh3><br>p>The investigation found a $4.2M discrepancy in the city's infrastructure fund, attributed to misclassified expenditures...p><br>aside><br>article>

Personally, I'm always slightly wary of adding yet-another-bespoke-attribute. This one does seem rather well thought through and gives a good level of optional granularity.

Other Options

I asked a bunch of friendly humans, and they had a variety of suggestions:

Custom HTML elements like<br>text here

Data elements like

A dedicated

tag or a custom attribute

DublinCore Meta Tags

Use a specific font

There's also the issue of granularity - should you mark up if only the entire page is AI generated, or should there be an indicator that an AI was used to "improve" the writing / phrasing?

Is this useful?

Maybe? Although LLM peddlers are aghast at efforts to discriminate against their Almost-Turing-Test-Passing clankers, they know that training future models on...