If your organisation publishes text, images, or other content online, you probably already have an opinion on whether AI companies should be allowed to mine it for training data. Since 2019, EU law has given rightholders a formal way to say no: the text and data mining opt-out under Article 4 of the DSM Directive. Most publishers who have set it up treat the job as done once a robots.txt line or a TDMRep tag is live on the site. That is the easy half of the job. The hard half, the one that actually decides whether the opt-out holds up in a dispute, is proving when it was live and what it said.
What Article 4 actually gives you
Article 4 of the DSM Directive creates an exception allowing reproduction and extraction of lawfully accessible works for text and data mining, without needing a licence, in a broad range of cases. Article 4(3) then draws the line that matters to rightholders: the exception applies only if the rightholder has not expressly reserved their rights, and for content made publicly available online, that reservation has to be expressed "in an appropriate manner, such as machine-readable means." In plain terms, a written notice on a page somewhere is not enough. The reservation needs to be readable by the crawlers and pipelines that do the mining, which is why most implementations rely on a robots.txt directive, an HTML meta tag, or the TDM Reservation Protocol (TDMRep) developed for exactly this purpose.
Setting that signal is a one-time technical task. Keeping proof of it is not, and that second part is where most rightholders stop.
Why the opt-out now has teeth, and why that raises the stakes
For a few years, Article 4(3) sat mostly as a theoretical protection. That changed with the EU AI Act. Providers of general-purpose AI models placed on the EU market are now under a direct obligation, set out in Article 53 of the AI Act, to put in place a policy for complying with EU copyright law, and specifically to identify and comply with reservations of rights made under Article 4(3) of the DSM Directive. That obligation applies regardless of where the model provider is based or where the actual training took place, as long as the model is placed on the EU market.
This changes the opt-out from a passive legal position into something both sides now have to actively account for. A model provider has to show it checked for and respected your reservation. You, as the rightholder, may need to show that the reservation was in place, in the correct form, before the content was collected for training. Both of those are questions about a point in time, not about the current state of your site. A robots.txt file only ever shows what is there today. It says nothing about what it said six months ago, when a particular crawl may have taken place.
Where the evidentiary gap actually bites
Three situations come up repeatedly once you look past the initial setup:
- A site redesign or CMS migration overwrites the robots.txt file, and the old opt-out directive is gone with no local trace of when it was removed or what it originally said.
- A dispute turns on timing. A model provider claims the training run predates your reservation; you believe it does not. Without a dated snapshot of your opt-out signal, this becomes a matter of who is more convincing rather than what is provable.
- The opt-out changes over time, for instance moving from a blanket robots.txt disallow to a more granular TDMRep policy that reserves rights for AI training while still permitting search indexing. If a crawl happened between two versions, only a dated version history tells you which policy actually applied at that moment.
None of these are hypothetical edge cases. They are the ordinary consequences of running a live website for more than a few months. A machine-readable opt-out solves the "can a machine read this" problem. It does nothing for the "can you prove this was true on a specific date" problem, and that second question is the one that actually decides a dispute.
What a dated record needs to do
A defensible opt-out record needs three things a live robots.txt file cannot give you on its own: a snapshot of the exact file or tag as published, a timestamp from a source you do not control yourself, and a way for a third party, including one outside the EU, to verify both without taking your word for it. Sealing a dated hash of your robots.txt, TDMRep declaration, or HTML meta tag every time you publish or update it builds that record incrementally, at the moment each version goes live rather than after the fact when it is too late to reconstruct.
This is also where the value of the underlying proof matters beyond the EU. A dispute about training data or a licensing negotiation with a model provider will not always play out in front of an EU authority. It might land with a US-based AI lab's legal team, an international arbitration panel, or a court in a jurisdiction that has never heard of the DSM Directive. A record that is only as strong as "trust our server logs" does not travel well into that conversation. A qualified, independently timestamped record does, because its evidentiary value does not depend on which jurisdiction is asking. That is the same logic that applies to sealing any document you may need to defend outside the country where you created it: proof that stands up globally is worth more than proof that only works at home.
What to keep in the record
A useful sealed snapshot does not need to be complicated. At minimum, keep the full text of the robots.txt directive or TDMRep declaration exactly as published, the URL or subdomain it applied to, and the date it went live. Larger organisations running multiple properties, regional subdomains, or several language versions of the same site should seal each one separately, since a reservation set on one domain does not automatically extend to another, and a crawler that ignored a subdomain's policy will not be excused by a reservation that only existed on the main site. If your legal or compliance team is asked to produce evidence during an audit or a licensing negotiation, having a dated file for each property beats reconstructing a history from cached pages and hope.
It is also worth keeping the record even after you change or remove an opt-out. If you later decide to license your content for AI training instead of blocking it, a dated history showing exactly when the reservation was withdrawn protects you from the opposite problem: a claim that you were still opted out after you had, in fact, granted permission.
Building this into a normal publishing habit
The practical fix is smaller than it sounds. Treat your TDM opt-out signal the same way you would treat a contract or a dated design file: seal it at the moment it changes, keep the sealed record somewhere separate from the live site, and repeat the process every time the policy is updated rather than only once at launch. For organisations that publish datasets, model documentation, or licensing terms alongside their content, the same discipline applies to that material too. Our page on documenting AI training-data provenance covers how to build a dated, verifiable record across an entire dataset lifecycle, not just the opt-out signal at its edge.
Article 4(3) gives you the right to reserve your content. It does not give you the evidence that you exercised that right at a particular time. Build the dated record yourself, before you need it, and the opt-out actually means what it says.
Start sealing your TDM opt-out signals and dataset records with a dated, independently verifiable timestamp, so the proof is ready before any dispute forces you to reconstruct it.
