Provenance and censorship

They are not two sides of the same coin. But they are interlinked.

“What is real?” It’s no longer a mere philosophical question. Diffusion models can create photorealistic fakes of real people, and pitch-perfect audio replicas. Soon diffusion transformers will generate photorealistic fake videos … and the chattering classes will really freak out.

Mostly they’ll overreact. This ability has long existed; it was just restricted to Hollywood studios. Furthermore, the dread demon Disinformation is far more a demand problem than a supply problem. But at the same time, it's disingenuous to pretend that the ability to fake recorded reality at scale is anything less than a big change with significant repercussions. 

Not because people will believe fake videos. That’s a problem, sure. But people disbelieving real videos is a much larger one; belief in lies is nowhere near as bad as the extermination of truth. As such, we will need a reason, and a way, to believe in The Content Of The Future; we will need guarantees of a video's provenance.

Fortunately, a solution exists! Unfortunately, I'm not thrilled by it.

It's simple, really; camera companies — meaning, in practice, Apple/iPhone and Google/Android, plus the barely relevant long tail of cameras for weirdos and pros — will have to attest shots as real at the moment of capture. The very first thing that will happen to a video after it is shot is that Apple/Google will hash its file, sign that hash, and provide an API endpoint anyone can (anonymously) use to mathematically verify the video in question was not AI-generated or -modified.

(Minor technical details: if/when you edit or process the clip, you store in metadata a pointer to its original source file, along with a record of where/how you edited it, so anyone can reconstruct that process & verify if any fakery was added. Apple and Google can—and, speaking as a former Android/iOS dev, probably will—use different hashes, APIs, and metadata. This is not a 100.00% solution, but will get us back to today’s “reasonably presume a video is real without strong evidence otherwise.”)

Before you interject “No! Never! We need a decentralized solution, not one beholden to Big Tech megacorps!”, let me assure you I too would like that… but how? Again, you have to do this the moment the file is saved, at the OS level. If you delay its provenance so other apps can get their grubby mitts on it, the whole system is for naught.

This solution replaces “you can't trust anything is real” with “we all trust Apple and Google, a handful of other camera companies, and maybe someday some regulatory oversight, to be arbiters of whether anything is real.” …This is worrisome! ...But still a whole lot better than the alternative! As such, while it’s certainly not a perfect solution, it seems inevitable. (I wouldn't be shocked if one or both of Apple/Google are quietly doing, or at least testing, all this already.)

In addition to provenance for non-AI-generated work, because of our need for truth, we will also want it for AI creations, fueled by a far more compelling human emotion; the need to know who to blame. After the blame will come the calls for censorship…

…no, wait, my bad, silly of me to think that, really. Of course the calls for censorship came first. Within the last week India has called for preemptive licensing of all LLMs deployed in that nation (requiring “explicit permission of the government”), and a Microsoft engineer has called for OpenAI’s DALL-E 3 to be withdrawn and preemptively censored because — I am not making this up —

I'm honestly flabbergasted that anyone in the modern era can look at a tool (be it a diffusion model, Photoshop, Etch-A-Sketch, or pen) capable of creating such images and immediately demand that it be banned. Again I underestimated the primal human urge to angrily forbid expression. In fairness that engineer eventually watered his stance down from ‘shut down the service completely’ to ‘rerate that app as not-for-children,’ which, fine — but only after his initial demand was refused! I had thought, as a general rule, “if you wouldn't be shocked to see it on Netflix, don’t try to preemptively censor it from all computer screens” would be fairly common sense. Again I overestimated humanity.

India's approach is subtler, and encourages self-censorship, and is doomed and pointless, and has interesting historical parallels. I've written before about how LLMs somewhat recapitulate the history of the Gutenberg Press, civilization's last 100x step function in text generation. Well, shortly after Gutenberg was widely deployed, the authorities — the Fifth Lateran Council of the Catholic Church, then — declared that no book could be printed without their advance approval, on pain of harsh penalties. Sound familiar?

“OutRAGEous!”

(As an aside, I wrote this on an plane, and had a moment of ‘I am in a chair ten klicks high in the sky, moving at nearly 1000 km/h, communicating through the ether to ask a silicon-based thinking machine we don’t fully understand to create a new work of art for me to illustrate my essay. Truly we live in the days of miracle and wonder.’)

The Church also maintained a banned books list, but that's much less subtle and and effective than self-censorship seeded by the knowledge your work must be approved before can be published. As Ada Palmer writes in her superb essay Tools for Thinking About Censorship, "the majority of censorship is self-censorship, but the majority of self-censorship is intentionally cultivated by an outside power.."

Thankfully, most censorship is also doomed to ignominious failure. The 16th century Church were correct that Gutenberg was an existential threat to their power, but their attempts to control it failed nonetheless. India's attempt to control LLMs, and Western censors' attempts to forbid content you wouldn't blink twice at on Netflix, will be less effective yet. I suspect they know this already; they just want to be seen as ̶p̶o̶i̶n̶t̶l̶e̶s̶s̶l̶y̶ ̶p̶o̶s̶t̶u̶r̶i̶n̶g̶,̶ ̶p̶r̶e̶e̶n̶i̶n̶g̶,̶ ̶a̶n̶d̶ ̶p̶o̶s̶i̶n̶g̶ doing something.

This could segue into a larger essay about how journalism, along with a sizable fragment of the population, has internalized the message It Is Known Tech Is Evil And All Things Tech Are Evil, to the extent that the contortions exerted to turn anodyne stories into "tech industry bad!" are often actually comical. To be clear, criticism of the tech industry is often warranted! But the mindset Everything Tech Is Definitionally Bad, which I don't think I'm overstating, jumped the shark to self-caricature some time ago.

(You could argue that of course I would say that, as someone who is now, technically, an AI startup CEO — more on that new thing anon — but I've been saying it for a long time, including when I was a journalist.)

I may inflict that larger essay on you sometime. In the interim, let me leave you with a rubric I would like to be banally uncontroversial, but, alas, seems otherwise; with respect to AI-generated content, we should err on the side of provenance vs. none, and/but way on the side of none vs. censorship.