Provenance - Vincent's Substack
Vincent's Substack
SubscribeSign in
Provenance<br>A survival toolkit for an AI dominant information landscape<br>May 21, 2026
Share
I’ve encountered a few sobering moments in the comment section lately. Not the moments where I realize no one else has noticed what I deem to be obviously AI-generated content. But the ones where I’m made aware I’ve been deceived, only through help from commenters more vigilant than I. The physical senses were never perfect arbiters of authenticity online, but that gap is widening at an accelerating rate. The unfortunate truth is your grandma, someone’s teen nephew, and I are increasingly likely to be deceived as AI sharpens its understanding of reality.<br>Unless, we follow a path that leads to an ecosystem-level adoption of provenance .<br>I’m sure others have no trouble forecasting the potential harms, but let’s touch base on some dangers first.<br>A spectrum of harms: personal to societal
New-age catfishing<br>Who said AI users can’t be creative? Men are flooding TikTok, X, and Lord help them, Facebook with thirst traps of themselves posing as Gen Z women. Many grifters funnel the captured audience from these free, maximally engaging platforms over to Discord and Telegram where “premium” content is served to paying customers. I’ve seen “How to” guides going around on X to streamline this process. It’s a crude yet effective demonstration of the deception enabled by photorealistic generation.<br>Weaponizing (lazy) curiosity<br>Topical research routinely circulates beyond academia and onto mainstream platforms, informing the interested public of public matters. An engaged citizenry helps a democracy function, but a new attack gains appeal as science becomes increasingly politicized. The image below contains fictitious data depicting that increased water intake remedies Alzheimer’s disease. This was fabricated with a single prompt. The dense nature of research papers leads readers to visual aids for digestibility. This tendency lends itself to exploitation for those interested in quick takeaways, such as myself. The authority of believable microscopy imaging doesn’t help. Attackers of this kind would sidestep journals and peer-review in favor of publishing straight to social media. Note, this is a speculative future harm for now but precaution here is sensible.
Ruslan Rust@rust_ruslan
Can water intake prevent Alzheimer’s disease? No. This is fully AI-generated… but the data below could easily pass as real.
The new ChatGPT image model is truly impressive, but I think it poses a real risk for scientific integrity in future.
For example, I could just generate
4:44 PM · Apr 24, 2026 · 368K Views
55 Replies · 200 Reposts · 871 Likes
Coordinated campaigns<br>Adversarial state actors are using new-age deepfakes to create fake but plausible people. North Korea has been carrying out an IT worker infiltration conspiracy into Fortune 500 companies for years now, but advancements in generative AI allow new scale for such attacks. The number of companies that hired these operatives increased 220% within one year, amounting to 320 companies in that period alone. Once infiltrated, these operatives conduct revenue-generating activity on behalf of the regime, exfiltrate sensitive data, and in some instances have held companies hostage with their own data until ransom demands were met. The fraudulent scheme uses AI to forge identities, stream real-time deepfakes, and manage day-to-day work once hired.<br>Potential destabilizing forces<br>A report from the IPIE (International Panel on the Information Environment) found widespread use of gen-AI in election interference during the global 2024 election cycle, though its impacts on election outcomes remain inconclusive.<br>Since the last election cycle, we’ve transitioned from surreal and often absurd image outputs to renderings nearly indistinguishable from reality. When you play out the trajectory of model capabilities, peddling a single fabricated image on the internet is the rudimentary case. Cohesive, longer-format video incorporating both convincing dialogue and ambient audio to depict candid moments will define the next generation of deepfakes.<br>Compounding this threat, a phenomenon coined as the Liar’s Dividend emerges. As misinformation erodes societal trust, public figures acquire a basis for denouncing real media as fake if they deem it harmful to their reputation. This plausible deniability grows stronger as mistrust intensifies.<br>Promise of C2PA: Content Credentials
In a comically red-handed moment, an investigative journalist captures a video of a back-channel deal between a regulator and an industry counterpart. The video circulates on Twitter overnight but by morning the two parties publish a statement invoking the Liar’s Dividend.<br>How can the journalist prove to others what they witnessed firsthand?<br>Data about data
Metadata is data about the underlying data or object itself.<br>There is already established protocol in secure...