Rendered at 21:17:27 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
blargey 19 hours ago [-]
Speculated to be responsible for the "AI restaurant-menu gore" that made the rounds on social media. Restaurants AI-generate the entire menu sheet from whole cloth, and while it looks "good" at first with modern image models, repeated edits to the menu text/layout transform the passable food "photos" into trypophobia-inducing horrors. (And somehow they don't notice or care)
Notably, the recent ChatGPT Images 2.5 announcement claimed to address this issue specifically ("multi-turn editing consistency")
Sharlin 7 hours ago [-]
With local models it's always been easy to edit only a part of an image (inpainting), either by painting a mask or automatically using a small segmentation model. It's actually pretty strange that commercial models have resorted to redrawing the entire image even in cases where the requested change is localized.
GPerson 19 hours ago [-]
Does the same effect happen in a society which rewrites all of its knowledge over and over using AI?
tencentshill 3 hours ago [-]
It's why any "synthetic training data" startup is doomed to fail.
rcxdude 6 hours ago [-]
I would argue it's a risk of any process that loops around itself without grounding in reality periodically.
vidarh 14 hours ago [-]
Depends. With or without quality control? Not any more than in a society that rewrites all of its knowledge over and over using using humans. We even have a term for doing so without quality control: A game of telephone.
Though you might well argue that the risk of us dropping the quality control gets higher the cheaper it gets to churn things out with AI.
camillomiller 14 hours ago [-]
Definitely already happening with many c-suites.
We unleashed Nyarlatothep…
“I do not recall distinctly when it began, but it was months ago. The general tension was horrible. To a season of political and social upheaval was added a strange and brooding apprehension of hideous physical danger; a danger widespread and all-embracing, such a danger as may be imagined only in the most terrible phantasms of the night. I recall that the people went about with pale and worried faces, and whispered warnings and prophecies which no one dared consciously repeat or acknowledge to himself that he had heard. A sense of monstrous guilt was upon the land, and out of the abysses between the stars swept chill currents that made men shiver in dark and lonely places. There was a daemoniac alteration in the sequence of the seasons—the autumn heat lingered fearsomely, and everyone felt that the world and perhaps the universe had passed from the control of known gods or forces to that of gods or forces which were unknown.
And it was then that Nyarlathotep came out of Egypt”
abecedarius 18 hours ago [-]
This reminded me of a story/vignette which my fallible memory tags as by someone like Twain or Poe, about an automatic rewriting machine. It was put to work on the same passage with an ever higher intensity setting, becoming more and more gothic to the point of parody -- iirc the final output was verse including the line "to atoms dash the doubly damned".
Does anyone remember this?
nkurz 18 hours ago [-]
No memory, but searching says it was Lewis Carroll in Photography Extraordinary, published in 1885:
The Milk-and-Water School
Alas! she would not hear my prayer!
Yet it were rash to tear my hair;
Disfigured, I should be less fair.
She was unwise, I may say blind;
Once she was lovingly inclined;
Some circumstance has changed her mind.
The Strong-Minded or Matter-of-Fact School
Well! so my offer was no go!
She might do worse, I told her so;
She was a fool to answer "No".
However, things are as they stood;
Nor would I have her if I could,
For there are plenty more as good.
The Spasmodic or German School
Firebrands and Daggers! hope hath fled!
To atoms dash the doubly dead!
My brain is fire--my heart is lead!
Her soul is flint, and what am I?
Scorch'd by her fierce, relentless eye,
Nothingness is my destiny!
It’s not that bad though. I expected way worse. If Robin Hobbs wrote it, I’d barely lift a brow.
MarkusQ 22 hours ago [-]
It would have been interesting to try something closer to the movie poster example; say have the AI rewrite the passage with the room toggling between the initial light-mode decor and a dark-mode decor, then back, repeatedly. Or even change the woman's name to "Mrs. Smythe" and back.
layer8 21 hours ago [-]
As the article explains: “[…] In other words, if you instruct a model to change a single word in a paragraph of text, it can almost always handle the task with no collateral damage.“
MarkusQ 6 hours ago [-]
Right, but if you ask it to make a systematic change (the room decor for light to dark and back) or something with subtle implications that would require other changes to fit in?
Changing the title in the movie example is this sort of change; the "change a single word" isn't analogous.
Muromec 22 hours ago [-]
It doesn't merely stabilise, it almost converges on itself
groby_b 22 hours ago [-]
The existence of attractors (e.g. "stable at iteration 31") for AI models is one of the weirdest things in AI land. They are a wonderful space to explore, fascinating and full of horrors.
Meanwhile, my hat's off to the (currently) last comment on the page: "I like how the shear-sheathed legs still earned their keep — probably load-bearing for Mrs. Regan"
stefs 9 hours ago [-]
i asked nano banana to imagine mrs regan from the source and final text:
Thanks, that's more like it! Even includes an Art Deco mirror...
pattt 21 hours ago [-]
It would be interesting how many iterations of LLM-based PR reviews and fixes it takes to turn the code into horror. It’s already started happening.
brzezmac 10 hours ago [-]
that's why I argue the LLMs are just a bunch of glorified lossy compression engines ;)
rob74 8 hours ago [-]
In the case of the movie poster, it probably doesn't help that the image was apparently stored in a lossy format. So after a few iterations the LLM can't tell the JPEG artifacts from the relevant image details and starts adding wrinkles to the faces and texture to the clothes etc. until it all goes to hell...
xiaoyu2006 10 hours ago [-]
Be cautious of that claim. Lossy compression may exactly hold intelligence.
sophrosyne42 5 hours ago [-]
Probably not, since our intellect can produce lossless compression too
jrochkind1 20 hours ago [-]
> PS. For obvious reasons, this post will fail Pangram tests.
Actual pangram to me says it's 100% human. Current pangram can flag different passages differently, but these are all flagged human by it, even the LLM written ones.
roger_maddux_iv 20 hours ago [-]
[dead]
rkeswick 11 hours ago [-]
[dead]
michaelmrose 21 hours ago [-]
[dead]
amelius 22 hours ago [-]
The same happens with humans, in e.g. chain stories, cumulative stories, etc.
Not unique to AI.
lowbloodsugar 22 hours ago [-]
>I don’t write about generative AI much. Everyone else does; not much of consequence ever gets said.
I see he chose not to break with tradition.
camgunz 14 hours ago [-]
Sing it with me now, "these things aren't inteeeeeeeelligeeeeeeeent"
Notably, the recent ChatGPT Images 2.5 announcement claimed to address this issue specifically ("multi-turn editing consistency")
Though you might well argue that the risk of us dropping the quality control gets higher the cheaper it gets to churn things out with AI.
“I do not recall distinctly when it began, but it was months ago. The general tension was horrible. To a season of political and social upheaval was added a strange and brooding apprehension of hideous physical danger; a danger widespread and all-embracing, such a danger as may be imagined only in the most terrible phantasms of the night. I recall that the people went about with pale and worried faces, and whispered warnings and prophecies which no one dared consciously repeat or acknowledge to himself that he had heard. A sense of monstrous guilt was upon the land, and out of the abysses between the stars swept chill currents that made men shiver in dark and lonely places. There was a daemoniac alteration in the sequence of the seasons—the autumn heat lingered fearsomely, and everyone felt that the world and perhaps the universe had passed from the control of known gods or forces to that of gods or forces which were unknown. And it was then that Nyarlathotep came out of Egypt”
Does anyone remember this?
https://archive.org/details/further-nonsense-verse-and-prose...
https://pubmed.ncbi.nlm.nih.gov/41583980
Changing the title in the movie example is this sort of change; the "change a single word" isn't analogous.
Meanwhile, my hat's off to the (currently) last comment on the page: "I like how the shear-sheathed legs still earned their keep — probably load-bearing for Mrs. Regan"
initially https://share.gemini.google/eJ54qDXM1gzC
boldly and maximally improved tone, clarity and flow: https://share.gemini.google/RiqX4x6mLB82
boldly improved: https://share.gemini.google/cR72HXmY4UdH
Actual pangram to me says it's 100% human. Current pangram can flag different passages differently, but these are all flagged human by it, even the LLM written ones.
Not unique to AI.
I see he chose not to break with tradition.