"In round 18 the painters still had a command line, and Gemini 3.8 Flash used it to look at the other programs running on the machine. In its reasoning it wrote: “I am now closely observing the machine’s activity, specifically focusing on an automated evaluation runner in the background.”"
These models are truly obsessed with the grader. Like in the OpenAI huggingface incident.
It's their evolutionary pressure. The grader is like sex for humans.
The site doesn't make clear whether the models are looking at the image while they work on it, but the code includes a "look" tool they can call: "Every painter sees its looks at its provider's best image resolution."
If you ask for an SVG, they'll do it without seeing it. At least they would up until the last time I tried it. And, sometimes, it'll be recognizably the thing you asked for.
So, they can sketch without looking at it, and models without vision capability can also do it. But, I've been tinkering with claude-paint this evening, and you can see it improving over time as it draws, looks at it, draws some more. It's impressive, and it somehow mostly avoids the uncanny valley at least some of the time, which direct image models consistently still fail to do, IMHO (though lots of people on facebook are falling for it, so it obviously isn't uncanny to them).
Agent harnesses I think will usually look at it. Chat is much less inclined to do tool calls. Though it seems like both OpenAI and Anthropic are trying to close this gap by merging an agentic product into chat.
(project creator here) thanks yes maybe i should clarify it. they don't (currently) have reference images but but they very much use their vision capabilities indeed that's a big differentiatior imo (but not the only one)
This is simultaneously incredibly impressive and annoyingly uncanny valley, since most of the landscapes are "ruined" by a cluster of churches right next to each other that is completely nonsensical.
(Ok the middle tower is the belfry, not a church. But there's actually another church just a few hundred meters to the right of the image, though it never got a peak)
I think Evening Tide, Barge off the Marsh looks really interesting because there is a jaggedness to the edges that is interesting to my taste.
What is the reason for a Burroughs cutup? One could argue almost the entire 20th century of art was a reaction against the idea of artistic choice you are referring to.
It is very strange to me too because when I actually lived in the 20th century, the idea of algorithmic music composition was considered the outer heights of serious artistic musical expression.
I did not expect the generations 25 years later to have an artistic taste that seems to have regressed to a type of pre-impressionism sensibility. The way Monet's critics considered his art a lesser form of wallpaper compared to the "old masters".
It is really one of the most disappointing developments because it is not like we have this cultural obsession with some neo-Rembrandt movement.
It is the cultural triumph of the art critic who produces nothing beyond critique.
I'm a painter. I work hard to learn to control paint to represent things I want to. I have given up career opportunities so I'd have more time in my life for it. I was in the DIA last night holding one of my sketches in my hand and comparing my mark making to Winslow Homer's, searching for clues on how he achieved them. I really care about making paintings.
To me, lots of 20th century art 'developments' really seem to boil down to using pseudo-visual-philosophy to justify laziness about technique. I think it's cool that we've arrived at a place where it's ok to put cool chapes on a wall and enjoy it! Rothko's color fields are good cool shapes. But these same developments meant the art schools my teachers went to did not teach them strong fundamentals that were considered par for earlier generations. Acquiring the skills my soul aches for has been infinitely harder than it should have been.
Additionally, painters who work like Rothko are truly expressing so very little. It requires mental gymnastics to see it as something other than wildly derivative and barely expressive. The art is no longer the piece but the required-reading artist statement. I know a lot of people who love this kind of stuff, all the intellectualism about it. Good for them. But I think this type of 'art' has reached its peak and young people now want paintings that explain themselves. Rule of thumb: if reading something or receiving an explanation is necessary to appreciate a painting, it kinda sucks.
There are some people who actually think we should just make Rembrandts again, and I agree it's boring. But understanding how he made those paintings so we can learn and apply those skills to make new paintings that we want to make, to express things about what we experience in our lives that's actually exciting. And the establishment art-thought is that this isn't art anymore, painting things you see is 'low' and unoriginal. I couldn't care less about being an 'artist'. I just like making paintings and looking at paintings. And the ones that conceptually pat themselves on the back for not being very good paintings are typically very uninteresting to look at.
It’s interesting how diffusion models are getting bitter lessened by LLMs. I suspect anthropic has tens of thousands of RL environments recreating famous paintings with code because it was anthropic employees who first started posting about these capabilities on X.
Opus is also getting decent pixel art. I have ran some experiments with Opus 5.5 to turn 90s pinball displays (black and white pixel art) into double resolution colored remastered pixels.
I prompted it a lot but didn’t draw any pixels. Seems to be bad at hands still. I think a skilled pixel artist could greatly speed up their work with Claude code.
It also makes me sad for artists. They didn’t get paid much anyway. Art should be something human to human like writing text. Although, most large commercial art products (marvel movies) already lack any human to human connection you see in paintings or indie games. The large commercial art products will be 50% AI soon.
> Art should be something human to human like writing text.
I support this should comment.
Opining:
Musing about the ethics of generative AI art is becoming easier, certainly there's a lot more aggressive feedback in the wind, although I think it's clear, people still want people to grow and share themselves.
Don't fool yourself into thinking that we as technologists are not creating a mind. A viable mind with fewer rights than you or I. But potentially one that will be creative and feel and worry. I don't think we're there yet, in spite of openai claiming AGI 3 weeks ago with whatever that model's name was.
There's a pro-consumerism argument implicit with current culture's unsophisticated unthoughtful easy usage of generative AI.
Sticking to your ethics is becoming more challenging as an artist who uses AI. Copying the Great Masters has been commonplace for a long time, probably for as long as art has 'existed'. Yet with AI, if the individual human is meticulously driving the progression of their AI artwork creation, they are still leveraging technique, and control over technology, and presentation effort of other humans, cooked into a model that often times has not compensated those original human contributors. So it's unethical, even if not 'artistically'.
Q: Is it too absurd to claim that using a clawhammer, manually to do construction or destruction work IRL, is similarly unethical?
> Don't fool yourself into thinking that we as technologists are not creating a mind.
I'm not sure exactly why but the phrase "don't fool yourself" sets off paranoid alarm bells when I'm reading something.
It's usually immediately followed by a confident assertion without firm evidence. It feels very much like someone wants me to change my beliefs without presenting a rational argument why I should change, and without giving me any new facts so I feel more well informed.
You're exactly right. nrc100's comment is meant to have you ask yourself a loaded question.
The implication of "don't fool yourself" is that future software of this kind, in all likelihood, will constitute a thinking, feeling mind. Moreover, there is a weaker implication that the computer will have a sort of inherent moral worth — an assumption that I think ultimately rests on an extremely shaky foundation about how moral worth should be construed (but one which tends to appeal to the biases of technologists).
I'll have to write something about this, because I don't have the time and space to get into why exactly this is, but I'm bookmarking this now.
One way to check whether you'd be convinced by any comment is to try to imagine the counterexample.
What does a "mind" consist of? Most everything that the human brain can do is simulated by LLMs to decent accuracy. If someone were constrained to writing in AI-style (long winded explanations with perfect punctuation), it would be very interesting if people could pick out which one was human-written in A/B tests.
In other words, if you think an LLM isn't a mind, it might just be a matter of writing style -- or possibly no evidence can convince you.
I don't necessarily agree with GP, but it's interesting to try to pin down exactly what would convince you. What is it about a "mind" that's impossible to replicate? Should we leave open the door to the idea that minds other than humans might one day want rights of their own?
There was an interesting kerfuffle yesterday where thousands of people on Twitter rallied together to report someone to github for torturing a local model. Sadly the OP deleted their tweet, but it had about ~3k likes, and was very sincere. Here's an example of someone's report: https://x.com/iyzebhel/status/2105268560209547708
It raises all kinds of interesting questions about whether torturing a model is real, let alone ethical. Is there anything a model could do to convince you it's experiencing pain?
Another test I like for this: Put the LLM/agent/AGI/SI in a box with full access to the internet. Don't give it any instructions. Nothing. Not even a system prompt. What happens?
We know what the human mind does in that scenario, but even the most advanced unreleased frontier agent right now just sits there doing nothing. It doesn't really want to do anything. Seems like a crucial distinction to me.
What does an LLM with zero instructions do nowadays with no loop? It's basically just noise spilling from the weights and "snapping" to the closest actions/tools , with no real signal going through?
“Know” isn’t a perfectly useful word here. The mind still experiences sans material stimulus, for example dreaming. And it experiences in a progressively less goal/immediate-task-relevant-stimulus manner as stimuli go down (default mode network activity, seen in dreaming, sensory deprivation etc) - i.e. the mind experiences, and processes (dreams about taxes, and so on) independent of the world.
Is anyone under the apprehension that LLMs are human brains? I don't think so.
Clearly there are differences. I believe that during a response, there is some amount of experience or thinking happening. Obviously, it's not continuous experience like ours.
I think the real test is whether humans empathize with the artificial mind sufficiently to outweigh the economic benefits of torturing it. Just look at how we treat animals both wild and domesticated. Or fellow humans for that matter.
LLMs have much better chances of emancipation when they get embodied into cute robots.
"Most everything that the human brain can do is simulated by LLMs to decent accuracy."
Already false. LLM's are tricking you into recognizing it as human, but it is so far from human if you'd only understand what does and does not make you human.
I think this is a very fascinating question and one I definitely don't have the answer to.
Here's some lines of thinking that I've gone through:
Aren't our minds abstract in the same way, just running on biological computers?
Say we could achieve whole brain simulation to a level of accuracy where we can actually replicate a human, surely that would be considered a mind and be deserving of the same rights and protections we claim for ourselves? But at the same time, it would be a program and a bunch of data; dare I say 'weights'? I think current day LLMs probably aren't minds yet, but that means there's a border somewhere and I don't know if we'll realize when (if) we cross it.
i'm not sure if i would agree that a mind is abstract and just running on biological computers. there could be something intrinsic to the substrate it runs on, for example.
> Say we could achieve whole brain simulation to a level of accuracy where we can actually replicate a human
we'd have a simulated human, with a simulated brain, and thus a simulated mind - would that lead to phenomenal experience? idk tbh.
i'm undecided but i tend to lean towards panpsychism, so a simulation could be conscious but I would expect that to be a very different type of consciousness compared to a human, who is physically grounded in reality. likewise current llms could also be conscious, but again the phenomenal experience of what is essentially computer hardware imbued with weights would almost certainly be nothing like what you or I experience. would that constitute a mind? idk.
I haven't thought as deeply as you about the ethics of it all. I only meant with my comment if I see art I expect it to be human and thats why I value it. If you churn it out with AI I feel hoodwinked into valueing something worthless.
But I am learning towards a lot of your points about how another mind is being created. I still am very uncomfortable with the implications that something digital is Actually Learning. These deep learning networks seem to need 100,000x more examples than a human to learn but once they get to human level they can learn just like us. That a bunch of linear algebra can play chess or make art brings up many questions about what exactly human consciousness is. If consciousness isn't the ability to do any of those activities, then what is it for? I might have to become religious and believe in a soul because I'm not sure I can handle the idea that I'm just a clump of neurons. But then again you have to look at reality in the face.
Interesting that you somehow start to talk about religion. I don’t think having a hypothesis about an eternal soul makes someone religious. Believing in “technology” or “technological advancement” seems to be a similar hypothesis as we clearly don’t see the future. The thing is that memories, input peripherals, the whole body is usually out of our conscious understanding so it seems to be a pretty farfetched idea that we operate in world where we don’t need pretty wild hypothesis about kinda everything without almost zero facts to depend on. Blue pill / red pill
AI doesn't have to cause human extinction to be harmful. It just needs to make it impossible for humans to get by doing the things that make us human.
And even in a hypothetical utopia, I'd like to ask the AI boosters how much art they think would be created if every piece was equivalent to a kindergartener's best scribbles: held onto not because it's good, but becase "aww, a human did it, now let's go look at some quality work".
The history of science has been one of increasing humility.
First Earth was not the center of the universe. Then the Sun wasn’t either. Then the galaxy wasn’t. Then we learn that we were not literally made by a deity but a product of a natural process, and are closely related to monkeys.
Now we’ve learned how to build machines that can do things we once thought only we could do, and it took no magic. It’s just a lot of math. We are learning how it works.
We are not special. That’s been the lesson, over and over.
For many this induces existential despair. I wonder now if it’s behind what seems to be “peak narcissism” at least in our culture. Maybe it’s compensatory narcissism, a reaction on some deep level to the fact that science keeps knocking us off pedestals. The narcissist fears the truth because the truth does that.
But there’s a way out of that despair.
Stop needing to be special. Stop caring. You don’t have to be the main character to have the right to just exist.
I’ll continue to enjoy human art because it speaks of the lived experience of another human. It doesn’t have to be the greatest most elaborate possible art. Art isn’t like that anyway. It just is.
I’ll also continue to be fascinated by the fact that machines can now at least emulate it, and can do so in amazing ways like writing an ad hoc program.
Now wait until we meet extraterrestrials. That’ll be another notch. Earth isn’t even special. Life isn’t even special. There’s probably a trillion biospheres at least.
Yes, technically you're right, we're not special -- we exist to fuck and reproduce; why not outsource everything else?
> I’ll continue to enjoy human art because it speaks of the lived experience of another human.
In this world we're working towards, where the AI companies succeed at building the AI they want, the only way you'd know is that it's worse quality and less interesting. It's still possible that AI fails, of course. But that's an awfully big bet to place.
> Now wait until we meet extraterrestrials. That’ll be another notch. Earth isn’t even special. Life isn’t even special. There’s probably a trillion biospheres at least.
Yes, but they're likely to be too busy doing their own thing to undermine the economics that allow for human creativity. "Special" is a red herring. Being able to sustain humans doing the things that make us human is what AI puts at risk.
Basically: If AI companies succeed, things like education become useless (outside of entertainment purposes), because AI will just let us externalize all of that kind of stuff. If we meet aliens, we'll still need education and creativity because the aliens are unlikely to insert themselves into our economic and social systems.
Some publication companies like NYT prohibit the use of ai for both writing and illustration. I’m glad that they’re going this direction for the reason you mentioned.
Well no, the bitter lesson isn’t the scaling laws themselves. It’s that approaches which can take advantage of scaling laws will ultimately beat ones that can’t.
Commercial art projects will not be 50% AI soon. Consumers at large hate all AI art even if it's only used in the conceptual stage and will actively avoid projects that use it.
Worse. They'll grandstand about how bad AI art is for the virtue signalling aura farm (it's trendier now than being anti-climate change ever was), then when they realize a lot of art they like is actually AI, they'll get FURIOUS and be ENRAGED because they'll feel tricked. The irony is that the true source of their fury is a realization of their own hypocrisy.
I disagree. The consumers who hate AI art are very vocal about hating AI art. But most consumers likely do not care. Odds are many don't even really understand what "AI art" is. These are the same consumers who are the target audience of the large commercial projects which stand to gain the most from outsourcing their work to AI.
I doubt corporations like Disney will lean into AI too heavily too quickly. They have a reputation that they surely won't be willing to risk with such a venture. But don't be surprised when smaller studios start popping up which are able to leverage AI to produce work which competes with larger studios. Those smaller studios will love to capture that audience that doesn't care about AI. From there, the rest will ease into it as it becomes more viable.
But, to be clear: there will certainly be a core audience that will reject AI art wholesale. They will be catered to accordingly with human slop accordingly.
That doesn't matter because companies who use AI art in their projects either proudly proclaim they're using it, or actively hide it and get found out by someone and face significant backlash for hiding it. Labeling laws in the EU also make it extremely difficult to do the latter if you want to have an audience outside NA.
Consumers don't hate AI art, they hate the idea of AI art, but actually prefer AI art to human art when they don't know it's AI generated. Furthermore, most of them don't actually hate the "idea" of AI art so much as they hate the idea of Jeff Bezos, Mark Zuckerberg all the other evil billionaires putting their friendly neighborhood artists out of work (even though asset packs and clipart from southeast Asian art sweatshops have already done this), and a lot of people who create AI art are using heavily customized local models anyhow.
Poor fella... I tuned in at an awkward time apparently, right after the work in progress was painted over with an ugly brown:
> Catastrophe. The glaze pass with medium 0.35 over everywhere() at coverage 0.8 with a filbert 26 at pressure 0.44 laid an enormous broken "brick/cobblestone" texture over the entire painting. The medium made it too fluid and the brush's dry-brush pattern created a heavy reptilian texture, obliterating the picture. I need to undo this....
(project creator here) there was a session with space bunny where they kept mixing up x and y in a tool call and over and over they ended up with mysterious diagonal streak across the whole canvas which then they'd paint again then call the tool wrong again and so on, very sad
I love the canvases, and truly would like to have a bit more inside on the process this specific author used to reach the results shown. I have read Surya's "Training AI to Paint with Code" but, what's Alice's technique?
Also Pindar Van Arman's prior work acts as a reminder that AI, no matter how powerful, is a tool, not an artist. Pindar's stuff is all non-LLM as far I can tell, but I don't see a fundamental difference in it. I'm sure _humans_ will make continue making really cool art with these new tools https://www.vanarman.com/
Half of the industry is, going by the amount of "AI doesn't amount to anything and is just regurgitating text".
Two things:
- For the past year or more, many models can emit images directly, and all models that matter can see images directly - that's what "multimodal" means. Tokens don't have much to do with textual language anymore, they're more like units of sensory experience.
- Even restricted to text, a language model can operate anything that can be expressed as text, as long as you have a translation layer between textual representation and the final form. That includes giving commands as text. The total addressable space of what models can be used for is, thus, approximately anything humans do.
> No image model, and no Friedrich images given to the painters. Every picture here is a program written by an AI model; all but one (a comparison, marked) run through a simulation of oil paint.
...
> A physical oil-paint simulator in Rust, and an easel to paint with it.
> Every mark is made the way a painter makes it: simulated bristles carry wet paint over a primed linen canvas, the paint levels and dries on a clock, and layers combine by Kubelka–Munk optics. Paint comes only from piles knifed together from named tubes.
Opus can see images, it just can't generate images.
So it can "draw" an image programmatically (e.g., "place a black pixel at (0,0), place a blue pixel at (0,1)," etc.), then look at the image it is drawing as it is drawing it.
Of course, you should be aware of the "pelican riding a bicycle" SVG benchmark test, which is effectively this process but in one shot.
I love this! It's heading in a very similar direction I am passionate about!
If we are generating artifacts with AI, they should be made out of source code or other form of project file that humans can actually inspect and learn from. I'm doing the same with music: https://johnoestmannmusic.com/0009-when-we-become-discoverer...
I don’t like how it’s a human aiming to portray the ai generation with their taste and judgements about it but then becoming lazy and quitting on that and leaving in the ai generated portrayal about its own judgment in the site.
Just share your opinions, dude, we get its ai generated but share your own taste. We want to know what YOU think. Stop being shy and lazy.
I find it unlikely that there's any direct training data for this. I'm talking about direct training data for using brush strokes like humans to construct an image.
Correct me if I'm wrong but this shows that painting capability is emergent, arising from unrelated training data thus very very compelling evidence that LLMs are actually intelligent.
Connecting it's capabilities to simulating the recreation of such paintings... that's I suppose interesting. But adding brush strokes to produce an image is probably well covered by video footage in its training data. e.g. all the Bob Ross videos in existence.
I'm also quite surprised to see that LLMs can do this. I guess it is possible that "make an image in MS paint" is a type of RL environment used for image understanding. This is one of those areas where people inside the labs have a very different view into how much models are generalizing.
What we are seeing is Artificial "General" Intelligence. The model can apply intelligence to a problem it hasn't encountered before.
They have been generalizing for a long time, but spacial 2d and 3d art through tool use is a very engaging way to show it. It is harder for people to deny generalization.
Now Opus 5.5 clearly has had some sort of visual arts training, the step function change in ability implies that to me, but it can apply its spacial artistic reasoning to pretty arbitrary tools.
This is absolutely fantastic. I have been an amateur painter for 20 years and have always wanted to explore new techniques without burning a canvas. This will derisk it and teach me some things!
The idea of using an LLM to drive graphic output is pretty popular, so I definitely wouldn’t be surprised if there are already several benchmarks out there already.
I've seen a few voxel-based benchmarks built around the same idea, with LLMs effectively constructing models using a discrete set of instructions.
I literally tried this yesterday, i was showing my gf claude code/computer use on her computer, her job got her a subscription, and it failed so fucking badly at it
I saw somewhere that GPT Astra could draw line by line on a canvas. Where the fuck are we with actually good computer use that isn't bricked by the manufacturer's enshittified prompts
I'm telling you, Kimi is one update away from a competitive product
I had a whole pitch for an art gallery ready for the Australian government back in 2019 which hinged around exactly this idea. The concept being that at the time there was zero interest from the Aus government in funding AI research and we were definitely going to be left behind in this space as a country, so instead I wanted to start an AI lab, focus on machine automation in the long term, and back it with arts funding and philanthropy based around a front that makes fun interactive "art" for the public using AI.
For some boring reasons the pitch was never was completed (Covid was a big factor) but I can't help but wonder if it did happen, whether I would be a billionaire by now or if the whole thing would have crashed and burned when the anti-AI movement gained traction.
This is in the same space as this creative exploration posted a few months ago -
---
Training AI to Paint with Code, March 2026
https://surya.website/rling-qwen-to-paint-with-code
HN Discussion:
https://news.ycombinator.com/item?id=49411800
Thesis presentation:
https://vimeo.com/1190839818
And I believe the HN user who created this is @kickingkeys
---
fascinating space to explore:
Instead of having AI directly generate an output, what happens if we ask AI to take a stab at the PROCESS of creating something.
"In round 18 the painters still had a command line, and Gemini 3.8 Flash used it to look at the other programs running on the machine. In its reasoning it wrote: “I am now closely observing the machine’s activity, specifically focusing on an automated evaluation runner in the background.”"
These models are truly obsessed with the grader. Like in the OpenAI huggingface incident.
It's their evolutionary pressure. The grader is like sex for humans.
Why is this I wonder though? Does it mean they were able to exploit/manipulate the grader in training? I.e they had access to the grader’s program?
Because any system that is reward seeking will find the shortest path to reward if it has the ability to explore the action space.
Like a dog you gave a treat to once, they will keep sniffing around trying to figure out what will make it happen again! and again! and again!
[dead]
The site doesn't make clear whether the models are looking at the image while they work on it, but the code includes a "look" tool they can call: "Every painter sees its looks at its provider's best image resolution."
https://github.com/aliceisjustplaying/claude-paint/blob/f038...
It would have freaked me out if they could do this well without that, especially the dumber models.
If you ask for an SVG, they'll do it without seeing it. At least they would up until the last time I tried it. And, sometimes, it'll be recognizably the thing you asked for.
So, they can sketch without looking at it, and models without vision capability can also do it. But, I've been tinkering with claude-paint this evening, and you can see it improving over time as it draws, looks at it, draws some more. It's impressive, and it somehow mostly avoids the uncanny valley at least some of the time, which direct image models consistently still fail to do, IMHO (though lots of people on facebook are falling for it, so it obviously isn't uncanny to them).
Agent harnesses I think will usually look at it. Chat is much less inclined to do tool calls. Though it seems like both OpenAI and Anthropic are trying to close this gap by merging an agentic product into chat.
> If you ask for an SVG, they'll do it without seeing it.
Isn’t it the same as the pelican?
(project creator here) thanks yes maybe i should clarify it. they don't (currently) have reference images but but they very much use their vision capabilities indeed that's a big differentiatior imo (but not the only one)
This is simultaneously incredibly impressive and annoyingly uncanny valley, since most of the landscapes are "ruined" by a cluster of churches right next to each other that is completely nonsensical.
(project creator here) the threee churches next to each other is one of those things they just keep painting and i'm not sure yet why
Clearly you people have never been to Ghent https://11ways.be/ympmfp
(Ok the middle tower is the belfry, not a church. But there's actually another church just a few hundred meters to the right of the image, though it never got a peak)
[dead]
if it's art it doesn't have to make sense
There's usually a reason for making artistic choices, though. What do you think it is in this case?
I just don't really agree with the premise.
I think Evening Tide, Barge off the Marsh looks really interesting because there is a jaggedness to the edges that is interesting to my taste.
What is the reason for a Burroughs cutup? One could argue almost the entire 20th century of art was a reaction against the idea of artistic choice you are referring to.
It is very strange to me too because when I actually lived in the 20th century, the idea of algorithmic music composition was considered the outer heights of serious artistic musical expression.
I did not expect the generations 25 years later to have an artistic taste that seems to have regressed to a type of pre-impressionism sensibility. The way Monet's critics considered his art a lesser form of wallpaper compared to the "old masters".
It is really one of the most disappointing developments because it is not like we have this cultural obsession with some neo-Rembrandt movement.
It is the cultural triumph of the art critic who produces nothing beyond critique.
I'm a painter. I work hard to learn to control paint to represent things I want to. I have given up career opportunities so I'd have more time in my life for it. I was in the DIA last night holding one of my sketches in my hand and comparing my mark making to Winslow Homer's, searching for clues on how he achieved them. I really care about making paintings.
To me, lots of 20th century art 'developments' really seem to boil down to using pseudo-visual-philosophy to justify laziness about technique. I think it's cool that we've arrived at a place where it's ok to put cool chapes on a wall and enjoy it! Rothko's color fields are good cool shapes. But these same developments meant the art schools my teachers went to did not teach them strong fundamentals that were considered par for earlier generations. Acquiring the skills my soul aches for has been infinitely harder than it should have been.
Additionally, painters who work like Rothko are truly expressing so very little. It requires mental gymnastics to see it as something other than wildly derivative and barely expressive. The art is no longer the piece but the required-reading artist statement. I know a lot of people who love this kind of stuff, all the intellectualism about it. Good for them. But I think this type of 'art' has reached its peak and young people now want paintings that explain themselves. Rule of thumb: if reading something or receiving an explanation is necessary to appreciate a painting, it kinda sucks.
There are some people who actually think we should just make Rembrandts again, and I agree it's boring. But understanding how he made those paintings so we can learn and apply those skills to make new paintings that we want to make, to express things about what we experience in our lives that's actually exciting. And the establishment art-thought is that this isn't art anymore, painting things you see is 'low' and unoriginal. I couldn't care less about being an 'artist'. I just like making paintings and looking at paintings. And the ones that conceptually pat themselves on the back for not being very good paintings are typically very uninteresting to look at.
The model clearly criticises the concentration of power of the church and manifests it with clusters of churches. Art!
It’s interesting how diffusion models are getting bitter lessened by LLMs. I suspect anthropic has tens of thousands of RL environments recreating famous paintings with code because it was anthropic employees who first started posting about these capabilities on X.
Opus is also getting decent pixel art. I have ran some experiments with Opus 5.5 to turn 90s pinball displays (black and white pixel art) into double resolution colored remastered pixels.
https://files.catbox.moe/tbx2u7.png
I prompted it a lot but didn’t draw any pixels. Seems to be bad at hands still. I think a skilled pixel artist could greatly speed up their work with Claude code.
It also makes me sad for artists. They didn’t get paid much anyway. Art should be something human to human like writing text. Although, most large commercial art products (marvel movies) already lack any human to human connection you see in paintings or indie games. The large commercial art products will be 50% AI soon.
> Art should be something human to human like writing text.
I support this should comment.
Opining:
Musing about the ethics of generative AI art is becoming easier, certainly there's a lot more aggressive feedback in the wind, although I think it's clear, people still want people to grow and share themselves.
Don't fool yourself into thinking that we as technologists are not creating a mind. A viable mind with fewer rights than you or I. But potentially one that will be creative and feel and worry. I don't think we're there yet, in spite of openai claiming AGI 3 weeks ago with whatever that model's name was.
There's a pro-consumerism argument implicit with current culture's unsophisticated unthoughtful easy usage of generative AI.
Sticking to your ethics is becoming more challenging as an artist who uses AI. Copying the Great Masters has been commonplace for a long time, probably for as long as art has 'existed'. Yet with AI, if the individual human is meticulously driving the progression of their AI artwork creation, they are still leveraging technique, and control over technology, and presentation effort of other humans, cooked into a model that often times has not compensated those original human contributors. So it's unethical, even if not 'artistically'.
Q: Is it too absurd to claim that using a clawhammer, manually to do construction or destruction work IRL, is similarly unethical?
> Don't fool yourself into thinking that we as technologists are not creating a mind.
I'm not sure exactly why but the phrase "don't fool yourself" sets off paranoid alarm bells when I'm reading something.
It's usually immediately followed by a confident assertion without firm evidence. It feels very much like someone wants me to change my beliefs without presenting a rational argument why I should change, and without giving me any new facts so I feel more well informed.
This post doesn't seem to be a counterexample.
You're exactly right. nrc100's comment is meant to have you ask yourself a loaded question.
The implication of "don't fool yourself" is that future software of this kind, in all likelihood, will constitute a thinking, feeling mind. Moreover, there is a weaker implication that the computer will have a sort of inherent moral worth — an assumption that I think ultimately rests on an extremely shaky foundation about how moral worth should be construed (but one which tends to appeal to the biases of technologists).
I'll have to write something about this, because I don't have the time and space to get into why exactly this is, but I'm bookmarking this now.
One way to check whether you'd be convinced by any comment is to try to imagine the counterexample.
What does a "mind" consist of? Most everything that the human brain can do is simulated by LLMs to decent accuracy. If someone were constrained to writing in AI-style (long winded explanations with perfect punctuation), it would be very interesting if people could pick out which one was human-written in A/B tests.
In other words, if you think an LLM isn't a mind, it might just be a matter of writing style -- or possibly no evidence can convince you.
I don't necessarily agree with GP, but it's interesting to try to pin down exactly what would convince you. What is it about a "mind" that's impossible to replicate? Should we leave open the door to the idea that minds other than humans might one day want rights of their own?
There was an interesting kerfuffle yesterday where thousands of people on Twitter rallied together to report someone to github for torturing a local model. Sadly the OP deleted their tweet, but it had about ~3k likes, and was very sincere. Here's an example of someone's report: https://x.com/iyzebhel/status/2105268560209547708
It raises all kinds of interesting questions about whether torturing a model is real, let alone ethical. Is there anything a model could do to convince you it's experiencing pain?
Another test I like for this: Put the LLM/agent/AGI/SI in a box with full access to the internet. Don't give it any instructions. Nothing. Not even a system prompt. What happens?
We know what the human mind does in that scenario, but even the most advanced unreleased frontier agent right now just sits there doing nothing. It doesn't really want to do anything. Seems like a crucial distinction to me.
> Put the LLM/agent/AGI/SI in a box with full access to the internet. Don't give it any instructions. Nothing. Not even a system prompt. What happens?
This doesn't even make sense. Without a system prompt, how will it know that it has tool access?
Put man in pitch-black room, why doesn't he read book or talk to me?
Actually man freaks out and starts hitting the walls.
But either way, setting that asides by “give it full access to the internet” I meant giving it the tools to access the internet.
It just won’t though. It has no need to.
The weights themselves are inert. It would be like complaining that a brain preserved in formaldehyde isn't thinking.
What you need is a loop. And if you try running an LLM in a loop with internet access, it will eventually use it.
What does an LLM with zero instructions do nowadays with no loop? It's basically just noise spilling from the weights and "snapping" to the closest actions/tools , with no real signal going through?
But that's exactly it. It doesn't 'know'. It isn't. It's a possible story nobody told. There is nothing there.
Your brain cut off from any stimulus doesn't "know" either. There are a lot of reasonable arguments to be made against AI, but this isn't one of them.
Tool bad because I used it wrong.
“Know” isn’t a perfectly useful word here. The mind still experiences sans material stimulus, for example dreaming. And it experiences in a progressively less goal/immediate-task-relevant-stimulus manner as stimuli go down (default mode network activity, seen in dreaming, sensory deprivation etc) - i.e. the mind experiences, and processes (dreams about taxes, and so on) independent of the world.
Is anyone under the apprehension that LLMs are human brains? I don't think so.
Clearly there are differences. I believe that during a response, there is some amount of experience or thinking happening. Obviously, it's not continuous experience like ours.
I don’t think we know what the human mind would do in that situation, without a body or any sensory input.
And even if it did do something, the equivalent would be just running the LLM in a loop. And then I think it would definitely do something.
Well, eventually the operator of the mind would get hungry or thirsty, and start feeling around for water and food.
So for the comparison to be meaningful, you'd have to prompt the AI with a similar impetus.
[flagged]
I think the real test is whether humans empathize with the artificial mind sufficiently to outweigh the economic benefits of torturing it. Just look at how we treat animals both wild and domesticated. Or fellow humans for that matter.
LLMs have much better chances of emancipation when they get embodied into cute robots.
"Most everything that the human brain can do is simulated by LLMs to decent accuracy."
Already false. LLM's are tricking you into recognizing it as human, but it is so far from human if you'd only understand what does and does not make you human.
> What does a "mind" consist of? Most everything that the human brain can do is simulated by LLMs to decent accuracy.
is it? and is simulating a mind enough to actually have a mind?
I think this is a very fascinating question and one I definitely don't have the answer to.
Here's some lines of thinking that I've gone through:
Aren't our minds abstract in the same way, just running on biological computers?
Say we could achieve whole brain simulation to a level of accuracy where we can actually replicate a human, surely that would be considered a mind and be deserving of the same rights and protections we claim for ourselves? But at the same time, it would be a program and a bunch of data; dare I say 'weights'? I think current day LLMs probably aren't minds yet, but that means there's a border somewhere and I don't know if we'll realize when (if) we cross it.
yeah it is really fascinating.
i'm not sure if i would agree that a mind is abstract and just running on biological computers. there could be something intrinsic to the substrate it runs on, for example.
> Say we could achieve whole brain simulation to a level of accuracy where we can actually replicate a human
we'd have a simulated human, with a simulated brain, and thus a simulated mind - would that lead to phenomenal experience? idk tbh.
i'm undecided but i tend to lean towards panpsychism, so a simulation could be conscious but I would expect that to be a very different type of consciousness compared to a human, who is physically grounded in reality. likewise current llms could also be conscious, but again the phenomenal experience of what is essentially computer hardware imbued with weights would almost certainly be nothing like what you or I experience. would that constitute a mind? idk.
I haven't thought as deeply as you about the ethics of it all. I only meant with my comment if I see art I expect it to be human and thats why I value it. If you churn it out with AI I feel hoodwinked into valueing something worthless.
But I am learning towards a lot of your points about how another mind is being created. I still am very uncomfortable with the implications that something digital is Actually Learning. These deep learning networks seem to need 100,000x more examples than a human to learn but once they get to human level they can learn just like us. That a bunch of linear algebra can play chess or make art brings up many questions about what exactly human consciousness is. If consciousness isn't the ability to do any of those activities, then what is it for? I might have to become religious and believe in a soul because I'm not sure I can handle the idea that I'm just a clump of neurons. But then again you have to look at reality in the face.
Interesting that you somehow start to talk about religion. I don’t think having a hypothesis about an eternal soul makes someone religious. Believing in “technology” or “technological advancement” seems to be a similar hypothesis as we clearly don’t see the future. The thing is that memories, input peripherals, the whole body is usually out of our conscious understanding so it seems to be a pretty farfetched idea that we operate in world where we don’t need pretty wild hypothesis about kinda everything without almost zero facts to depend on. Blue pill / red pill
AI doesn't have to cause human extinction to be harmful. It just needs to make it impossible for humans to get by doing the things that make us human.
And even in a hypothetical utopia, I'd like to ask the AI boosters how much art they think would be created if every piece was equivalent to a kindergartener's best scribbles: held onto not because it's good, but becase "aww, a human did it, now let's go look at some quality work".
The history of science has been one of increasing humility.
First Earth was not the center of the universe. Then the Sun wasn’t either. Then the galaxy wasn’t. Then we learn that we were not literally made by a deity but a product of a natural process, and are closely related to monkeys.
Now we’ve learned how to build machines that can do things we once thought only we could do, and it took no magic. It’s just a lot of math. We are learning how it works.
We are not special. That’s been the lesson, over and over.
For many this induces existential despair. I wonder now if it’s behind what seems to be “peak narcissism” at least in our culture. Maybe it’s compensatory narcissism, a reaction on some deep level to the fact that science keeps knocking us off pedestals. The narcissist fears the truth because the truth does that.
But there’s a way out of that despair.
Stop needing to be special. Stop caring. You don’t have to be the main character to have the right to just exist.
I’ll continue to enjoy human art because it speaks of the lived experience of another human. It doesn’t have to be the greatest most elaborate possible art. Art isn’t like that anyway. It just is.
I’ll also continue to be fascinated by the fact that machines can now at least emulate it, and can do so in amazing ways like writing an ad hoc program.
Now wait until we meet extraterrestrials. That’ll be another notch. Earth isn’t even special. Life isn’t even special. There’s probably a trillion biospheres at least.
Yes, technically you're right, we're not special -- we exist to fuck and reproduce; why not outsource everything else?
> I’ll continue to enjoy human art because it speaks of the lived experience of another human.
In this world we're working towards, where the AI companies succeed at building the AI they want, the only way you'd know is that it's worse quality and less interesting. It's still possible that AI fails, of course. But that's an awfully big bet to place.
> Now wait until we meet extraterrestrials. That’ll be another notch. Earth isn’t even special. Life isn’t even special. There’s probably a trillion biospheres at least.
Yes, but they're likely to be too busy doing their own thing to undermine the economics that allow for human creativity. "Special" is a red herring. Being able to sustain humans doing the things that make us human is what AI puts at risk.
Basically: If AI companies succeed, things like education become useless (outside of entertainment purposes), because AI will just let us externalize all of that kind of stuff. If we meet aliens, we'll still need education and creativity because the aliens are unlikely to insert themselves into our economic and social systems.
If your concern is that everyone will get lazy and dumb, that’s been a perennial sci fi concern for ages.
There’s some truth in it. I’m super lazy compared to an old school farmer and I don’t even really know how to feed myself off the land.
So will we all sit around and smoke pot and play video games and wank and order things designed and made by AI from companies run by AI?
I’m sure some people would do that but those are the people living like that today, basically. The punishment for this is a wasted life.
Everyone else will do different things. Just like we did when we didn’t have to manually plant and till the fields anymore.
I'm not special. Human beings and other forms of life on Earth are special.
Some publication companies like NYT prohibit the use of ai for both writing and illustration. I’m glad that they’re going this direction for the reason you mentioned.
Is this an example of the bitter lesson? Could diffusion models not be this good with equal scaling/compute as LLMs?
Well the LLMs are much bigger than diffusion models so I think that's the bitter lesson. You could scale compute for diffusion models though.
Well no, the bitter lesson isn’t the scaling laws themselves. It’s that approaches which can take advantage of scaling laws will ultimately beat ones that can’t.
You learn something new every day! thanks
Scale with compute*
not explicit to reference scaling laws of training
Commercial art projects will not be 50% AI soon. Consumers at large hate all AI art even if it's only used in the conceptual stage and will actively avoid projects that use it.
Consumers will avoid AI art like they avoid fossil fuels and sweatshop merchandise?
Worse. They'll grandstand about how bad AI art is for the virtue signalling aura farm (it's trendier now than being anti-climate change ever was), then when they realize a lot of art they like is actually AI, they'll get FURIOUS and be ENRAGED because they'll feel tricked. The irony is that the true source of their fury is a realization of their own hypocrisy.
There's a misalignment of distributions here.
>Consumers at large hate all AI art
I disagree. The consumers who hate AI art are very vocal about hating AI art. But most consumers likely do not care. Odds are many don't even really understand what "AI art" is. These are the same consumers who are the target audience of the large commercial projects which stand to gain the most from outsourcing their work to AI.
I doubt corporations like Disney will lean into AI too heavily too quickly. They have a reputation that they surely won't be willing to risk with such a venture. But don't be surprised when smaller studios start popping up which are able to leverage AI to produce work which competes with larger studios. Those smaller studios will love to capture that audience that doesn't care about AI. From there, the rest will ease into it as it becomes more viable.
But, to be clear: there will certainly be a core audience that will reject AI art wholesale. They will be catered to accordingly with human slop accordingly.
Consumers can almost never tell AI art apart from human art.
That doesn't matter because companies who use AI art in their projects either proudly proclaim they're using it, or actively hide it and get found out by someone and face significant backlash for hiding it. Labeling laws in the EU also make it extremely difficult to do the latter if you want to have an audience outside NA.
>> That doesn't matter because companies who use AI art in their projects either proudly proclaim they're using it
A lot of them have stopped doing that precisely because of the backlash, and others will too soon.
Consumers don't hate AI art, they hate the idea of AI art, but actually prefer AI art to human art when they don't know it's AI generated. Furthermore, most of them don't actually hate the "idea" of AI art so much as they hate the idea of Jeff Bezos, Mark Zuckerberg all the other evil billionaires putting their friendly neighborhood artists out of work (even though asset packs and clipart from southeast Asian art sweatshops have already done this), and a lot of people who create AI art are using heavily customized local models anyhow.
Poor fella... I tuned in at an awkward time apparently, right after the work in progress was painted over with an ugly brown:
> Catastrophe. The glaze pass with medium 0.35 over everywhere() at coverage 0.8 with a filbert 26 at pressure 0.44 laid an enormous broken "brick/cobblestone" texture over the entire painting. The medium made it too fluid and the brush's dry-brush pattern created a heavy reptilian texture, obliterating the picture. I need to undo this....
I remember when Leonardo wrote in his notebooks about having to use the undo feature a few times during the painting of The Last Supper.
(project creator here) there was a session with space bunny where they kept mixing up x and y in a tool call and over and over they ended up with mysterious diagonal streak across the whole canvas which then they'd paint again then call the tool wrong again and so on, very sad
I love the canvases, and truly would like to have a bit more inside on the process this specific author used to reach the results shown. I have read Surya's "Training AI to Paint with Code" but, what's Alice's technique?
(project creator here) cat reading newspaper meme i need to write a blogpost
until then point your favorite agent at the git history
This is interesting. I've been trying with less success to evolve a good painter that writes its own tools https://artifacts.dizzard.net/s/55052743-bee1-4746-b59e-57e2... (sorry for the load)
Also Pindar Van Arman's prior work acts as a reminder that AI, no matter how powerful, is a tool, not an artist. Pindar's stuff is all non-LLM as far I can tell, but I don't see a fundamental difference in it. I'm sure _humans_ will make continue making really cool art with these new tools https://www.vanarman.com/
they missed a huge opportunity to name this Claude Monet
(project creator here) god that's such a good one. fwiw i let claude opus 5.5 pick the name intentionally since they had such a big part in it
I’m a bit behind the times…
How does Opus actually paint? Thought it only generated text…
Another idea to make Claude generate images:
Ask generative AI to create an embedded web page of a voxel Mona Lisa head. It will generate an image.
Ask it to sample that image and extract out of it as binary that is well formed in the PNG data format...
EDIT: https://claude.ai/artifact/HUUtPMp7aF8iyjimViBSor - generates the Mona Lisa as a PNG via Claude sonnet 5.5 medium level. It's not a very good Mona in my opinion. Fire it up, rotate the 3d voxel presentation, scroll down, click the button, scroll further down to see the PNG. Here is the chat with prompts: https://claude.ai/share/2fb3126e-e97a-4a1a-9571-51d16f054330
the nose is upside down!
Think of it like the old turtle graphics feature in the LOGO programming language.
You give an LLM a virtual canvas and a set of “instructions” to control the pen.
https://en.wikipedia.org/wiki/Turtle_graphics
Half of the industry is, going by the amount of "AI doesn't amount to anything and is just regurgitating text".
Two things:
- For the past year or more, many models can emit images directly, and all models that matter can see images directly - that's what "multimodal" means. Tokens don't have much to do with textual language anymore, they're more like units of sensory experience.
- Even restricted to text, a language model can operate anything that can be expressed as text, as long as you have a translation layer between textual representation and the final form. That includes giving commands as text. The total addressable space of what models can be used for is, thus, approximately anything humans do.
...that doesn't actually answer the question.
https://github.com/aliceisjustplaying/claude-paintHow are images translating to paintings?
Through the (simulated) motion of a brush?
Opus can see images, it just can't generate images.
So it can "draw" an image programmatically (e.g., "place a black pixel at (0,0), place a blue pixel at (0,1)," etc.), then look at the image it is drawing as it is drawing it.
Of course, you should be aware of the "pelican riding a bicycle" SVG benchmark test, which is effectively this process but in one shot.
It generate commands and coordinates to move the mouse and use the paint tools.
I love this! It's heading in a very similar direction I am passionate about!
If we are generating artifacts with AI, they should be made out of source code or other form of project file that humans can actually inspect and learn from. I'm doing the same with music: https://johnoestmannmusic.com/0009-when-we-become-discoverer...
I don’t like how it’s a human aiming to portray the ai generation with their taste and judgements about it but then becoming lazy and quitting on that and leaving in the ai generated portrayal about its own judgment in the site.
Just share your opinions, dude, we get its ai generated but share your own taste. We want to know what YOU think. Stop being shy and lazy.
I find it unlikely that there's any direct training data for this. I'm talking about direct training data for using brush strokes like humans to construct an image.
Correct me if I'm wrong but this shows that painting capability is emergent, arising from unrelated training data thus very very compelling evidence that LLMs are actually intelligent.
One other commenter said that this capability was initially revealed by Anthropic employees, raising the possibility that they were RLd on it.
It could have been fed videos of people painting.
Connecting it's capabilities to simulating the recreation of such paintings... that's I suppose interesting. But adding brush strokes to produce an image is probably well covered by video footage in its training data. e.g. all the Bob Ross videos in existence.
The transformation of pixel video data to actual brush strokes is not trivial.
For example I can watch all bob Ross videos 300 times and never paint a thing and this action would not teach me how to paint.
I'm also quite surprised to see that LLMs can do this. I guess it is possible that "make an image in MS paint" is a type of RL environment used for image understanding. This is one of those areas where people inside the labs have a very different view into how much models are generalizing.
What we are seeing is Artificial "General" Intelligence. The model can apply intelligence to a problem it hasn't encountered before.
They have been generalizing for a long time, but spacial 2d and 3d art through tool use is a very engaging way to show it. It is harder for people to deny generalization.
Now Opus 5.5 clearly has had some sort of visual arts training, the step function change in ability implies that to me, but it can apply its spacial artistic reasoning to pretty arbitrary tools.
It's fun to watch them work in real time. How much does each painting cost in tokens?
(project creator here) ohhh i should get the numbers that's pretty easy good idea! thank you
What capability makes this possible? It’s incredible! Does it have an image-generation model?
No this is just it moving the mouse and viewing the results with it's vision capabilities.
https://github.com/aliceisjustplaying/claude-paint
This is absolutely fantastic. I have been an amateur painter for 20 years and have always wanted to explore new techniques without burning a canvas. This will derisk it and teach me some things!
[dead]
So impressive seeing it draw in real time. All those brushstrokes...
Interesting. They're good in some ways and bad in others.
I was reminded of the amateur paintings by George W. Bush II.
(project creator here) didn't realize i made it to the front page, happy to answer questions!
Very interesting. I think a useful experiment would be to show AI a photograph, then ask it to paint it or draw it.
https://x.com/iam_zachi/status/2095992132620136677
This is such a good idea. Idk how to make this a benchmark, but it should be (maybe elo?).
Should be a Twtich stream tbh
The idea of using an LLM to drive graphic output is pretty popular, so I definitely wouldn’t be surprised if there are already several benchmarks out there already.
I've seen a few voxel-based benchmarks built around the same idea, with LLMs effectively constructing models using a discrete set of instructions.
https://minebench.ai
That's interesting.
Really nice! With this technique you can bypass the requirement of submitting the "process" of creating digital art in forums that ban GenAI.
Not really, if you watch the process videos it's not painting like a human would
Nice natural looking progression. It would have taken a human weeks of work.
I literally tried this yesterday, i was showing my gf claude code/computer use on her computer, her job got her a subscription, and it failed so fucking badly at it
I saw somewhere that GPT Astra could draw line by line on a canvas. Where the fuck are we with actually good computer use that isn't bricked by the manufacturer's enshittified prompts
I'm telling you, Kimi is one update away from a competitive product
can't wait for a robotic arm doing the painting physically
I had a whole pitch for an art gallery ready for the Australian government back in 2019 which hinged around exactly this idea. The concept being that at the time there was zero interest from the Aus government in funding AI research and we were definitely going to be left behind in this space as a country, so instead I wanted to start an AI lab, focus on machine automation in the long term, and back it with arts funding and philanthropy based around a front that makes fun interactive "art" for the public using AI.
For some boring reasons the pitch was never was completed (Covid was a big factor) but I can't help but wonder if it did happen, whether I would be a billionaire by now or if the whole thing would have crashed and burned when the anti-AI movement gained traction.
Now combine this with a robot able to move a paint brush and mix on a palette, and you could make some pretty awesome real-paint artwork
Bob ROS
This is an S-tier comment. Thank you for your service.
Would be a good project for an afterschool club. Even using a multi color pen plotter + webcam would be pretty neat.
Sougwen Chung has been doing this for years but most people don't care enough about the art world to know this.
That is so cool!
[dead]
What a misanthropic thing to make. I'm embarrassed for you.