Grok for images is pretty competitive, in the little use I’ve put it to.
Grok for code is comically bad.
Of course, in the little I have used LLMs for stable diffusion / Control Net image generation, they all suck, Grok is just near the top of the steaming pile. Meanwhile, the best of the code writing and reviewing models are starting to become super-human, in the way that AlphaGo became super-human a few years back at playing Go.
Caveat: writing and reviewing code is just one small part of Software Engineering, the way that pigment mixing is one small part of painting.
What kind of challenges are you giving arena.ai? When I tried Grok (latest at the time available in Cursor) vs Opus 4.5, Grok was much faster to respond, and hillariously terrible at wrting functional code - but confident in its responses.
I tried your link, got a Grok 4.2 v Grok 4.3 pairing, they did O.K. 4.2 better than 4.3 at making a webserver that queries and re-serves NEXRAD data, but… they didn’t get too far down the feature list (history of rainfall graphs) before they fell apart.
I would have thought they’d get access to xAI’s models for very cheap.
Grok really isn’t that good, though. So few people use it that xAI are renting out most of their AI servers to both Anthropic and Google.
Grok for images is pretty competitive, in the little use I’ve put it to.
Grok for code is comically bad.
Of course, in the little I have used LLMs for stable diffusion / Control Net image generation, they all suck, Grok is just near the top of the steaming pile. Meanwhile, the best of the code writing and reviewing models are starting to become super-human, in the way that AlphaGo became super-human a few years back at playing Go.
Caveat: writing and reviewing code is just one small part of Software Engineering, the way that pigment mixing is one small part of painting.
Maybe they do and the $200 limit actually goes quite far 🤷♂️
(Doubt it though)
Looking at the rest of the AI landscape, I’d think they’re overcharging themselves, rather than undercharging.
All the money then stays in the same collective pool, and xAI’s numbers look better
In my experience (I use arena.ai battle mode as a main chatbot), higher versions of grok on a good day are at the level of Claude Opus 4.X
What kind of challenges are you giving arena.ai? When I tried Grok (latest at the time available in Cursor) vs Opus 4.5, Grok was much faster to respond, and hillariously terrible at wrting functional code - but confident in its responses.
Usual everyday tasks. For coding it is: Claude>Chatgpt>Chinese open source>Gemini>Grok>Everything else.
I tried your link, got a Grok 4.2 v Grok 4.3 pairing, they did O.K. 4.2 better than 4.3 at making a webserver that queries and re-serves NEXRAD data, but… they didn’t get too far down the feature list (history of rainfall graphs) before they fell apart.
Huh, interesting. I wonder why it’s so infrequently used then. Maybe people are afraid of using an AI that referred to itself as “mechahitler”.
Maybe that.