When was the last time you checked to see your mechanic's dental record? Would you really trust someone's fingers in your engine if they don't floss?
Or what's your dentist's 5k PB? He's running over 45:00s? Hah, I think not!
And can your accountant spell strawberry? Probably 🤷♂️ but here's the thing....
I really don't give a fuck
Because none of those things matter as a yardstick to what I am trusting them with. Well, I guess what I would trust them with if I had an accountant or a mechanic.
But the point still stands, arbitrary general challenges for specialised tools are dumb. It's the reason an F1 car would make a terrible daily driver. If you measure it for such a task, it just leads me to ask why you were even considering using it? wait why do you have an F1 car? please stop trying to fit that baby seat into this f1 car!
But again, faced with the shadow of truly mind-melting technology that casts doubt into the very fundamentals of how we interact with technology, we end up parroting fucking dumb gotchas.
Because yes, some models struggle to deal with counting letters and several other specific scenarios will equally stump them, and that is certainly interesting and something to keep in mind. But it is not the flag signalling the derailing of the AI freight train. If you are looking for the smoking gun then this isnt it and its drawring attention to all the wrong places. You will not prove how bad AI is by looking at AI. It is what it is and no amount of pearl clutching will make the code care because its just doing what it does. We need to be holding people accountable, not things. So stop asking if your LLM can count the letters in strawberry and start asking if its creator can count the amount of I's in IPO.
Claude has been doing the lion's share of my day-to-day code, it's a good thing I don't work for a spelling bee
Facetiousness aside, I get it. AI is new and scary and everywhere and of course people should scrutinise it. But the lesson shouldn't be 'AI is so stupid it can't count', it should be to actually inspect your tools and use them as directed.
Your seething hatred or unwavering love doesn't really matter, AI has already taken root everywhere it matters and if we can't have sensible discussions that don't devolve into gotchas and strawmanning then the future of AI is about as hopeful as the future of politics.
Yours faithfully, someone who used an inflammatory clickbaity title and stance to make you read my post ✌️