You get all your teams to build some expertise and (hopefully) get a sense of where the technology might have some ROI
You’d run the risk that they instead develop a dependency or overreliance on the tech. They probably won’t think about the “I” part of ROI and evaluate how many tokens a given task produces relative to the saved time and effort.
Throttling it later might then cause a drop in productivity until they relearn how to do simple stuff they could do themselves but delegated to AI instead, whether or not it’s ideal for the task.
For example: “search and replace” requires the LLM to ingest and then produce the whole document as output. Aside from the question whether it’ll have caught all instances and replaced them without otherwise altering the text (which a casual user won’t check), the amount of output tokens correlates with the size of the text.
That’s a lot of wasted tokens for a task they could have done without AI, but so long as asking the computer is quick and convenient, they won’t think twice. Then, once the tokens are throttled, they’ll suddenly realise they’ve run out of tokens early because they burned a ton on tasks that seem trivial to them, leaving none for the more complex tasks they’d actually prefer to delegate (whether or not they should). They might not make the immediate connection which tasks eat so many tokens either, so they’ll take a while having to try all their use cases again, see how expensive they are, run out of their allotment early and wait for the next period.
If you’re gonna have people figure out how to use it, you’ll have to throttle from the start to make them also figure out how to use it economically.
Also, mandatory classes on the limitations and reasonable uses. Don’t let it get to the point where they find out the hard way that it’s not actually intelligent and has no concept of truth.
I hadn’t thought about this, thanks.
Personally, if the messaging had been “play with this new thing, see where it helps, report where it doesn’t or where it’s actively harmful”, I would have had a much better time with it. The fact that it was “use AI for everything or else you’ll lose your job to someone else who does” created all sort of perverse incentives to use it for the sake of using it (even where it doesn’t make sense), to lie about the results and to generate more anxiety in others to keep up with your made-up achievements.
I think at least some of the wasteful or even harmful ways you describe of using LLMs come from this push to use it and “be more productive” with it.
But you’re right that there are people who became overly reliant and even ruined their lives with LLMs without the tech being forced on them.
to lie about the results and to generate more anxiety in others to keep up with your made-up achievements
Emperor’s New Clothes style “we all need to pretend to like it” is an unfortunately common effect of decision-makers deciding they know some brilliant thing and any naysayers just aren’t suited to appreciate the brilliant thing.
I think at least some of the wasteful or even harmful ways you describe of using LLMs come from this push to use it and “be more productive” with it.
Some, sure.
Others from a fundamental misunderstanding of the nature of language models. They’re text processors and generators designed to sound human. They can’t tell facts from filler.
Just earlier, I saw a post elsewhere about someone having generated an article or something which cited three experts – wrongly, because it doesn’t actually know what the relation between the text in quotes and some supposed source is or why it needs to be verbatim to be a correct quote. That’s not a bug, nor a hallucination or whatever anthropomorphic euphemism people come up with for “random output happened to be wrong” (though, to be fair, “random” glosses over a highly complex prediction system that can predict plausible text quite impressively, even if it can’t predict truth).
Students relying LLMs to generate their coursework are falling into that trap without any pressure of productivity. They don’t get that the purpose of coursework is to learn about the source material and the structure of academic writing rather than just produce text. They also don’t get that the LLM won’t look up, interpret and cite sources accurately in accordance with the subject of the question. It will generate a plausible-sounding answer to the question, and therein lies the danger: If you don’t already know the answer, how could you tell if it’s true?
The same goes with people “looking up” information. Gemini will produce some text statistically correlated to the text it has read, but you never know whether that correlation reflects facts or whether it falsely attributes some shady business to companies who had nothing to do with it (about which there was a court case in Germany recently).
Vibe coders without programming experience cannot qualify the output of their generator. It’s always harder to understand code you didn’t write (or maybe wrote long ago), but if you don’t even know how to write code, you’ll have no experience to compare it to.
The common thread behind these is that these AIs lack the understanding of the concepts they’re producing text about and semantic connections between them, and accordingly cannot treat these things with the same nuance and precision that humans can.
But the ways they’re harmful doesn’t immediately become apparent. “Report where it’s harmful” doesn’t really work if it takes two years for a critical security flaw to surface that some code generator produced and nobody with experience caught. You may never notice your ability to deal with stress being eroded until some day you can’t ask your robot buddy for help and just crack instead.
They plant traps in your education, your knowledge, your work, your psyche. To encourage people to use them without thoroughly preparing them for those traps is reckless.
You’d run the risk that they instead develop a dependency or overreliance on the tech. They probably won’t think about the “I” part of ROI and evaluate how many tokens a given task produces relative to the saved time and effort.
Throttling it later might then cause a drop in productivity until they relearn how to do simple stuff they could do themselves but delegated to AI instead, whether or not it’s ideal for the task.
For example: “search and replace” requires the LLM to ingest and then produce the whole document as output. Aside from the question whether it’ll have caught all instances and replaced them without otherwise altering the text (which a casual user won’t check), the amount of output tokens correlates with the size of the text.
That’s a lot of wasted tokens for a task they could have done without AI, but so long as asking the computer is quick and convenient, they won’t think twice. Then, once the tokens are throttled, they’ll suddenly realise they’ve run out of tokens early because they burned a ton on tasks that seem trivial to them, leaving none for the more complex tasks they’d actually prefer to delegate (whether or not they should). They might not make the immediate connection which tasks eat so many tokens either, so they’ll take a while having to try all their use cases again, see how expensive they are, run out of their allotment early and wait for the next period.
If you’re gonna have people figure out how to use it, you’ll have to throttle from the start to make them also figure out how to use it economically.
Also, mandatory classes on the limitations and reasonable uses. Don’t let it get to the point where they find out the hard way that it’s not actually intelligent and has no concept of truth.
I hadn’t thought about this, thanks. Personally, if the messaging had been “play with this new thing, see where it helps, report where it doesn’t or where it’s actively harmful”, I would have had a much better time with it. The fact that it was “use AI for everything or else you’ll lose your job to someone else who does” created all sort of perverse incentives to use it for the sake of using it (even where it doesn’t make sense), to lie about the results and to generate more anxiety in others to keep up with your made-up achievements. I think at least some of the wasteful or even harmful ways you describe of using LLMs come from this push to use it and “be more productive” with it.
But you’re right that there are people who became overly reliant and even ruined their lives with LLMs without the tech being forced on them.
Emperor’s New Clothes style “we all need to pretend to like it” is an unfortunately common effect of decision-makers deciding they know some brilliant thing and any naysayers just aren’t suited to appreciate the brilliant thing.
Some, sure.
Others from a fundamental misunderstanding of the nature of language models. They’re text processors and generators designed to sound human. They can’t tell facts from filler.
Just earlier, I saw a post elsewhere about someone having generated an article or something which cited three experts – wrongly, because it doesn’t actually know what the relation between the text in quotes and some supposed source is or why it needs to be verbatim to be a correct quote. That’s not a bug, nor a hallucination or whatever anthropomorphic euphemism people come up with for “random output happened to be wrong” (though, to be fair, “random” glosses over a highly complex prediction system that can predict plausible text quite impressively, even if it can’t predict truth).
Students relying LLMs to generate their coursework are falling into that trap without any pressure of productivity. They don’t get that the purpose of coursework is to learn about the source material and the structure of academic writing rather than just produce text. They also don’t get that the LLM won’t look up, interpret and cite sources accurately in accordance with the subject of the question. It will generate a plausible-sounding answer to the question, and therein lies the danger: If you don’t already know the answer, how could you tell if it’s true?
The same goes with people “looking up” information. Gemini will produce some text statistically correlated to the text it has read, but you never know whether that correlation reflects facts or whether it falsely attributes some shady business to companies who had nothing to do with it (about which there was a court case in Germany recently).
Vibe coders without programming experience cannot qualify the output of their generator. It’s always harder to understand code you didn’t write (or maybe wrote long ago), but if you don’t even know how to write code, you’ll have no experience to compare it to.
People using AI for coping with stress may run into a trap where they end up unlearning to cope on their own and potentially take on even more stress.
The common thread behind these is that these AIs lack the understanding of the concepts they’re producing text about and semantic connections between them, and accordingly cannot treat these things with the same nuance and precision that humans can.
But the ways they’re harmful doesn’t immediately become apparent. “Report where it’s harmful” doesn’t really work if it takes two years for a critical security flaw to surface that some code generator produced and nobody with experience caught. You may never notice your ability to deal with stress being eroded until some day you can’t ask your robot buddy for help and just crack instead.
They plant traps in your education, your knowledge, your work, your psyche. To encourage people to use them without thoroughly preparing them for those traps is reckless.