I think the issue is LLMs can’t do a humans job, only be a tool for a human to use. And the tasks it’s good at weren’t large bottlenecks to begin with.
the tasks it’s good at weren’t large bottlenecks to begin with.
Disagree. There are plenty of tasks it is being tried with which it’s not good at, but to an extent you don’t know that until you’ve tried.
I use LLMs for code review, they help - a lot. Like so many new technology applications, we just didn’t do code review at anything like the level that LLMs enable doing it at, with LLMs we’re doing more of what we “always wished we had the time / attention span for” but never actually did, in practice. LLMs are lowering the total cost of finding issues in code, making the reward of doing them in depth worth the cost. In the past we’d do a much higher level review, and miss a lot of detail issues, but the cost of finding those detail issues without LLMs was very high and for the most part we deemed the detail issues “not worth the cost to find.” Using LLMs for code review is actually making more work than we used to put into code reviews, but the returns are there - worthwhile.
You might think of it like finite element analysis for mechanical design. Sure, that was possible with paper drawings and slide rules, but it wasn’t worth the cost using those tools. Now, there are areas of mechanical design where FEA modeling is the standard practice - if you don’t do it you’re considered to be slacking, not applying state of the art tools to the job.
There are some things that I find LLMs to be horrible at - like 3D drawings in Blender. However, they are actually really good at making specialized Blender usage tutorials to make it easier / faster for human artists to learn how to use the tool themselves - and once in awhile they can whip up a very helpful python script to automate something that would have been tedious / time consuming to do over and over but not quite worth learning python and the APIs to automate.
That’s a lot of words just to reiterate what OC said. None of your examples disagree with the fact that these companies are using AI for a ton of stuff they’re not good at. And the stuff it is good at are not bottlenecks. If you removed LLMs from the face of the planet today, code quality will not suffer significantly. Sure, doing code review en masse is good. But that is not what was holding back computer programming. It’s existence is not progressing software in any significant way either. It’s a nice to have, not a must have. Code was fine for four decades without LLMs. Indeed it makes me wonder, if a machine learning model was purpose made to do code review, instead of general purpose LLMs doing it, how much better it could be, if we actually leveraged what they’re good at.
Disagree. Code review done right is a virtually impossible bottleneck for most companies to handle, so in the past they didn’t do it and we have the shitshow of security and other bugs that we are experiencing today.
If you removed LLMs from the face of the planet today, code quality will not suffer significantly.
But LLMs aren’t going anywhere, and they are already being used to find vulnerabilities by hats both white and black. They are also finding functional bugs that affect life safety and financial stability.
Code was fine for four decades
The way that municipal water was fine for centuries before chlorination. Cholera outbreaks were just one of those unavoidable realities, like mass school shooting events in the US.
if a machine learning model was purpose made to do code review, instead of general purpose LLMs doing it, how much better it could be, if we actually leveraged what they’re good at.
From what I have seen over the past year, this type of specialization is happening, and the progress is real and significant.
Comparing code bugs to cholera. Nice way to be utterly disconnected from reality and what really matters in life. Let’s burn the planet down and waste all of our clean water because of code review.
We’re getting to a point where code failures are causing deaths, large numbers of deaths… Boeing’s MAX10 was a code / design / training / management failure.
Dogs will do it for belly rubs and $20 in kibble
I think the issue is LLMs can’t do a humans job, only be a tool for a human to use. And the tasks it’s good at weren’t large bottlenecks to begin with.
Agree
Disagree. There are plenty of tasks it is being tried with which it’s not good at, but to an extent you don’t know that until you’ve tried.
I use LLMs for code review, they help - a lot. Like so many new technology applications, we just didn’t do code review at anything like the level that LLMs enable doing it at, with LLMs we’re doing more of what we “always wished we had the time / attention span for” but never actually did, in practice. LLMs are lowering the total cost of finding issues in code, making the reward of doing them in depth worth the cost. In the past we’d do a much higher level review, and miss a lot of detail issues, but the cost of finding those detail issues without LLMs was very high and for the most part we deemed the detail issues “not worth the cost to find.” Using LLMs for code review is actually making more work than we used to put into code reviews, but the returns are there - worthwhile.
You might think of it like finite element analysis for mechanical design. Sure, that was possible with paper drawings and slide rules, but it wasn’t worth the cost using those tools. Now, there are areas of mechanical design where FEA modeling is the standard practice - if you don’t do it you’re considered to be slacking, not applying state of the art tools to the job.
There are some things that I find LLMs to be horrible at - like 3D drawings in Blender. However, they are actually really good at making specialized Blender usage tutorials to make it easier / faster for human artists to learn how to use the tool themselves - and once in awhile they can whip up a very helpful python script to automate something that would have been tedious / time consuming to do over and over but not quite worth learning python and the APIs to automate.
That’s a lot of words just to reiterate what OC said. None of your examples disagree with the fact that these companies are using AI for a ton of stuff they’re not good at. And the stuff it is good at are not bottlenecks. If you removed LLMs from the face of the planet today, code quality will not suffer significantly. Sure, doing code review en masse is good. But that is not what was holding back computer programming. It’s existence is not progressing software in any significant way either. It’s a nice to have, not a must have. Code was fine for four decades without LLMs. Indeed it makes me wonder, if a machine learning model was purpose made to do code review, instead of general purpose LLMs doing it, how much better it could be, if we actually leveraged what they’re good at.
Disagree. Code review done right is a virtually impossible bottleneck for most companies to handle, so in the past they didn’t do it and we have the shitshow of security and other bugs that we are experiencing today.
But LLMs aren’t going anywhere, and they are already being used to find vulnerabilities by hats both white and black. They are also finding functional bugs that affect life safety and financial stability.
The way that municipal water was fine for centuries before chlorination. Cholera outbreaks were just one of those unavoidable realities, like mass school shooting events in the US.
From what I have seen over the past year, this type of specialization is happening, and the progress is real and significant.
Comparing code bugs to cholera. Nice way to be utterly disconnected from reality and what really matters in life. Let’s burn the planet down and waste all of our clean water because of code review.
We were burning the planet down for billionaire yachts and dollar store trinkets so.
We’re getting to a point where code failures are causing deaths, large numbers of deaths… Boeing’s MAX10 was a code / design / training / management failure.