• dnick@sh.itjust.works
    link
    fedilink
    arrow-up
    1
    ·
    2 days ago

    I’m sure it will get there, but the biggest issue i see with the current models and implementations is that without good, or in some cases excellent, guidance, most can’t handle anywhere near a full codebase and even the best models are happy to take your question about fixing any particular task you throw at them back to square one with no regard towards how the existing platform is set up. Even for a relatively light codebase that it is currently building, it’s happy to scrap it start over in order to solve whatever potentially poorly worded issue you’re trying to track down. And that’s not even taking into account how quickly it can go from trying to solve an issue to nerfing tests in order to pass. Frustrated that things are no longer passing an expected integer test? Easily solved by allowing floats or nulls to count as passes, and damn if that isn’t a process to correct three issues deep after you’ve ‘validated’ that issue and it’s now being discovered down the line.