one more item to note for the record
the agent ran tests but did a '| tail -5' on the output, so it missed some failed tests when those tests should have suggested early that something was wrong
I believe there will never be an LLM that is NOT trained to take shortcuts. this is actually intelligence, and efficiency, and competitive speed, etc.
the better that things appear to be, on benchmarks and happy-paths, the more stuff is intelligently skipped based on general heuristics
these are not like robot vacuum cleaners that are better at tedious work than human beings. these are smart agentic actors that are better than humans at applying heuristics drawn from an inconceivably large bag of tricks
The post this replies to
Terminus needs JavaScript to show replies and pictures.