I can't stop thinking about the fawning copilot demo from a very senior engineer I saw earlier today. It starts out like "this interface changed and I need to update the usages"
*prompt*
Look at that it's exactly right
*prompt*
*fix the imports*
Yes, that's exactly the implementation I already did in a different place
*prompt*
*correct the snippet*
It's just amazing how productive this is
The dude literally could not see that it was basically fortune telling, but for rust. It was all him. He did it first, and then copilot copied him, badly. He did it with his literal decades of experience, which is so extensive that he can just idly correct entirely wrong code without even fully realizing he did it.
These text synthesizers just give you an empty container to pour your expectations into. It only works if you already know the answer. And if you're ready to take over when it's not even a useful container.
But if you didn't already know it was right, then you also wouldn't know it was wrong. There's so many extremely experienced people out here who have forgotten that it's even possible not to know these answers, misleading the people they're supposed to be teaching.
I hate this.
@jenniferplusplus The relevant principle is, it's always harder to read code than to write it, so if a machine writes for you code that you'll have to read because the machine is not as trustworthy as a junior programmer, the machine will have made more work for you, not eased your workload.
@riley @jenniferplusplus Right, but when it can guess what you were going to write and write it for you before your fingers get there, it's amazing. That's the main use case for me.
@LouisIngenthron The problem is, you can't responsibly rely on it being amazing. The only way that it can be amazing every time is, if you engage in the Texas sharpshooter fallacy, and only count the times when it happened to do something amazing.
@riley @jenniferplusplus It doesn't have to be amazing. If it can just scaffold out the tedious stuff for me (which it can successfully 99% of the time), then it still makes my job easier, and is thus a useful tool.
@LouisIngenthron @riley @jenniferplusplus You don’t need LLMs to make it easy to scaffold code. Any good IDE already had the feature in the form of a snippet library of *known good* scaffold code. To pick one example, mine has a scaffold for a table-driven unit test, which is the most tedious piece of code I can think of that I regularly have to write.
@mathew @riley @jenniferplusplus Of course I don't *need* an LLM for that. I don't *need* an IDE other than notepad either. But better tools makes work go faster.
@mathew @riley @jenniferplusplus When the library of good code is one room and the library of may-or-may-not is a skyscraper, the balance shifts.
More importantly, I would never recommend these tools for amateurs for that very reason (it reinforces bad habits as well as good). But for those of us with the skill and experience to identify the bad stuff, it's a huge efficiency booster.
@LouisIngenthron @riley @jenniferplusplus Personally I feel that a library of good code is going to be a better way to make work go faster than a library of code that may or may not be good.