I feel like this is true for lots of kinds of conversations and not just tech interviews.

People are correctly pointing out that, if you dig into the logic of basically anything, it falls apart, but that's also generally true of actual humans, even experts.

Sure, https://twitter.com/YossiKreinin/status/1599114477022064640 is ridiculous, but have you tried asking an expert coach on almost any topic why you should do X? I think the level of reasoning is fairly similar to what Yossi observes ChatGPT doing.

E.g., try listening to one of the top paddling coaches in the world explain *why* (the what is good, the why is nonsense) https://www.youtube.com/watch?v=VqXIF4ToUcE

Why do you let the boat "run" between strokes? The explanation is because the boat is moving at top speed when you remove the paddle from the water, so putting the paddle back in slows you down.

But of course this is backwards. The boat decelerates when you're not pulling. Your top speed is then because you just finished pulling and started braking!

Lawler even notes that you only want to put the paddle back into the water ASAP when accelerating. But, if putting the paddle in ASAP slows you, why do only do it when accelerating? You only want to slow down when accelerating? None of his explanations of why you should do anything make sense.

And this is fairly typical of when you ask why, e.g., when I asked why wider tires have better grip, I had to go to page 7 of Google before getting an answer that wasn't ChatGPT-like obvious nonsense.

Per https://twitter.com/danluu/status/1304093800474636288, every explanation on the first six pages of Google results was missing at least one key factor, and most explanations were ChatGPT-like nonsense, in that they contained contradictions that someone with no expertise in the area can observe.

Although you can't publicly play with it, I've heard that Google's chat bot is much better at outputting text that doesn't contain these kinds of "obvious" logical contradictions than ChatGPT and it's been better for years.

Because I'm a fallible human who's prone to all sorts of blunders, it took me two hours to remember that, in 2015, I wrote a blog post about how AI doesn't need to be all that good to displace humans due to human failures in basic tasks:

https://danluu.com/customer-service/

Of course I would've integrated this into the original thread had I remembered it earlier, but my memory is only above average, which is to say that it's absolutely terrible when measured absolutely and not relative to other humans.

Now that ChatGPT has been out for a while, I've seen quite a few "thinkpieces" about how ChatGPT that talk about the danger of the ability to produce large amounts of vaguely plausible but also obviously wrong text since most people don't check plausibility.

I don't really disagree, but I find it ironic that these pieces tend to contain the same kinds of errors they're calling out, e.g., the recent one that said "/." is a CLI reference (I asked ChatGPT and it told me a similar wrong thing) and

had the standard pop-misconception about the dark ages and the englightenment (unike the author, ChatGPT didn't get this wrong).

Without context, these pieces could appear to be ironic ChatGPT fuelled articles, but they're all from people I recognize as having produced similar work for years, which makes it seem like the articles are genuine.

Scale and accessibility matter, there's a big difference between ChatGPT and these authors, but the irony still tickles my funny bone.

I've also seen a lot of the reverse — people saying that, e.g., professors saying that ChatGPT is a big deal is wrong because, actually, students produce better work because ChatGPT makes such obvious errors.

IMO, as someone who's graded assignments at two different decent universities (worldwide rank ~50 and ~100), that seems quite wrong.

For short essays (the kind people often get for homework or exams), ChatGPT is well above median student performance on short essays at decent schools.

Shortly after ChatGPT was released, I sat down with a professor from a top tier university and we gave it a bunch of prompts from her class. Her rating of the output, depending on prompt, was somewhere between "as good as a bad student" to "better than any student I've ever seen", with typical being above median.

The next day, she showed her (non-CS/EE) class and they were profoundly disturbed by the implications since they immediately recognized that ChatGPT's output was better than their own.

It's easy for a mid-career tech person who could retire today to say ChatGPT and its successors aren't a big deal because it doesn't really matter if they're wrong.

But, it turns out, if you give a room full of top non-CS students ChatGPT and ask them what they think, they understand which of their skills might be obsoleted soon and which ones are potentially on the chopping block and they're not so sanguine about the situation.

I think that anyone who really spends time talking to people outside of tech will have seen this long before ChatGPT, e.g., if you talk to lawyers, various tech things have automated away a lot of legal tasks and really changed how law firms look, etc.

Combined with the above, the unexpected bankruptcy of Dewey LeBoeuf a decade ago and the impact on some people's careers really made a lot of lawyers I know feel career-unsafe in a way tech folks haven't experienced since 1999.