Asked Bing AI how to switch Hercules into graphics mode. These things are fucking useless.
@sos Useless for facts, yes. Using an LLM to try to get factual data is like hammering a screw. Sometimes it works, but most of the time, it's simply the wrong tool for the job.
@LouisIngenthron I don't think it's a tool for any job
@sos I think you're wrong there. It's great at identifying, classifying, and condensing text. It's also useful in the medical simulation world for anonymizing patient data by making pattern-based fictional amalgams. It's also a great tool for scaffolding and creating placeholder content that would otherwise take a bunch of time to develop by hand. And all that's before we get into the code-completion versions of the software or entertainment purposes.
Right now it's a fad and the square peg is getting put into a lot of round holes, but when you apply the tool to what it's good at, you get really solid results.
@LouisIngenthron But how can it be good at these thigns if you can't rely on it to provide fatual data?
@sos Because those are all situations in which it doesn't need to draw on outside facts, or the factual nature of the output is irrelevant.
I.e. in identifying, classifying, and condensing text: I trained an ML algorithm to look at a website's HTML and tell me if it had a meal recipe (ingredients/directions/etc) somewhere in its contents. I was able to achieve over 99% accuracy, even when pulling from vastly different websites.
That's because, at their root, these are not fact machines; they're pattern matchers. So, when you give them a pattern-matching problem like "find the recipes", they don't need to draw on the larger model training to create content; they just do the job you trained them for. The medical example is similar; it's just extending a pattern identified from real patient data to generate fake patient data.
@LouisIngenthron I don't think that's a good example because you could as easily create a script to find you ingredients and snag steps from a website using some keywords and regex will similar accuracy.
@sos Go ahead and try it then. It needs to work on different websites with vastly different layouts and formats, and it needs to be able to reliably exclude pages that are not recipes (even when those pages are confusingly similar, such as lists of recipes).
I was able to build that system to work universally with a locally-trained LLM in just a couple days with over 99% accuracy (it used a web-crawler, so the input was pretty solidly random). The LLM was even able to fix measuring-unit formatting issues so ingredients could be imported uniformly into a database.
I could be wrong, but I think you'd be pretty hard-pressed to get similar results with more traditional methods in the same time frame.
