I woke up Wednesday to this nonsense on my driveway. A little pattern of pellets bisecting the driveway.
I’ve been trying to expand the way I use AI when possible, so I uploaded that photo to ChatGPT and said “What is this nonsense?”
In summary, it said, “deer poop.” You’ll get a deer who poops while walking — efficient, I get it — and instead of a pile you see it distributed in that scatter shot effect.
6 responses to “TWIL: How to identify the debris on your driveway”
Is it big enough for deer poop?
(but also: please do not try to expand your use of LLMs; ChatGPT lies and lies prolifically and confidently and flatteringly [did you see Richard Dawkins and Claudia???] and it rots the brain [so many studies showing decreased memory and also more drastically decreased capability and willingness to make mental effort; it is the opposite of doing crossword puzzles or learning how to dance] and it is just a Bad Plan even if it is sometimes correct. Text LLMs are legitimately useful for some things done by the average person – changing the tone of an email to be less angry when you’re angry [but: always read it over for introduced factual errors], generating banal plausible business newslettery content [but again: check for errors], doing some basic coding [but check for fake libraries that people have planted malicious code into] – but even then it’s a plagiarism machine enriching some fairly inhumane billionaires and raising electricity prices and using up scarce water in areas which are already in drought and making scams and phishing and disinformation campaigns and search-result-rigging a lot easier while poisoning the well of public information and making social cohesion even more fragile and threadbare than it needs to be?)
(I hate the stuff, can you tell? 🙂 )
KC – It is the right size for small deer poop, and it’s spring, so I figured it was a fawn.
But my question is how is ChatGPT less reliable than Google? I mean, I know ChatGPT is more likely to skew information to “please” me but I’m on guard for that. And I see your concern that it can be used for evil / plagiarism – but Google can be used that way too, right? I wonder if Snopes could make its own AI.
(Not just containing Snopes data like Factbot.)
So LLMs are predictive text; they’ve been fed the internet as a whole (including reddit, satire, random hobby forums) and a *ton* of books and other works (usually pirated). If everything they’ve used to “train” on a subject is correct, and if they do not get muddled by sentence structures into flipping “not” around, and if there is *enough* for them to train on, then more frequently they are correct. BUT if any of those fails, the LLM’s accuracy fails… while still remaining fully confident.
So as to which sources to use: if you personally would google, see an Onion article and take it as fact rather than satire that it’s likely Obama *is* a cactus, or uncritically accept “glue on your pizza toppings” or “it’s important to eat one rock per day” advice from a strange little corner of Reddit, then… well. That aspect would be about equivalent. (buuuut I’m guessing that you do in fact look at sources and consider text quality and go “huh, should I listen to Reddit’s misspelled concept of these test results or The Mayo Clinic’s concept of these test results or what medical textbooks from the late 1800s said about this?” and have a strong preference there; and it is *possible* that you trust the AP’s news reporting on recent events a smidge more factually than you would trust the factual level of a) SNL transcripts or b) vaccine-microchips-exposed!-blog reporting or c) “one weird trick” supplement company pages or d) historical archives reporting that is *from the wrong century for this event* – but AI will cheerfully use any of these as a training basis for its information-shaped sentences)
The other aspect, however, is *making up stuff about information that doesn’t exist in the training data* and then confidently presenting it, and if you do regular-search search for something and either 1. literally nothing comes up or 2. the results clearly don’t give an answer to your question, then searching without AI Assist for information that just *isn’t out there* will show you that: there are no sites, or the top options do not in fact tell you anywhere on the page that, say, peach pits are a great natural deodorant, or Queen Mediocretia is a pen name of Dave Barry, or what your right-hand neighbor’s favorite flavor of cake is, or John Scalzi was a chain smoker, etc., but AI will cheerfully weigh in on these things with confidence and flattery and without *any* real factual basis, other than, say, approximately the statistical probability generally for its conclusions. (i.e. how often, in text, anyone’s neighbors have X as a favorite cake flavor, or if you are asking how a named individual’s “battle with cancer” is going, it’ll give you an answer usually assuming that they *have had* or *are having* a battle with cancer, because few magazines/newspapers/biographies say “[celebrity name] has never battled with cancer” or “it is a popular misconception that [celebrity] has cancer, but they do not and never have”?)
So those are the two main “nooooo not AI” glitch modes: having pulled information from sources you would probably snort-laugh at the idea of using for information plus potentially misreading its data, or Confidently Making Things Up to fill in for inadequate training data on that topic (it takes a LOT of data on a topic to train an AI strongly enough that it *mostly* provides correct fill-ins).
KC – well, the source issue is the same with Google. They both show the source on screen and it’s on you if you miss it. Excellent, memorable point about the hallucinations. And it can’t do basic math; I found that out too. But the AI people at work would reccomend you write a prompt to make the AI cross-check itself. And of course I cross-check the AI when I’m at work. I did double-check the deer poop in Google.
AI will cite things that *don’t contain what the AI is saying* but yes, if it’s giving you “source: dolly-parton-is-elvis-in-disguise.net” for your important health information, and that doesn’t raise eyebrows, then that is on you to some degree [but also the font size is probably smaller?]… (but Always Click Through, since Google has also started messing with the regular-search page snippets and page titles in the past year, so websites and their contents are being misrepresented *by the search engine* which I [and a lot of news websites which specifically *don’t* give un-nuanced titles to their stories] find maddening.) (or use DuckDuckGo and turn AI off; there are options.)
Double-checking is good! Yay for you! 🙂 AI is just such an unreliable narrator… which a lot of people have come to rely on fully because it is So Confident and So Easy and because it’s being shoved on them, and people do get tired and just want to take shortcuts sometimes, but then when they do rely on it for a while, their “actually read and assess information in paragraphs with consideration for the source” metaphorical muscles are atrophying with impressive speed, leaving them increasingly resistant to doing any of the double-checking work.
It’s not healthy for any individual person, but also it is not a good loop for “informed public capable of acting with competent enlightened self-interest in a democracy” on the whole [in addition to the just plain hallucination, and the failure to screen sites during training for whether their information is Reliable or Deliberately Poisoning The Information Well, consider also who owns and can directly tweak the answers provided by AI and whether their interests have, uh, aligned with the general public interest, in recent years?].
KC – I think now that my job has me use AI, but also requires I fact check the AI, I know how lazy and sloppy the AI can be. It seems if it’s a single factual question it is fine (I.e. What is this nonsense on my driveway) but if you ask for fifty different multiple choice questions it takes shortcuts. (Giving you the same question with one word changed.)
But I am happy to do fact checking over creation.