Tuesday, May 3, 2022
Episode 198: We Are Not So Smart
Play Now!We humans bear complex mental mechanisms, which are often general enough for experiments to tease out some rules that govern our behavior. Knowing how these rules can manipulate us helps us realize the title of this Episode 198: We Are Not So Smart.In this episode, I read from: my computer's quickie dictionary; David McRaney's You Are Not So Smart (Penguin Random House, 2012); and Rose George's Ninety Percent of Everything: Inside Shipping, the Invisible Industry That Puts Clothes on Your Back, Gas in Your Car, and Food on Your Plate (Metropolitan Books, 2013). Strictly from memory, I shared details from the movie Flight From Death.I play: Pee Wee Herman commenting on large exceptions; and a bit from Animal House of Neidermeyer swinging the bat to induct a young Kevin Bacon to the ways of fraternity life… and economic life in general. Musically, Tristan Harris opens the show with KMFDM. I close with Julie and Rolf and the Campfire Gang doing "Over the Rainbow".
Subscribe to:
Post Comments (Atom)
I enjoyed this episode! However I would like to send you to some additional reading about the "Dunning Kruger Effect".
ReplyDeleteSorry to send you over to Fkkkkkbookkk. But awhile back I posted an article about Dunning Kruger that I think you should read.
( https://www.facebook.com/kevin.wohlmut/posts/pfbid0bJbQyGthm8TDXTgHBoFCJFPbD4wTBboZRZ8BFLwdNzT93FLwKmCZCjeuqhgzXetjl?__cft__[0]=AZWCoyFaW-fOtbgTko8TNChE8r0guV-O-nrOWC7bTS1q9bkPnRjQUQy-VLJR2Fb7qgVCG-FpI1EmSVjVH05VywsPRhBI5_j_lis3NVn7CsDMFA&__tn__=%2CO%2CP-R )
I must admit to a certain glee upon discovering this article, and general hostility to the D-K effect as it is typically used in colloquial discussion. Because in my experience, it's almost always directed _against_ me as a pejorative by laymen who think they're experts in politics, ironically enough. I'm not saying _I'm_ an expert in politics, but neither are the people who use the D-K effect as a weapon against me. The mere mention of D-K just sets my teeth on edge these days.
[SPOILERS]
[SPOILERS]
The gist is that the original writers of the study, Dunning and Kruger, regret launching that paper into the world, because they feel that everybody misuses their results. They emphasize that the only thing they measured was _self_ assessment, therefore it's utterly inappropriate for somebody _else_ to tell _me_ (or anyone else besides themselves) that they don't know as much as they think they know. The D-K effect applies only to self-assessment, nothing more. They emphasize that their study was numerical, where experts and non-experts estimated their numerical score on tests where they did or did not know the subject; they didn't take any steps to study or assess _confidence_ in their scores or their knowledge, which is how everybody seems to interpret their work.
And then besides that, the author of the article I linked to, says that some mathematicians conjecture that the D-K effect is nothing more than a statistical artifact which is basically the same as "returning to the mean". If you flip a coin six times and the first five of them come up heads, there's no Law of the Universe that says the sixth flip _has_ to come up tails, but tails is what usually happens because trials tend to "return to the mean". Likewise, if you self-estimate that you scored high on a test (without saying anything about your own confidence in your knowledge, only about how easy or hard the test was) then there's simply more "room" statistically for your actual score to fall below your high estimation, than for your actual score to beat your high estimation.
I thought the article was interesting and well-written, so I thought you'd like to read it. I'm sending you to Fukkkkkbookk instead because I wanted you to at least glance at my comments about how the D-K effect in my experience has always been used as a weapon _against_ me by others, rather than a tool of self-estimation as the original study team intended.
If I take a test in a subject that I _literally_ know _ABSOLUTELY_ nothing about, say I score a 0.000%. Then I estimate my score. I can't estimate my score at anything lower than zero. So it's more likely I will estimate something equal to or higher than zero, my actual score. I have nowhere else to go.
DeleteThe gist is that the original writers of the study, Dunning and Kruger, regret launching that paper into the world, because they feel that everybody misuses their results.
DeleteSadly, happens all the time, right?
The D-K effect applies only to self-assessment, nothing more.
That jives with what I remember.
They emphasize that their study was numerical, where experts and non-experts estimated their numerical score on tests where they did or did not know the subject; they didn't take any steps to study or assess _confidence_ in their scores or their knowledge….
Uhhh… isn't a self-estimate on their own score… a measure of confidence? I remember when the study made the rounds about a decade ago. That was the big take for me.
Hey, it's been years since I looked into it. Maybe McRaney has interviewed Dunning and/or Kruger. I should look.
Most importantly, though…
I enjoyed this episode!
Thanks, Dude!
—Jim, Still Attacking
Just to complete the pedantry, I would quibble that I do _not_ believe an estimated numerical score is a precise measure of confidence in the basic subject knowledge. There were plenty of times in college when I took a test, and I knew the material, but I was hung-over and so forth, and so I feared that I had bombed the test. A high numerical estimate _could_ be an indication of confidence in the base underlying subject matter, but it doesn't _have_ to be. I would not say it is a rigorous measure. To pick another example, if someone was about to give me a test about Old Norse Mythology, I wouldn't have much confidence in my grasp of the subject matter. But if the test was extremely easy and revolved entirely around what you see in Marvel movies ("Who is Thor's brother? How many eyes does Odin have?") I'm pretty sure I could ace it. That does not mean I consider myself an expert in real Old North Mythology, but I could pass that test with flying colors.
DeleteJust to complete the pedantry, I would quibble that I do _not_ believe an estimated numerical score is a precise measure of confidence in the basic subject knowledge.
DeleteYeah, I got that from your content. For me, it doesn't have to be "precise," only "just enough." Statistics may not show everything, but they can show something.
A few years after the DK study was all over the InterWebs, someone refined the study in a way more to your liking, allowing people to assign a confidence value to every question. If you were really, really confident, you gave it a 100%; if you were guessing, you gave it %50.
The adjustments led to an overall score more consistent with what you seem to be saying. Still, as a rough estimate of expertise….