Do you know how a bicycle works? If asked, could you say where the chain, pedals and frame are? According to a 2006 study by the University of Liverpool, maybe not.

Participants in the study were asked to draw a picture of a bicycle. Later, to make sure that lack of artistic skill wasn’t a factor, they were asked to view pictures with different arrangements of chains, pedals and frames to state which corresponds to a working bicycle.

The result was that over 40% of participants couldn’t do it. Many participants drew bicycles that would be completely non-functional.

It’s amusing to think of what the person drawing these bicycles must have been thinking, such as locking the two wheels together via the frame so it can’t steer, or attaching both wheels to the chain. Before you get too smug, though, try testing yourself. Without looking, could you explain how a can-opener or a zipper works?

This bias—to believe we understand how familiar phenomena work far better than we actually do—is called the illusion of explanatory depth.

This type of overconfidence may be even worse for explanations than it is for remembering facts and trivia. Explanations often have different layers of depth. For instance, you may have correctly remembered that the chain of a bike attaches to the pedals and rear wheel, a relatively superficial explanation. But you may not know how the bike changes gears from the handlebars or how the brakes work, a deeper explanation.

This bias plagues students and knowledge workers, into thinking they understand things they don’t. Worse, the overconfidence sometimes causes people to blame the wrong problem when they can’t perform on tests or tasks.

Why Students (Incorrectly) Fault Memory When They Fail Tests

I get a ton of student emails, and one of the most frequent is a student blaming memory for poor test performance. A typical email would read something like this:

Hi Scott,



I struggle with remembering things for my tests. I understand everything in the class, but when the test comes around I seem to forget all of it. How can I memorize things better?

-Student

On one view, the student isn’t entirely wrong. Failure to produce information for a test can be seen as a failure of memory. But this is misleading, since after this diagnosis, the student decides that the best way to improve is to simply get better at memorizing all the information.

Unfortunately, many of these students are suffering under the same illusion as the participants in the bicycle experiment. They mistake their ability to recognize the logical operation of something with their ability to understand and explain it.

Few people would be baffled as to the function of a bicycle when they see one. For one, the object is too familiar to be surprising. Second, the operation seems readily transparent to inspection. You can push a pedal, see how the chain moves and how that rotates the back wheel.

But this ability to recognize and probe an object in front of you, is very different from the ability to correctly explain the object when it is out of sight.

The solution? Self-testing and tools like the Feynman Technique. Only by forcing yourself to explain does it become apparent how little you understand. Practicing explanations is also the key to form the kinds of memory you need to perform later.

Students suffer from this tendency very obviously, but I believe this also applies as a logical extension to many areas of knowledge work. Often the tendency is to believe because you’re familiar with something that you necessarily understand how it works. While that ignorance may not be a detraction for routine usage, it does impede creative solutions for novel problems.

My theory: good students (as well as programmers, designers, writers, etc.) cultivate a curiosity to test their own understanding of familiar things which combats the illusion of explanatory depth and builds expertise.

This may sound ridiculous, but I'm serious. The goal is to write a spec for what's allowed to be put into a refrigerator. I intentionally picked something that everyone has lots of experience with. Here's a first attempt:

Anything that (1) fits into a refrigerator and (2) is edible.

#1 is hard to argue with, and the broad stroke of #2 is sensible. Motorcycles and bags of cement are off the list. Hmmm...what about liquids? Can I pour a gallon of orange juice into the refrigerator? All right, time for version 2.0:

Anything that's edible and fits into a refrigerator. Liquids must be in containers.

Hey, what about salt? It fits, is edible, and isn't a liquid, so you're free to pour a container of salt into this fridge. You could say that salt is more of a seasoning than a food, in an attempt to disallow it, but I'll counter with uncooked rice. This could start a long discussion about what kinds of food actually need refrigeration--uncooked rice doesn't, but cooked rice does. Could we save energy in the long haul by blocking things that don't need to be kept cool? That word need complicates things, so let's drop this line of thinking for now.

Anything that's edible and fits into a refrigerator. Items normally stored in containers must be in containers.

How about a penguin? Probably need some kind of clause restricting living creatures. Maybe the edibility requirement covers this, except leopard seals and sea lions eat penguins. No living things across the board is safest way to plug this hole. Wait, do the bacteria in yogurt count as living? This entire edibility issue is troublesome. What about medicine that needs to be kept cool?

Oh no, we've only been thinking about residential uses! A laboratory refrigerator changes everything. Now we've got to consider organs and cultures and chemicals and is it okay to keep iced coffee in there with them. It also never occurred to me until right now that we can't even talk about any of this until we define exactly what the allowed temperature range of a refrigeration appliance is.

In the interest of time, I'll offer this for-experts-only spec for "What can you put in a refrigerator?":

Anything that fits into a refrigerator.

That is the early “computer,” remember?:

Who made the famed Antikythera Mechanism, the astronomical calculator that was raised from an ancient shipwreck near Crete in 1901?

The complex clocklike assembly of bronze gears and display dials predates other known examples of similar technology by more than 1,000 years. It accurately predicted lunar and solar eclipses, as well as solar, lunar and planetary positions.

For good measure, the mechanism also tracked the dates of the Olympic Games. Although it was not programmable in the modern sense, some have called it the first analog computer.

We now learn that the calendar of this mysterious device begins in 205 B.C.The key point, in my view, is that we have discovered no other comparable machine from antiquity or any other era other than modern times.  It took us until 2006 to even understand what the device was supposed to do, using advanced tomography, and we had been holding it since 1901.

So what to infer?  The first option is that this device was a true outlier, standing sui generis above its time.  Cardiff University professor Michael Edmunds “described the device as “just extraordinary, the only thing of its kind””.

As an artifact that is true, but is that so likely in terms of broader history?  It is pure luck that we fished this thing out of the Mediterranean in 1901.  (By the way, further dives are planned to search for more parts of it.)  The alternative possibility is that antiquity had many more such exotic devices, which have remained unreported, at least in the manuscripts which have come down to us.  That would imply, essentially, that we don’t have a very good idea of what antiquity was like.  In my view that is the more rational Bayesian conclusion.  It is more likely than thinking that we just lucked out to find this one unique, incredible device.  To put it another way, if you found some organic life on a traveling comet, you ought to conclude there is more of that life, or something related, somewhere else.

And to me, the Antikythera Mechanism does not to me sound like a “lone genius” kind of device: “The gear teeth were in the form of equilateral triangles with an average circular pitch of 1.6 mm, an average wheel thickness of 1.4 mm and an average air gap between gears of 1.2 mm.” (Wikipedia)  That suggests it was made by some kind of regular industrial process.  It also had some sophistications which modern Swiss watches do not.

Given this Bayesian conclusions, which other strange claims stand a decent chance of being true of antquity?  Which other surprises await us?

I find this an interesting passage: “the mysterious device was already pretty ancient by the time it went down some time around 85BC to 60BC with a ship carrying a bride and her dowry, io9 reports…”  You don’t find a lot of people carrying around a lot of ancient PCs today, so might there have been an Antikythera Great Stagnation way back when?  I think maybe so.

Here is a Lego model of the device.  Here is an introductory YouTube video.  Here is Wikipedia on the Antikythera Mechanism, a very good entry.

I owe thanks to Vic Sarjoo for pointers and Robin Hanson for a useful conversation on this topic.