Information Theory - Part 2: Maxwell's Demon


How can a theory about information and noise be considered as profound as relativity and quantum theory? Sounds ridiculous, right? So let’s check out the first big science-y problem that information theory solved.

The second law of thermodynamics forbids the existence of a perpetual motion engine (That’s a machine that can run forever without any intervention that involves more work than the work you get from the machine itself). Why not, asked James Clark Maxwell, with his famous thought experiment known as Maxwell’s demon, devised in 1871.

But first, a recap of some basics. Heat flows from a hotter to a colder place. As that heat flows, you can use (part of) that heat flow to do work. By definition, this means that once the two sides are at the same temperature (same heat), heat stops flowing (neither end is hotter anymore). And without heat flow, no work can be done.

We also know that heat is just a measure of the average speed of molecules. And since those molecules have random speeds, some would be faster than others. Enter Maxwell’s demon:
1)      Take a container with a gas inside it and a barrier at the center. Some molecules would be moving faster (red), others slower (blue).
2)     The demon would measure the speed of a molecule approaching the barrier and open/close the barrier to ensure the faster molecules end up on one side and the slower ones on the other side. Pics from this site.
3)      Eventually, all the faster molecules will be on one side and the slower ones on the other side.
4)      By definition, the side with the faster molecules has a higher temperature than the other side with the slower molecules.

5)     Now connect the two side (without the demon) and heat will flow from the hotter to the colder side.
6)     Tap into that flowing heat to get some work done.
7)     Once the two sides get to the same temperature, heat would stop flowing. Bring in the demon again and repeat steps 2 to 7. Endlessly. And voila! You have a system that can work endlessly. The demon was just utilizing the inherent speeds of the molecules, not speeding or slowing them down!

Nobody could explain the flaw in this thought experiment. Until, in 1982, information theory solved the puzzle. The key lay in the realization that the demon is an information processing machine (measure speed; close/open the partition based on measured value). Information processing involves writing, reading and erasing information. No matter how large the memory of the demon, at some point he runs out of it, and then he has to erase to “make space” to continue doing what he does. It is that last action (erasing) that increases the entropy of the system (box + demon) more than the reduction of the entropy of the container. As Charles Seife put it in Decoding the Universe:
“Bennett proved that the demon always has to reduce entropy of the container at a cost: a cost of memory, and then at a cost of entropy of the universe. There was no free ride, no perpetual motion machine. Maxwell’s demon was dead at the age of 111.”

Information theory had solved one of the longest standing problems in physics!


Comments

Post a Comment

Popular posts from this blog

Student of the Year

Image
As a toddler, my daughter loved Alia Bhatt’s songs, esp. the ones from Student of the Year . (That movie did have many good songs and well-choreographed dances). Fast forward to present day, when she is 12 yo. She rarely watches Hindi movies now, but one of the few ones she likes of late, Raja aur Rani ki Prem Kahani has, yes, Alia Bhatt.   If I had any hopes that there was anything to be read in the name of her toddler-time favorite ( Student of the Year ), they have been dashed. Our conversations come exam time are perfectly captured by this photo and its caption: In fact, the pic below captures how she and I look at her answer sheets – me intently, she with an expression of “It’s over, so why are you even looking at it?”   Of late, we pull her leg and tell her that she isn’t showing any signs of academic prowess, she’d better find a rich guy to marry. Some time later, she showed us this meme on the Internet: “Me No Study, Me No Care Me Go Marry Millionaire If He Die
Read more

The Retort of the "Luxury Person"

Image
Coming up with biting and witty retorts. It’s something we dream of, but rarely manage to pull off. Worse, the more we stew over the offending remark, the more likely we are to come up with that dream retort, as Calvin ruefully points out: During our last holiday at the Taj Resort, seeing my wife carrying towels to our poolside chairs, one of the kids there mistook her for a member of the staff and asked for a towel. When my wife narrated this to us, I did not imagine that my 8 yo daughter would come up with her own retort. But a day later, my daughter recapped the incident, an incident that didn’t even involve her in the first place , and said, “You know what I’d have told that kid who asked for the towel?”. And without waiting for an answer, she told us anyway: “I don’t work here. I too am a luxury person who is living at this resort. Go get your own towel.” We were highly amused by the phrase she had coined: “luxury person”. It certainly captures the kind of holid
Read more

Animal Senses #7: Touch and Remote Touch

Instinctively, we think touch is based on contact. Not entirely true, explains Ed Yong in Immense World – there’s a more important aspect than contact: “These incredible feats are possible through movement. If you rest a fingertip upon a surface, you can get only a limited idea of its features. But as soon you’re allowed to move, everything changes. Hardness becomes apparent with a press. Textures resolve at a stroke.” As you run your fingers over a surface, they hit the tiny peaks and troughs and set off vibrations in the mechanoreceptors at their tips.   At the tip of its snout, the star-nosed mole has many pairs of hairless appendages. It uses them to touch things around itself. Alive or dead? Food or ignore? It does these movements at an insane speed. In the time it takes to blink our eyes, the star-nosed mole can probe, identify and swallow its food. “Light may be the fastest thing in the universe, but light sensors have their limits, and the star-nosed mole’s sense of
Read more