A lesson recalled...

The architecture of PalmPilots allows you to chunk information into 4 KB segments that eventually disappear in a crowd of similar 4 KB segments, particularly among so-called "memo" notes. I've taken to reviewing these items lately, since I seem to be running chronically short of memory despite having 4 MB of RAM on my Palm device. I stumbled across the following item today, which reminded me of the importance of...well...read on...

The memo in question was jotted down in the course of listening to a consultant lecture on time management. He told the story of a session he'd taught at a well-known business school, and of a demonstration he'd done to drive home a point.

As he stood in front of a class of high-powered overachievers, the expert placed a one-gallon, wide-mouthed Mason jar on the table in front of him. Then he produced about a dozen fist-sized rocks from a paper bag and carefully placed them, one at a time, inside the jar. When the jar was filled with rocks, to the point where no more rocks would fit inside, he asked the class, "Is this jar full?"

The general consensus of the class was "Yes."

"Really?" exclaimed the expert, whereupon he pulled out a container of gravel from a catalog case. He proceeded to dump a little gravel in the jar, then he'd shake the jar to let the gravel percolate down into the spaces between the rocks, and then he'd repeat the operation until no more gravel would fit in the jar. He then asked, "Is the jar full now?"

Suspicious of hearing what seemed to be a "trick" question, the class hedged their answer. "Probably not," was the reply.

"Good!" said the expert, reaching under the table. He brought out a bucket of sand and proceeded to pour it carefully into the jar so that it filled the spaces between the rocks and the gravel. After fiddling with the jar like that for a long time, no more sand would enter the jar. He again asked, "Is the jar full?"

"No!" came the answer. "Good!" cried the expert. He then took the pitcher of water that had been set out for him to refresh himself during his lecture and started to pour water into the jar until the water level was up to the brim of the jar.

Putting down the pitcher, the expert asked, "So what's the point of this exercise?"

One eager student volunteered to answer. "The point," said the student, "is that no matter how full your schedule is, if you try really hard, you can always fit some more things into it." His classmates nodded their heads in general agreement.

"No," said the expert, "you missed the point. The truth I'm trying to illustrate is that if you don't put in the big rocks first, you'll never get them in at all.

So, what are the "big rocks" in your life? I'm just starting to get reacquainted with mine...

Cheers...

The Code Book

I got Simon Singh's The Code Book as a Christmas present last year. It's a good read and covers quite a bit of ground in the area of cryptography, starting with the earliest cryptanalytic efforts that eventually cost Mary, Queen of Scots her head, and finishing with a description of cryptographic techniques that could make use of quantum theory to assure the secrecy of underlying messages.

Besides the information in the text, the author includes a series of ten cryptographic challenges, starting with a monoalphabetic substitution cipher and progressing through more complex techniques (and through history, at the same time), arriving at messages encoded using state-of-the-art techniques. A sizeable cash prize was offered for the first solution to all ten puzzles. Back around the time I got the book, solutions to the first seven or eight puzzles had been found. Today, I learned that the tenth and hardest puzzle had been cracked by a group of Swedish programmers. You can find the details on Singh's Web site.

Not having a lot of time (and apparently, not a lot of patience, or talent, or what-have-you), my efforts to find solutions sort of tapered off just after solving the fourth challenge. But it was enough of an experience to let me know that it's one thing to read a description of how to slay a dragon (or solve a cryptogram), and quite another to actually deal with one "in the wild." It's also quite a bit of fun, if you're a puzzle-solving kind of person (the British, by the way, enlisted a number of puzzle-lovers in their effort to break the German Enigma cipher during WW II, including chess players and crossword puzzle whizzes). In tackling the fourth cryptogram, I experienced quite a rush after writing a Perl script that successfully analyzed the puzzle, which was encoded using a Vigenère cipher scheme, and then rewriting the script to actually solve for the underlying message.

Running the first script allowed me to determine, to a fair degree of certainty, that the "key" to the cipher was five letters long. Assuming that only the 26 letters of the alphabet were used and that case was not important, that meant there were "only" 26 x 26 x 26 x 26 x 26, or about 12 million possible keys to consider. Since computers are really good at doing things over and over, the solution, therefore, lay in trying them all.

But this raised a secondary problem, which was finding a way to avoid having to eyeball the output of the "brute-force" solution. (Assuming it takes one second to review the output from one "try," it'd take just over two solid months to review half the output. Yech!) I met the challenge by having the script analyze the output for the presence of common English digrams and trigrams (e.g., letter combinations such as th, the, an, ion, ed, etc.) and the absence of uncommon letter combinations (e.g. bx, qj, and so on). Each "try" was then assigned a numerical score, based on the result. The script ran overnight on my old 486 box and presented me with one solution whose analysis score far outstripped the scores for the remaining 12 million tries.

Upon viewing the solution, I realized I'd come close to missing the mark altogether, because one of my assumptions was dead wrong (but not so wrong as to prevent finding the solution). Can you guess what my slipup was?

Cheers...