All About Cryptography

Podcast Transcript

Ever since people have had secrets, people have taken measures to protect those secrets.

The first methods to hide secrets were simple and mechanical. Over time they became more elaborate and used machines. Today, they are mathematical and would require an enormous amount of computing power to decipher.

Learn more about cryptography and how communications are kept secret, on this episode of Everything Everywhere Daily.

Cryptography is simply defined as “The process or skill of communicating in or deciphering secret writings or ciphers.”

As we’ll see, the definition has changed over time, and the modern definition has much more to do with mathematics than it does with the more generic idea of ciphers.

That being said, as with pretty much everything I talk about on this podcast, the origins of cryptography go back to ancient times.

If you go back far enough, there really was no need for cryptography. Literacy was something that only a small number of people might have had in a society, so they would have been cryptographers by default.

Ancient Egyptian hieroglyphics is a good example. Unless you know the code, it is pretty much impossible to decipher exactly what the symbols meant. This was why Egyptologists couldn’t crack hieroglyphics until the discovery of the Rosetta Stone, which I talked about in a previous episode.

As literacy spread and systems of writing became simpler, there arose a need to hide messages.

The earliest known system ciphers arose independently in many places including Mesopotamia, India, Greece, Rome, and Israel.

The system that all of these civilizations used was simple substitution ciphers. A substitution cipher is when you just substitute one letter of the alphabet for another. In its simplest form, you just shift the letters over by a set number.

Julius Caesar used a simple substitution cipher by moving the letters by three. So A would become D, B would become E, and C would become F.

If I wanted to encrypt the word “apple” I would write “dssoh”.

In order to decrypt the message, you would just need to know by how many letters I offset the code.

The Spartans used something called a Scytale cipher. With this, you would wrap a long thin piece of parchment around a baton. You would then write the message across the wound parchment.

When it was unwound, it was just a bunch of letters on a strip. To decrypt it, you’d just need a baton of the same diameter as the person encoding the message had.

These early cipher systems were really easy to use, but also really easy to decrypt. Once you knew that the person whose message you were trying to read used a substitution cipher, you only had to try a set number of possibilities before you could crack it.

Cryptography wasn’t a huge focus for the ancients. A message could use a basic substitution cipher, but mostly you’d just want to hide the physical message altogether.

The big advancement in cryptography came from Arab mathematicians in the 8th century.

In particular, the Islamic scholar Al-Kindi was the first person to a study of frequency analysis on characters in the Arabic alphabet. What he found, and is true of pretty much every language, is that some characters are used more than others.

For example, for the average text written in English, the letter ‘E’ will be used most frequently at 12.6% of the time. The next most common letter is ‘T’ at 9.37% of the time, and then ‘A’ at 8.34% of the time.

The least most commonly used letters are “Q” and “Z” which is not at all surprising if you have played Scrabble.

If you can look at the frequency of the letters used, then you can go a long way towards deciphering a message.

This sort of frequency analysis is still used today, and it has become even more important as the ciphers became more complex.

Instead of just shifting the alphabet like the Caesar Cipher, far more complicated systems were developed which replace letters with random letters, multiple letters, and numbers.

I should note that for any system of encryption that was developed at this time, it had to be able to be decrypted by someone, and it had to be able to be decrypted by hand. There were no computers at this time, or for that matter, even mechanical encryption machines.

There wasn’t really a lot of advancement in cryptography up through the 19th century, because all communications were still via written letters.

However, with the advent of the telegraph and wireless communication there arose a new problem. Encrypting messages where you could be certain the contents of the message would be intercepted by your adversaries because it would be broadcast openly.

It was poor encryption that resulted in the decryption of the Zimmerman Telegram which was one of the reasons that brought the United States into World War I, which I covered in a previous episode.

One huge advancement which was made in the early 20th century was the development of the only method of cryptography which cannot even theoretically be broken: the one-time pad.

A one-time pad can’t be decrypted because there is no code. It’s all random.

To use a one time pad, you need a key that is at least as long as the message to be encoded, has to be randomly generated, can never be used more than once, and of course, the keys have to be secret between the two parties with the message.

The whole trick is keeping the keys secure.

The hotline between the White House and the Kremlin during the Cold War used an electronic one-time pad. There still might be military and diplomatic communications that use a one-time pad today if it absolutely, positively needs to be kept secret.

The problem with one-time pads is the distribution of the keys. It might be fine if you wanted to send infrequent messages between two points, but what if you wanted to send messages to all your military forces in the field?

A one-time pad just wouldn’t work very well.

World War II saw the introduction of electromechanical encryption machines. The most famous of which was the German Enigma machine.

The Enigma machine was a device that had three rotors of letters, plus a keyboard and electronics circuits. Every day, the machine would have to be set up with new settings for that day. When the operator would input a letter from the encrypted message on the keyboard, it would make the rotors rotate, such that the next letter which was entered was using an entirely different substitution.

So, if you saw an encrypted letter, the letter ‘A’ might represent an ‘L’ the first time it is used, but the next time it might represent the letter ‘R’.

The only way you could decrypt a message would be to have both the machine and the correct settings for that day.

There were 150 trillion possible ways that messages could be encoded using the machines.

The British eventually cracked the Enigma code with great difficulty and one of the first applications of an electronic computer.

Japan’s encryption machine known as the Type B was also cracked during the war.

Most people don’t know that the British and Americans both used rotor-based encryption machines of their own during the war and afterward. The British system was known as TypeX and the American system was known as SIGABA.

As far as we know, no one ever cracked either system.

It wasn’t until after World War II that cryptography really came into its own as a science. This was due to the rise of digital computers and the digitization of communications.

The father of digital cryptography was Claude Shannon who established the foundation of the discipline with his 1949 paper “A Mathematical Theory of Cryptography”. Claude Shannon is important enough to the field of computing that I’m going to devote an entire episode to him in the future.

He took the idea of cryptography out of the realm of extremely complex letter substitution and brought it into the realm of mathematics.

You probably use digital encryption every day without knowing it. Every time you visit a website that starts with HTTPS, you are viewing a page that was encrypted, and then decrypted before it was displayed on your browser.

Many email and messaging apps use encryption to protect messages from prying eyes.

Digital cryptography is an enormous field, but there are a few concepts regarding digital encryption that I wish to highlight that I think are important for everyone to understand.

The first is public-key encryption. Prior to the development of public-key encryption, both the sender and the receiver of an encrypted message had to have keys to both encode and decode the message.

Public key cryptography was proposed in 1976 by Whitfield Diffie and Martin Hellman, and a working system known as the RSA cryptosystem was developed in 1977 by three mathematicians: Ron Rivest, Adi Shamir, and Leonard Adleman Their initials make up the RSA.

Public key cryptography eliminates the need to share a copy of the key with all the parties involved in an encrypted communication. Each party can have a public key, which is known to the world, and a private key which is known only to themselves.

That means there doesn’t have to be any communication between the two parties before they communicate. You just have to find their public key which could be posted anywhere.

One of the things which makes a system like this work is what is known as trap door or one-way functions.

A one-way function is something which is very easy to calculate, but very difficult to calculate in reverse.

One example of this is multiplying two very large prime numbers together. Any computer can easily do this. However, figuring out what the two prime factors are from a given very large number is very very difficult to do.

The difficulty in decrypting something which was encoded digitally has to do with the size of the original key used. This is measured in bits, which is the number of 1’s or 0’s used in the key.

The larger the key, the harder it is to decrypt. An increase in the size of a key by just 1 bit will double the time it takes to decrypt it.

It is common today for keys to be 1,024 bits to 2,048 bits in size. Estimates are that it would take a normal computer about 14 billion years to crack a code encrypted with a 2,048-bit key, which is about the age of the universe.

That being said, computers are always getting faster, and you can just throw a lot of computers at a problem.

The amount of energy required to crack a 4096-bit key is estimated to be enough to boil away all the oceans and turn the Earth into a wasteland.

The other thing which is important for digital cryptography is what are called hash functions.

A hash function is a way to map any number onto a number that is a set size. For example, a common hash function is the SHA-256. This stands for Secure Hash Algorithm and it was developed by the US National Security Agency.

Anything you put into this algorithm will result in an output that is exactly 256 bits long, even if what you input is longer than 256 bits.

Moreover, the output is seemingly random. The difference in outputs between 1 and 2 isn’t 1 digit. It is a completely different number.

256 bit is a stupefyingly large number. It is 115 quattuorvigintillion, which is a number in base 10 that has 77 digits. If you had that many liters of water, you could fill up the galaxy.

So the hash function is a one-way function, but because the function is public, you can use it to verify results. If two people put in the same input, they will get the same output.

Hash functions are one of the foundations for blockchain transactions and for cryptocurrencies such as bitcoin, but again, I’ll leave that for another episode.

One other element of modern cryptography I should address isn’t cryptography per se. It is called steganography.

Steganography isn’t encrypting messages with a code or cipher, but rather the science of hiding messages.

Digitally, this usually means embedding text messages inside of other files without anyone knowing that there is a message inside. For example, using steganographic programs you can put a secret message inside of a jpeg file, or even inside of an mp3 file.

An mp3 file the likes of which are used in podcasts. Like this one. This podcast which you are listening to which is an mp3 file. There are websites like Stegonaut which can hide and uncover text messages inside of mp3 files.

Cryptography has gone from something simple which was used to discourage prying eyes two thousand years ago, to something which is the foundation for much of our modern economy. Today cryptography is the basis for almost all electronic commerce and digital communication, and the internet would be a very different place without it.

Everything Everywhere Daily is an Airwave Media Podcast.

The associate producers are Thor Thomsen and Peter Bennett.

As you know, I encourage everyone to leave reviews for the show on whatever podcast player you use.

Well, for the rest of the month of April, you can both leave a review, and do some good in the process.

Until the end of the month, if you leave a review for this podcast over at Podchaser.com they will donate 25-cents to World Central Kitchen, which will be using the money to provide food for Ukranian refugees. Moreover, everytime the podcaster, that’s me, responds to a review, which I will, they will double the donation to 50-cents!

Plus there are several podcasting company who will also be matching donations as well, on top of that.

Podchaser is a website that is sort of like IMDB for podcasts. It is a database of podcast as well as the people who work on them, as well as people who have been guests on shows.

Even if you have left me a review before on some other platform, you can still leave a review over on Podchaser. I will also be reading your reviews at the end of the shows just like I always do.

So this is a very easy way to help raise money for a good cause, and it doesn’t cost you a thing.

I have a link directly to my Podchase page at the top of the show notes for this episode.