Decrypting Ciphers

The letter A is replaced with G and B is replaced with H and so on…

It looks like a whole lot of nonsense, right? Well, to many people, for many hundreds of years, it was nonsense – and the simple substitution cipher worked just fine to keep secret documents just that: secret. But eventually people started to realise these ciphers could be broken quite easily, using nothing but simple pattern recognition, frequency analysis and a general knowledge of grammar and vocabulary. Soon enough, simple-substitution ciphers were no longer enough to keep encoded documents safe from prying eyes, and those who wanted to keep their secrets secret moved on to more complex encryption methods.

While no one uses cryptograms (a type of simple-substitution cipher) today to encode their secret documents, they have survived to become a wildly popular puzzle that is available in thousands of books and newspapers… and of course websites!

Until modern times cryptography referred almost exclusively to encryption, which is the process of converting ordinary information (called plaintext) into unintelligible gibberish (called ciphertext). Decryption is the reverse, in other words, moving from the unintelligible ciphertext back to plaintext. A cipher (or cypher) is a pair of algorithms that create the encryption and the reversing decryption. The detailed operation of a cipher is controlled both by the algorithm and in each instance by a key. This is a secret parameter (ideally known only to the communicants) for a specific message exchange context. A “cryptosystem” is the ordered list of elements of finite possible plaintexts, finite possible ciphertext, finite possible keys, and the encryption and decryption algorithms which correspond to each key. Keys are important, as ciphers without variable keys can be trivially broken with only the knowledge of the cipher used and are therefore useless (or even counter-productive) for most purposes. Historically, ciphers were often used directly for encryption or decryption without additional procedures such as authentication or integrity checks.

Cryptograms are solved primarily by two methods; first by pattern recognition. The easiest to recognize are single-letter words, which generally can only be A or I (or, rarely, O). Then there are a limited number of two-letter words such as IN, IS, IT, TO, AN, AT, AS, WE, HE, US, etc. One trick in particular is to look for the common TH- words, i.e. THE, THAT, THEN, THEY, THERE, THEIR.

Secondly, the successful cryptogrammer will use letter frequency to help suss out a difficult puzzle. The twelve most frequently used letters in the English language are ETAOIN SHRDL, in that order. The least common letters are JXQZ. If you notice a certain letter being used again and again in any given cryptogram, at a frequency much higher than any other letter, it is a good bet that its unencrypted form will be one of the ETAOIN group. Finally, when all else fails, simple trial and error can generally fill in the gaps.

Cryptology prior to the modern age was almost synonymous with encryption, the conversion of information from a readable state to apparent nonsense. The sender retained the ability to decrypt the information and therefore avoid unwanted persons being able to read it. Since WWI and the advent of the computer, the methods used to carry out cryptology have become increasingly complex and its application more widespread.

Modern cryptography follows a strongly scientific approach, and designs cryptographic algorithms around computational hardness assumptions, making such algorithms hard to break by an adversary. Such systems are not unbreakable in theory, but it is infeasible to do so by any practical means. These schemes are therefore computationally secure. There exists information-theoretically secure schemes that provably cannot be broken, an example is the one-time pad, but these schemes are more difficult to implement than the theoretically breakable but computationally secure mechanisms.

The development of digital computers and electronics after WWII made it possible for much more complex ciphers. Furthermore, computers allowed for the encryption of any kind of data representable in any binary format, unlike classical ciphers which only encrypted written language texts; this was new and significant. Computer use has thus supplanted linguistic cryptography, both for cipher design and cryptanalysis. Many computer ciphers can be characterised by their operation on binary bit sequences (sometimes in groups or blocks), unlike classical and mechanical schemes, which generally manipulate traditional characters (i.e., letters and digits) directly. However, computers have also assisted cryptanalysis, which has compensated to some extent for increased cipher complexity. Nonetheless, good modern ciphers have stayed ahead of cryptanalysis; it is typically the case that use of a quality cipher is very efficient (i.e., fast and requiring few resources, such as memory or CPU capability), while breaking it requires an effort many orders of magnitude larger, and vastly larger than that required for any classical cipher, making cryptanalysis so inefficient and impractical as to be effectively impossible. Alternate methods of attack (bribery, burglary, threat, torture…) have become more attractive in consequence.

Essentially, prior to the early 20th century, cryptography was chiefly concerned with linguistic and lexicographic patterns. Since then the emphasis has shifted, and cryptography now makes extensive use of mathematics, including aspects of information theory, computational complexity, statistics, combinatorics, abstract algebra, number theory, and finite mathematics generally. Cryptography is also a branch of engineering, but an unusual one as it deals with active, intelligent, and malevolent opposition, other kinds of engineering (e.g., civil or chemical engineering) need to deal only with neutral natural forces. There is also active research examining the relationship between cryptographic problems and quantum physics.

Cryptography is a particularly interesting field because of the amount of work that is, by necessity, done in secret. The irony is that today secrecy is not the key to the goodness of a cryptographic algorithm. Regardless of the mathematical theory behind an algorithm, the best algorithms are those that are well-known and well-documented because they are also well-tested and well-studied! In fact, time is the only true test of good cryptography; any cryptographic scheme that stays in use year after year is most likely a good one. The strength of cryptography lies in the choice (and management) of the keys; longer keys will resist attack better than shorter keys.