THE ADFGX / ADFGVX CIPHERS ,GEORGES PAINVIN

1. * If the German efforts at codebreaking still remain obscure, it is clear the Germans were able to put together strong codes and ciphers that were difficult to crack. They were not, however, uncrackable, and the French were particularly competent at penetrating German codes and ciphers. Unlike the British, the French actually had formal codebreaking groups, five of them, in place before the war. After the outbreak of hostilities the French built up their "Bureau du Chiffre (Cipher Bureau)" in specific. The Bureau du Chiffre was one of the most professional Black Chambers ever organized. It grew out of the humiliating defeat of the French in the Franco-Prussian War in 1870, and their subsequent fear of a new united Germany. The French took their inspiration from the Dutch cryptanalyst Auguste Kerckhoffs, who wrote the text LA CRYPTOGRAPHIE MILITAIRE mentioned in a previous chapter. Kerckhoffs did most of his work in France, and the French were his enthusiastic students. The Bureau du Chiffre performed cryptanalysis on an industrial scale. A group of "stars" were put to work breaking new ciphers, while teams of experts with specialized training did the dayto-day "grunt work". One of the stars was Lieutenant Georges Painvin. The bright Painvin had not been interested in cryptanalysis before the war, but after the shooting began he ran into a member of the bureau, who recognized that Painvin had the "knack". The Germans threw their greatest cryptological challenge at the French in the spring of 1918. The Allies became aware that the Germans were preparing to launch an offensive, in hopes of victory before American reinforcements arrived in strength and presented the Kaiser's army with certain defeat. The tipoff was increased radio traffic and the introduction not merely of new trench codes, but of a cipher as well, which became known as the "ADFGX" cipher, which went into service on 5 March 1918. The ADFGX cipher had been invented by Colonel Fritz Nebel, a communications officer of the Kaiser's army, to provide an army on the move with encryption more convenient than trench codes but still secure. In fact, the Germans believed the ADFGX cipher was unbreakable. The ADFGX cipher was a fractionating transposition cipher and was in fact very difficult to solve. The cipher is constructed in two steps. The first step requires building a checkerboard with the rows and columns indexed by the letters ADFGX, and then randomly placing the letters of the alphabet in the grid:
A D F G X A D F G X f g y k o z l v c m t d p u x n s e w q i/j b a r h

The placement of the letters establishes a "substitution key" for the cipher. In this example, the substitution key leads to the cipher equivalents:
a b c d e FX DX GD DF FG f g h i/j k AA DA XX AX GA l m n o p DD XD AG XA FF q r s t u XG GX DG AF GF v w x y z FD GG XF FA AD

For example, the message:

urgent send more ammunition now

-- would first be enciphered into its letter pair ciphertexts as follows:

GFGXFGAGAF DGFGAGDF XDXAGXFG FXXDXDGFAGAXAFAXXAAG AGXAGG -> GFGXFGAGAFDGFGAGDFXDXAGXFGFXXDXDGFAGAXAFAXXAAGAGXAGG

The reason the letters ADFGX were used was because in Morse code they are very distinctive from each other, reducing the likelihood of transmission errors. At this stage of encryption, the ADFGX cipher is just a simple Polybius cipher, just another form of a monoalphabetic substitution cipher, which is of course very weak. However, the next step involves a transposition, using a transposition keyword, say "WARTHOG", and then writing the ciphertext produced by the first step in consecutive rows underneath it:
W A R T H O G _____________ G G A A X G X A F A G G D A A G G F D X X X A G X D F F D A G F G X G G F A G F D F F A G A G X X A X X

The columns of this transposition array are then shuffled so that the letters of the key are in alphabetical order:
A G H O R T W _____________ F A G G D A A G A G X X A X X F G X G G F A G F D F F A G G F D X X X A G X D F F D A G G G A A X G X A

Finally, the letters are written out in column order to produce the ciphertext:
FAGGDAAG AGXXAXX FGXGGFA GFDFFAG GFDXXXAG XDFFDAG GGAAXGXA -> FAGGDAAGAGXXAXXFGXGGFAGFDFFAGGFDXXXAGXDFFDAGGGAAXGXA

Long messages sent in the ADFGX cipher were broken into sets of messages of different and irregular lengths. The Germans had learned that fixed-sized blocks left their ciphertexts vulnerable to multiple anagramming, and knew that if they gave the French any fingerhold into a cipher it would be methodically broken. * Georges Painvin was given the job of cracking the ADFGX cipher. Of course Painvin had the powers of concentration characteristic of any good codebreaker, but the knowledge that the ciphertexts he was being handed had clues in a life-or-death game of national survival focused his concentration all the more. Since the cipher only used five letters, it was clearly a checkerboard scheme of some sort. The first step was to eliminate the obvious. He did a frequency analysis on the letter pairs to see if they were just a simple Polybius square substitution. The result showed a "noisy" distribution of the letter pairs, which meant that as he had expected the letter pairs had been split up and shuffled. Now he assumed that the cipher was a transposition by columns of a simple monoalphabetic checkerboard substitution. A grille transposition or a polyalphabetic substitution, using different checkerboards for different letters, would have been harder to crack but also harder to use, so Painvin decided to work using a relatively simple assumption before moving on to more complicated ones. * Figuring out the transposition scheme was very tricky. As the ciphertexts were of irregular length, the Germans were obviously not using a fixed size block, which made breaking the messages into its columns a puzzle. At first, traffic was light, making progress on the small handful of intercepts he received very difficult, all the more so because he had to assume the Germans were changing keys every day. On 21 March 1918, the German offensive began like a thunderclap, smashing great holes in Allied lines and throwing British and French troops back in confusion. The pressure on Painvin increased accordingly, though at least now he had an increasing adequate number of intercepts to work from. Changes in the letter frequency distributions from day to day told him the Germans were in fact changing the keys every day, though the fractionating nature of the cipher ensured that his analyses told him no more than that. By the beginning of April, he was receiving enough intercepts on a single day to give him some hope of obtaining a fingerhold. On 4 April, he finally noticed that some ciphertexts sent on the same day possessed certain patterns that gave a clue to the column breakdown of the messages.

* Suppose two or more messages were encrypted with the ADFGX cipher using the same keys, and that these messages began (or ended) with the same text, giving Painvin a crib. The encryption procedure ensured that when the letter pairs for these messages were written down in an array for transposition, they would have the same text at the top of the columns. The columns would be transposed in the same way for all these messages, with the common text transposed along with them. This meant that when the transposed columns of these messages were transmitted in sequence, they would all feature distinctive "clusters" of the same text at more or less regular intervals corresponding to the crib. Painvin could use these clusters to find clues on the column breakdown of the messages. Painvin now had a strong clue concerning the way the messages were broken down into columns, but he did not know the way in which the columns had been shuffled, or in other words he did not know the transposition key. The fact that the ADFGX cipher did not use a fixed-size transposition block actually helped, to an extent. Most of the time, the columns would be broken down into sets of two lengths, with one set a single letter shorter than the other. In the example for the ADFGX cipher above, for instance:
A G H O R T W _____________ F A G G D A A G A G X X A X X F G X G G F A G F D F F A G G F D X X X A G X D F F D A G G G A A X G X A

-- some columns are 8 letters long and some are 7 letters long. Now recall that before the columns were shuffled we had:
W A R T H O G _____________ G G A A X G X A F A G G D A A G G F D X X X A G X D F F D A G F G X G G F A G F D F F A G A G X X A X X

-- with all the 8 letter columns to the left. If the longer columns can be identified, that gives a clue to the column shuffling.

Among the messages Painvin received on 4 April were one that was 110 ciphertext letters long and another 104 ciphertext letters long. His use of cribs suggested they were both broken down into 20 columns. Given 20 columns, a message with 120 letters would have broken down into columns that were all six letters long, while a message with 100 letters would have broken down into columns that were all five letters long. The message lengths of 110 and 104 meant that the lengths of the columns were a mix of five and six letters. Of course, the number of five-letter columns had to be the same as the message length subtracted from 120. It's just simple logic: if there were 20 six-letter columns, the message would be 120 letters long; if 19 six-letter columns and one five-letter column, it would be 119 letters long; if 18 six-letter columns and two five-letter columns, it would be 118 letters long; and so on. The message 110 letters long had ten columns five letters long, and ten columns six letters long, while the message 104 letters long had 16 columns five letters long and four columns six letters long. Painvin could use the crib patterns to get an idea of how to break the message down into 5letter and 6-letter columns. He could then assume that the longer columns, with six letters, had been on the left side of the transposition array before they were shuffled. This gave Painvin a few plausible column matches. He performed a frequency analysis on the letter pairs that resulted from them. Some yielded a noisy result, suggesting a mismatch, but others yielded a frequency distribution approximating that of German text, indicating a successful match. This not only confirmed a valid match, but his assumption that the Germans were not using a polyalphabetic substitution. Given multiple column matches, he was able to glean a little more information on the ordering of the columns he had matched. For example, the most common pair in each analysis very likely represented "e". If the most common pair in one analysis was "DG" and the most common pair in another analysis was "GD", then obviously the column order of one of the pairs of columns was reversed from that of the other. He could switch one and then perform a single frequency analysis on the combination of the two pairs. It didn't really matter which pair he switched; the order of the letter pairs was unimportant, as long as he was consistent for all the letter pairs. * Painvin was able to come up with another subtle trick to help narrow down the possibilities for the transposition order of the columns. The ADFGX cipher's substitution was, as explained above, based on a grid, with the letters "ADFGX" along the side and the same letters along the top:
A D F G X A D F G X f g y k o z l v c m t d p u x n s e w q i/j b a r h

Painvin knew that each substitution was created with one letter taken from the left side of the grid, followed by another letter taken from the top of the grid. The reverse might have been true, but once again, as long as he was consistent, it didn't matter at all, and so we'll assume a "side:top" arrangement in this discussion. This meant that after substitution but before transposition, the "side" letters would be the oddnumbered letters and the "top" letters would be the even-numbered letters. Going back to the ADFGX example above, the initial conversion of plaintext into letter pairs gave:
GFGXFGAGAFDGFGAGDFXDXAGXFGFXXDXDGFAGAXAFAXXAAGAGXAGG ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ "side" letters GFGXFGAGAFDGFGAGDFXDXAGXFGFXXDXDGFAGAXAFAXXAAGAGXAGG ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ "top" letters

Now remember this would be written out in row order before being transposed by columns. If there were an even number of columns, then the columns would be composed entirely of "side" letters or "top" letters, and not a mix of the two. For example, if we take the list of letter pairs with the "side" letters marked above and chop it into eight columns, we get:
G ^ A ^ D ^ F ^ G ^ A ^ X ^ F G ^ F D ^ F X ^ G F ^ F A ^ X X ^ A G ^ X F ^ G F ^ D X ^ X X ^ G A ^ A A ^ G G A ^ G A ^ A G ^ D X ^ X A ^ G A ^ G G X D F G

One of the characteristics of frequency analysis of letters is that while the distributions of individual letters may vary widely from the norm, the law of averages dictates that groups of letters vary less. Now, with the ADFGX cipher, each "side" letter or "top" letter is associated with five plaintext letters. Going back once again to the example:
A D F G X A D F G X . g . . . z l v c m . d . . . . s . . . . b . . .

-- the "side" letter "D" is associated with the plaintext letters "g l d s b", while the "top" letter "D"

is associated with the plaintext letters "z l v c m". Since these two groups of five letters have different cumulative frequency distributions, then if there were an even number of columns -which Painvin could figure out using the crib trick described above -- a frequency analysis of the "D" letter in odd-numbered columns will have a distinctively different result from those of the "D" letter in even-numbered columns. This trick allowed Painvin to tentatively identify which columns were odd and which columns were even. He could then pair up odd and even columns and perform a frequency analysis on the pairings to see if they were noise, or real pairings that corresponded to plaintext letters. Once he had the proper pairings, he could then use frequency analysis to figure out the actual plaintext letters. The result was still transposed, but at that point all he had to do was unscramble a simple transposition. Once he determined the transposition scheme for one message, he would then be able to crack any other message enciphered with the same transposition key. * Painvin was finally able to decrypt some ADFGX ciphertexts on days when German radio traffic was heavy. By the end of May, given enough traffic, he could crack the messages in a day. The Germans were on the move again and proving very hard to stop. Hopefully, one of the decrypts would give warning of where they intended to strike next. Then, on 1 June 1918, the ciphertexts suddenly began to include the letter "V". The Germans had changed the cipher. Painvin had no idea if they had simply added the new letter to expand the existing system to a 6 by 6 grid, or if they had completely changed the scheme, wiping out all his hard work. He fell into despair for a moment, then went to work. The simplest assumption was that the new cipher was just a straightforward extension of the old. As he dug into the ciphertexts, he became increasingly aware that this assumption was correct, and quickly regained confidence. Adapting his work on the ADFGX cipher to the ADFGVX cipher was straightforward. He had a solution by the evening of 2 June. The additional "V" used by the ADFGVX cipher allowed encryption of the ten digits "0" through "9", with the following example grid:
A D F G V X A D F G V X 6 k s z 8 b g 4 l f h 5 e c p 7 m o f 1 2 x q y a 9 j w u v 3 n r t i d

The encryption scheme was otherwise unchanged, with plaintext letters converted into pairs, which were then transposed according to a keyword.

Painvin's decrypts provided Allied leadership with clues about enemy operations, though by this time the German offensive was running out of steam; stories that persist that the decrypts were a critical tool in stemming the tide seem to be exaggerations. In any case, as American reinforcements arrived the Germans found themselves increasingly on the defensive, and in November they would have to admit defeat. * It is a testimony to the good security of the ADFGX / ADFGVX ciphers that the Allies never developed a general solution to them during the war. The only way to crack them was through a lucky break, such as messages enciphered with the same keys that had the same beginnings or endings. In fact, only ten keys were ever discovered, though unsurprisingly these keys were on days of the heaviest traffic and allowed the cracking of a disproportionately large amount of ciphertext. After the war, Georges Painvin would become a prominent leader of the French business community, and granted high prestige and many honors. To the end of his days, however, he regarded the cracking of the ADFGVX cipher as his greatest distinction. He had lost ten kilograms during the exercise. Painvin's work in cracking the ADFGVX cipher was typical of the story of cryptography during the Great War. No matter how clever a cipher or code a codemaker came up with, the codebreakers were one step ahead. Clearly something entirely new needed to be done to make codes harder to break.