AZdecrypt 1.24

[Provoni]

AZdecrypt 1.23 released: https://drive.google.com/file/d/1zenlhO ... share_link

New features:

- Reduced per thread memory usage by 3/4.

- Adapative sleep times for multi-threading. This should result in a small speed gain for most solvers and a few big gains here and there.

- Start of a Double columnar transposition solver.

[CdnCoder]

First, let me say thanks for developing this app!

I'm looking for direction on how to use certain features. Running on a current release of Win10.

Question 1: How to select options for solving
Example: load Ciphers\Transpositions\Kryptos3.txt

Select Simple Transposition (or Columnar Transposition) -> Solve
(dialogue box opens, but there is no "OK" button to lock in my selections; I can only close it with the X)

I cannot figure out how to launch something like Simple Transposition or Columnar Transposition.

Question 2: Syntax for crib lists
I have attempted to create my own crib lists, following the sample crib list file, but it doesn't seem to load. My crib list is sparse.

Example:
7 TRIAL
15 FLOUNDER

Does the whitespace between the line number and string need to be a TAB? A specific # of spaces?

Many thanks!

[Provoni]

Hey CdnCoder. Sorry for the late reply.

Select Simple Transposition (or Columnar Transposition) -> Solve
(dialogue box opens, but there is no "OK" button to lock in my selections; I can only close it with the X)

I cannot figure out how to launch something like Simple Transposition or Columnar Transposition.

Just click Solve again after the extra window has opened.

Does the whitespace between the line number and string need to be a TAB? A specific # of spaces?

The crib word needs to be on the 6th position. I think I did it that way because the crib could also contain spaces or tabs (per the letter n-gram alphabet).

Working on kryptos?

[CdnCoder]

Thanks for both your answers. I'm working on kryptos, and also trying to put some book theory to the test with actual examples. AZdecrypt makes a great environment to workshop my learning.

Cheers!

Hey CdnCoder. Sorry for the late reply.

Select Simple Transposition (or Columnar Transposition) -> Solve
(dialogue box opens, but there is no "OK" button to lock in my selections; I can only close it with the X)

I cannot figure out how to launch something like Simple Transposition or Columnar Transposition.

Just click Solve again after the extra window has opened.

Does the whitespace between the line number and string need to be a TAB? A specific # of spaces?

The crib word needs to be on the 6th position. I think I did it that way because the crib could also contain spaces or tabs (per the letter n-gram alphabet).

Working on kryptos?

[ArchOptic]

This is an amazing tool and I've had great luck with alot of ciphers from puzzles but I'm up against a real challenge and am trying to create my own ngram file to better match the language style of the current puzzle so I can optimize my searching and get more relevant outputs.

I am not a cryptography expert and don't have a good grasp on how to analyze the cipher I am working on to identify it's type but I'm attacking it as polysubtitution because I don't know what else to do. I actually thing it might be a chaocipher style cipher. It is a 570char base9 string.

I've generated a text based ngram file from a moderate size body of text. I have successfully loaded the ngram and generated the associated ini file. The problem I am having is that I do not know how to specify ENGLISH as the language! I get the following error:

- Unspecified language, word spacing will not be applied.
Please use the following n-gram file format:
n-grams_language_author_source_version.txt/bin/gz

Also, reference to or help with in depth resources about how to use this tool to attack an unknown base9 cipher would be considered a bonus!

Thank you.

[ArchOptic]

UPDATE:

I figured out how to specify the language in the filename.

I think my body of text is too small or the ngram is formatted incorrectly. I get the following type of output:

Score: 11169.89 IOC: 0.0570 Multiplicity: 0.0157 Seconds: 28.31
Repeats: LLES EYVE LESS UEV IYW WWI ULE NHL WYE YSM UWW MVW
PC-cycles: 50

Polyalphabetism: 37.36%

NY EV DW DEV N DL VAD HUY WY IVY NQ V V UNZ RL W SW
I IN VW BS RN IE Y WHV PPV IS HES I HM I DOW YES NG
A NXT FW IS NFL S MUA IE J FIY WL LE I W NG BBS W WIN
X I Y W KOV LESPN Q I MY G VIK W YEEL X H HUW WIE YVEL
HH BZU A A YE JTV NL WP KOE NY BY SRS GVY GI I WD XLS
W I SYN Q HP Y W FYL POS IS PTL WCW DEY VE WWF W I
WO QU W WDS SYS MONS N H W XY WOW KY GW MEN H LEES
WGNS LRS DOWN HH NLY W X N VE U EVN VI YS DVL HI YG
VU EVIE JD Q W A GY SMV WHN HL KSP MY LUS YHBZQS R
W Z V V WVU HVNVY WY LINZ OUT I HE I WE C BY IN OH
G IN J WOW Y I WOH J MUL IS DS VS Q HHN VE CMV WY GH
W X U WOZ IN I BT BS PM U EQ FE VNC VTX I YY LE HT
D BC W K LNG HOW I ES RL LES L LESS NJ XRF GS UCT RECE
FSN VW VILKS WWN VE WN FZNNXWWLY Z YEA YE I U U LEVLS
SL N RN V RVD WHP VWN I I SWF FW LESS Y HG ZX SULE
HR JYA I H HLS CDR LLS LVY LI HIT Q X R HH SSD SY

As you can see it is not generating complete words! Please help!

[ArchOptic]

Update 2:

I think I found a solution by creating an ngram bias file to run in conjunction with the Beijinghouse 8gram. I hope I am using it correctly.

This tool is so much more powerful than I know what to do with yet.

I am not sure if it is always better to run a higher ngram?

How do I get ngrams for other languages? I'm looking for Italian in particular!

[Provoni]

Hey ArchOptic,

Sorry for the late reply.

If you are using 1.23 go to AZdecrypt/N-grams/Languages/, there should be about all languages you need.

Usually higher n-grams are much more powerful. Here are Italian 6-grams for you to download: https://drive.google.com/file/d/1FP7XWC ... sp=sharing

You say it is a base 9 cipher. So there are only 9 unique symbols? In terms of classical cryptography the first thing I then think of is that for example each unique symbols could polyalphabetically substitute to 3 letters, or that groups of 2 characters may form one letter.

I work on ciphers by hypothesis testing:

1) Generate a hypothesis based on the stats of the cipher.
2) Create at least a few examples of your own and try to solve them and also compare stats.
3) Now try it on your cipher to solve.

Try to get someone to work with to bounce ideas back and forth.

Here is a cipher with 9 unique symbols where each symbol may map to up to 3 different letters:

Code: Select all

ADDAAEBCBAGBGHCBGCBFAFFDICECBE
FAGGGBBEGGCCFEFBEFACFAEGEFBEFA
EGCGCEGDBFFEHCBBGCBFBBBFBBGHCE
GBEAEBGAFFBAFCEGCBCFAFFDCAAGCE
EGHCGCAEFCBGEBAFFBGAFCABBFBBBF
BEABGCBBGADGEAHACDAADBBFEEGCBB
BFAFGEBEGADEBBCABEEGGFAAACGGGC
EFDICECEGEGDCFFBFFEGCGCEEHCBEG
FAHBDDCECGCBDBCGGGADBBGEBBFAEB
EEBBBGGCEHCECFEGBAGGGEEBHCDDGF
FAHDEGGGEACEEGBAAEHGCAGADDEHGA
GEEIGEDCBBEHEBHBECBCCGAAADDBCG
BEBGFCECECFEGEBAAEFEBFAGGCBBEF
BFGEEBFEBGCBGGFHCHADCEFFEFCBEF
BFBGCBBEGBHCDCEHEDHBGEGACEEFBG
CAEGCEFDIAFBBFCECAFEGEBAGFEEDI
CEGGBGDAICECBBBEFEBBEGGAGCEEGG
CAGHAEGGEBAGIEGCBIEGAFBAADBGEA
GGBEBAEGDBIEGFDBAGBAEEGAAGGGGE

So now first thing I would do is compare stats with your cipher to solve. We want to look for repetitions in the horizontal direction since the encoding did not randomize the order of the text. Go to Statistics, Find plaintext direction, Using trigrams. Scroll down in the output:

Code: Select all

Directional: (transposition, untransposition)
---------------------------------------------------------
Normal: 303, 303 (6.53, 6.53) <---
Mirror: 297, 297 (5.58, 5.58)
Flip: 297, 297 (5.58, 5.58)
Reverse: 303, 303 (6.53, 6.53) <---
---------------------------------------------------------
Columnar 1: 262, 272 (0.01, 1.60)
Columnar 2: 263, 268 (0.17, 0.96)
Columnar 3: 263, 268 (0.17, 0.96)
Columnar 4: 262, 272 (0.01, 1.60)
---------------------------------------------------------
Diagonal 1: 258, 265 (-0.62, 0.49)
Diagonal 2: 260, 259 (-0.30, -0.46)
Diagonal 3: 261, 265 (-0.14, 0.49)
Diagonal 4: 258, 262 (-0.62, 0.01)
Diagonal 5: 258, 262 (-0.62, 0.01)
Diagonal 6: 261, 265 (-0.14, 0.49)
Diagonal 7: 260, 259 (-0.30, -0.46)
Diagonal 8: 258, 265 (-0.62, 0.49)

We find a 6+ sigma result for trigrams at the Normal direction, which is horizontal from left-to-right. If the hypothesis match up you also want to find this in your cipher to solve.

Additionally we can go to Statistics, Unigrams.

Code: Select all

Index of coincidence:
- Raw: 46068
- Normalized: 0.1420405142909999
- Versus random: 1.278364628618999
[b]- Flatness: 0.771251193887297
Alternative flatness: 0.7386363636363638[/b]

Flatness for this hypothesis should be around 0.666 (Same as English or your other typical Latin alphabet using language). It's simply the flatness of the unigram frequency distribution by some formula. 0.77 is a tad bit high but acceptable.

Then to solve it, it's about picking the right solver for your hypothesis, which I can guess can be very daunting if you don't know the program well.

Take the solver "Substitution + polyphones" and click on solve. In the next window select User defined polyphones and "Set letters# for all symbols" to 3. The right-hand side list should update and letters should 3 for all symbols. Click on Solve again in the main window. I recommend at least 7-grams and an overnight run on your computer since it is very hard.

Another way to do it is if you have any assumptions, and since I made the hypothesis cipher myself I know that A maps to ABC, B to DEF, C to GHI, D to JKL, E to MNO, F to PQR, G to STU, H to VWX, I to YZ. Take the solver "Manual polyalphabetic" and fill in these assignments and click on Solve.

Now I was able to solve the hypothesis cipher at least Let me know how it goes

[Provoni]

AZdecrypt 1.24 released:

https://drive.google.com/file/d/1LhXKTc ... drive_link

New features:

- Better integration of the homophone weight, and this setting is now changeable in the Main window (as is the entropy weight) and is available to all solvers that list the amount of homophones in the Output window.
- Patterns and Contacts analysis under Stats.
- Substitution + verbose solver.
- Manual polyalphabetic solver.
- Find words solver (technically not a solver).
- More languages added, about 20 languages have spacing support now.
- Bugfixes.

Imported from Louie's release at https://github.com/louiehelm/azdecrypt/ ... =louiehelm

- Faster and integrated zlib.
- Jemalloc for improved and faster memory management.
- Beijinghouse v7 n-grams.
- Optimized fastpow1 function.
- Icon location fix.
- More imports to come.

[Provoni]

Example of the Substitution + verbose solver:

1) Example cipher, put it in the Input window:

Code: Select all

abcdefghijkldeamdenopmopqdefgh
qbcmrstopdefghbcijijubboklfghd
eubbofghvrwubboopafghbcdefghij
axyvnubbofghbcdezopdeubborfght
xysvop{ijubbonijubbo|fghbcop|d
efghopde|deijubbofghbcdefghkls
opqdeyklopvxxmijubbo|y}ijubbo{
txubbofghx{mopadeklfghbcopbcy|
opklfghfghtopopijbcdenopzoptkl
opopubboijmubboxfghfghdevwijub
bo|sij|ijmopdeubboijmmopubbokl
opqxvxklkldevijm}xklklijsvopfg
hbcopfghtyubbowdefghfghbcopsde
klopmyklfghbcdezopsopopubbofgh
bcijtfghrrdetnkldeqtxklklnijnr
xyopzoptklopopde}ijqfghytopx{f
ghbcxklopfghtopopklijubboqdevi
j{xtubboijdefghbcdefghbcdezopd
efghyubboubboopvqyfghfghbctxy|
bcfghbcopsdekloprxyqxyvnntijzo
pdeqdetfghbctxy|bcaop

2) Select the Substitution + verbose solver.
3) Change the Homophone weight to 1 (this is not a homophonic substitution cipher).
4) Click Solve and change the "Verbose symbols amount" in the new window to 10. (depends on the cipher!)
5) Click Solve again in the Main window, and wait for the solution to develop in the Output window:

Code: Select all

Restart: 1 Hill climber: 2871/5000 @ 500000
Score: 23139.79 IOC: 0.0648 Multiplicity: 0.0680 Seconds: 12.46
Repeats: THROUGH EVERSEE THEBASE THAVEA THAVE ATHA HATI THAT
PC-cycles: 338 Homophones: 0

Verbose units: ij,bo|,kl,deam,op,bc,ops,ubbo,de,fgh

WHAT IS JADE ME CATCH MY BREATHI INSTANTLY KNEW THAT 
I WOULDNT HAVE ANY TROUBLE FINDING THE GATE AGAIN THATS 
BECAUSE LOOMING UP IN FRONT OF ME WAS THE HUGEST TREE 
I HAD EVER SEEN IM NOT TALKING BIG I MEAN IMMENSE COLOSSAL 
IMPOSSIBLE THE TRUNK AT THE BASE MUST HAVE BEEN THIRTY 
YARDS ACROSS DID YOU EVER SEE A PICTURE OF THOSE TREES 
IN CALIFORNIA THAT HAVE A TUNNEL CUT THROUGH THE BASE 
YOU COULD DRIVE A CAR THROUGH WE

[Provoni]

Example of the Manual polyalphabetic solver:

You use this solver when you have a good idea or suspicion of how the polyalphabetism in the works. Or if you want to repair a message.

1) Example cipher, put it in the Input window:

Code: Select all

xfkkthbnkyovtonvd
hidhotfhfrfortfvf
rbkrfbtontbdtvbkk
yfirttoffikovfhfr
thfhbtbffnthrovhh
tonvdhbnditonfoft
hfttronhfttffnbkf
hfroinftikovfthff
bdtthbtthfhbtdbbb
kfdonbothtidftoft
hfkinfbvtinthffnd
thfbkxbythoftforx
hbtthfknoxtitrihh
tthfxikkbkxbytbfb
nbvfnhfribktodhot
fhfrbfdbvtfixbntf
dbdottvnfdhbkkfnh
fhfrdottvnfhbtnfv
frbffnxfkkvodbtfd
tothffxtfntthfdft

2) Select the Manual polyalphabetic solver.
3) Click on solve. In the new window put in the following table manually:

Code: Select all

b = AB
d = CD
f = EF
h = GH
i = IJ
k = KL
n = MN
o = OP
r = QR
t = ST
v = UV
x = WX
y = YZ

4) Click on Solve again in the Main window and wait for the solution to develop in the Output window:

Code: Select all

Score: 23951.54 IOC: 0.0718 Multiplicity: 0.0382 Seconds: 0.03
Repeats: CHOSEHER COSTUME ALWAYS HATTHE SOMUCH ILOVE OFTHE
PC-cycles: 340

WELL THANK YOU SO MUCH I CHOSE HER FOR SEVERAL REASONS 
ACTUALLY FIRST OFF I LOVE HER THE HAS BEEN THROUGH 
SO MUCH AND IS ONE OF THE STRONGEST FEMAKE HER PINES 
I LOVE THE FACT THAT THE HAS DABBLE COMBOTH SIDES OF 
THE LINE BUT IN THE END THE ALWAYS GOES FOR WHAT THE 
KNOWS IS RIGHT THE WILL ALWAYS BE AN AVENGER I ALSO 
CHOSE HER BECAUSE I WANTED A COSTUME CHALLENGE HER 
COSTUME HAS NEVER BEEN WELL UPDATED TO THE EXTENT THE 
DES

[Provoni]

Example of the Patterns analysis:

1) Put a cipher in the Input window, for example the Z340:

Code: Select all

HER>pl^VPk|1LTG2d
Np+B(#O%DWY.<*Kf)
By:cM+UZGW()L#zHJ
Spp7^l8*V3pO++RK2
_9M+ztjd|5FP+&4k/
p8R^FlO-*dCkF>2D(
#5+Kq%;2UcXGV.zL|
(G2Jfj#O+_NYz+@L9
d<M+b+ZR2FBcyA64K
-zlUV+^J+Op7<FBy-
U+R/5tE|DYBpbTMKO
2<clRJ|*5T4M.+&BF
z69Sy#+N|5FBc(;8R
lGFN^f524b.cV4t++
yBX1*:49CE>VUZ5-+
|c.3zBK(Op^.fMqG2
RcT+L16C<+FlWB|)L
++)WCzWcPOSHT/()p
|FkdW<7tB_YOB*-Cc
>MDHNpkSzZO8A|K;+

2) Go to Statistics and click on Patterns and see the results in the Output window.

Code: Select all

AZdecrypt patterns stats for: Zodiac 340.txt
---------------------------------------------------------
- Only n-grams that do not overlap are considered for this analysis.

N-gram: repeats (probability)

G2: 3 (4.092629983590227e-011)
|5F: 2 (2.302104365769503e-010)
FB: 3 (6.431275688498929e-010)
FBc: 2 (7.235185149561294e-010)
M+: 3 (1.725725643080546e-009)
UZ: 2 (1.828041392670302e-008)
)L: 2 (4.570103481675755e-008)
p7: 2 (5.02711382984333e-008)
8R: 2 (5.118515899476844e-008)
(#: 2 (6.398144874346056e-008)
(): 2 (6.398144874346056e-008)
++: 3 (6.557757443706075e-008)
+&: 2 (8.408990406283389e-008)
#O: 2 (1.371031044502726e-007)
Np: 2 (1.675704609947777e-007)
By: 2 (2.010845531937332e-007)
5F: 2 (2.879165193455725e-007)
Fl: 2 (2.879165193455726e-007)
|5: 2 (2.879165193455726e-007)
Op: 2 (7.540670744764995e-007)
Bc: 2 (9.048804893717993e-007)
+R: 2 (2.354517313759349e-006)
O+: 2 (3.784045682827525e-006)

You can see that the "G2" bigram is the most improbable repeat, followed by the trigram "|5F" etc.