{"id":856,"date":"2014-02-16T11:33:13","date_gmt":"2014-02-16T10:33:13","guid":{"rendered":"http:\/\/blog.rewolf.pl\/blog\/?p=856"},"modified":"2015-06-01T22:35:56","modified_gmt":"2015-06-01T20:35:56","slug":"solving-warsaws-java-crackme-3","status":"publish","type":"post","link":"http:\/\/blog.rewolf.pl\/blog\/?p=856","title":{"rendered":"Solving warsaw&#8217;s Java Crackme 3"},"content":{"rendered":"<p style=\"text-align: justify;\">Every once in a while I&#8217;m posting solution to some crackme that I consider interesting. By <em>interesting<\/em>, I mean the solution, so it is not exactly about key generation algorithm but also about technology and tricks that are utilized. Looking at the traffic statistics, it seems that this topic isn&#8217;t exactly the one that people would like to read (three posts &#8211; 5,63% of total unique page views), but I&#8217;m truly convinced that it has great potential for every single person that wants to learn something new. All in all, there is at least one person that benefits from those tutorials &#8211; ME :) Back to the topic, in this post I&#8217;ll describe <a href=\"http:\/\/crackmes.de\/users\/warsaw\/java_crackme_3\/\" title=\"warsaw's Java Crackme 3\" target=\"_blank\">warsaw&#8217;s Java Crackme 3<\/a>. Crackme was published on <em>14th October 2012<\/em> on <a href=\"http:\/\/crackmes.de\" title=\"crackmes.de\" target=\"_blank\">crackmes.de<\/a>, I&#8217;ve picked it up around February 2013, so literally speaking, it took me one year to solve it (of course I had some huge breaks meanwhile). Difficulty of the crackme was set to 5 (<em>Professional problem to solve<\/em>) in the crackmes.de scale and I must fully agree with it. It is <strong>Java <\/strong>crackme, but it wasn&#8217;t written in <strong>Java<\/strong>, I&#8217;m 99% sure that it was written in <strong>Jasmin <\/strong>or other assembler for <strong>Java Virtual Machine<\/strong> (<strong>JVM<\/strong>). Hand-crafted assembler and bunch of obfuscation tricks renders all existing decompilers pretty much useless, so it will not be yet another simple <strong>Java <\/strong>analysis.<\/p>\n<p><!--more--><\/p>\n<h3>Table of Contents<\/h3>\n<p style=\"text-align: justify;\">\n<ol style=\"text-align: justify;\">\n<li><a href=\"#firstblood\">First Blood (initial analysis)<\/a><\/li>\n<li><a href=\"#firstblood2\">First Blood Part II (second attempt)<\/a><\/li>\n<li><a href=\"#obfuscation\">Obfuscation tricks<\/a>\n<dl style=\"text-align: justify;\">\n<dd><a href=\"#exceptions\">Exception records<\/a><\/dd>\n<dd><a href=\"#jsr\">JSR instruction<\/a><\/dd>\n<dd><a href=\"#goto\">GOTO instruction<\/a><\/dd>\n<dd><a href=\"#antidisasm\">Anti-disasm trick<\/a><\/dd>\n<dd><a href=\"#modifications\">Modifications applied before debug session<\/a><\/dd>\n<\/dl>\n<\/li>\n<li><a href=\"#alg_analysis\">Algorithm analysis<\/a><\/li>\n<li><a href=\"#alg_reversing\">Algorithm reversing<\/a><\/li>\n<li><a href=\"#finalwords\">Final words<\/a><\/li>\n<\/ol>\n<h3><a id=\"firstblood\">First Blood (initial analysis)<\/a><\/h3>\n<p style=\"text-align: justify;\">Crackme is shipped as a <strong>JAR <\/strong>file, there is one .class file inside called <em>CON.class<\/em>. I&#8217;m not sure how it looks on UNIX but on windows <strong>CON <\/strong>is forbidden name and <em>CON.class<\/em> file cannot be easily created on disk, so to unpack this file, one need to change file name inside <strong>JAR <\/strong>archive (or use <strong>7zip<\/strong>, it changes it to <em>_CON.class<\/em> automatically). Such file with changed name will not run, as <strong>JVM <\/strong>treats file name as a class name. To get it working without jar file, class name inside a .class file needs to be changed to <strong>_CON<\/strong>, but I&#8217;ll get back to this topic later. <strong>JAR <\/strong>file with crackme is very small: <em>2620 bytes<\/em>, it doesn&#8217;t have any fancy GUI, just command line interface:<\/p>\n<pre lang=\"dos\">x:\\xxx>java -jar crackme3.jar\r\nPlease enter a key on the command line.\r\n\r\nx:\\xxx>java -jar crackme3.jar test\r\nIncorrect!\r\n<\/pre>\n<p style=\"text-align: justify;\">As a first step, I&#8217;ve tried to use java decompiler, but results generated by <strong>JD-Gui<\/strong> and <strong>JAD <\/strong>are not even worth mentioning. My second try was <strong>dirtyJOE<\/strong>, as it is really nice tool to quickly check what is really inside a .class file. There is only one method defined: <strong>main<\/strong>, it has <strong>22 exception records<\/strong> (it doesn&#8217;t look good, trust me) and the code is pretty much mind blowing. There are a lot of <strong>monitorenter<\/strong>\/<strong>monitorexit <\/strong>opcodes, some random <strong>goto<\/strong>, few <strong>jsr <\/strong>(<em>jump subroutine<\/em>) and even one <strong>athrow<\/strong>. Level of complication is nicely visible on the screen-shot from <strong>IDA <\/strong>graph view:<\/p>\n<p><a href=\"http:\/\/blog.rewolf.pl\/blog\/wp-content\/uploads\/2014\/02\/crackme3_graph.png\"><img decoding=\"async\" loading=\"lazy\" src=\"http:\/\/blog.rewolf.pl\/blog\/wp-content\/uploads\/2014\/02\/crackme3_graph.png\" alt=\"crackme3_graph\" width=\"451\" height=\"502\" class=\"aligncenter size-full wp-image-882\" srcset=\"http:\/\/blog.rewolf.pl\/blog\/wp-content\/uploads\/2014\/02\/crackme3_graph.png 451w, http:\/\/blog.rewolf.pl\/blog\/wp-content\/uploads\/2014\/02\/crackme3_graph-269x300.png 269w\" sizes=\"(max-width: 451px) 100vw, 451px\" \/><\/a><\/p>\n<p style=\"text-align: justify;\">As you can see, there are multiple entry points to the <strong>main<\/strong> method, there are even separate group of blocks not connected with the main execution path. I had an idea to manually rebuild proper code flow and recompile it so maybe some decompiler would work with this code, but I&#8217;ve realized rather quickly that it doesn&#8217;t make any sense, as I&#8217;ll not be able to properly rebuild all those peculiar links between so many exception handlers. That was the end of my first attempt at this crackme. What I knew, is that I need <strong>java bytecode debugger<\/strong> and it doesn&#8217;t exist&#8230;<\/p>\n<h3><a id=\"firstblood2\">First Blood Part II<\/a><\/h3>\n<p style=\"text-align: justify;\">Second attempt was taken few months later, after researching java bytecode debugging stuff. You can read about it in my previous post: <a href=\"http:\/\/blog.rewolf.pl\/blog\/?p=786\" title=\"Java bytecode debugging\" target=\"_blank\">Java bytecode debugging<\/a>. As I&#8217;ve mentioned in that post, there was still something missing to fully experience <em>java bytecode debugging<\/em> in a fashion similar to <em>x86 debugging<\/em>:<\/p>\n<blockquote><p>(&#8230;) Java VM is a stack based virtual machine, which means that most of all opcodes operates on the operand stack. Having information about values pushed onto the stack sometimes can be crucial to understand what is really going on. Unfortunately neither JDB nor JSwat supports previewing of jvm operand stack. This is an unresolved problem for now.<\/p><\/blockquote>\n<p style=\"text-align: justify;\">Well, recently I&#8217;ve resolved this problem and it enabled me to properly debug this crackme. I&#8217;ve wrote small tool called <strong>JVM Operand Stack Viewer<\/strong> (very original name!) &#8211; currently it&#8217;s private, because it wasn&#8217;t extensively tested &#8211; but I&#8217;ll be merging it with <strong>dirtyJOE <\/strong>very soon (or not), so stay tuned. Below you can find screen-shot of my java debugging environment, it is basically:<\/p>\n<ul>\n<li><strong><a href=\"https:\/\/github.com\/nlfiedler\/jswat\" title=\"JSwat\">JSwat<\/a><\/strong> debugger with .class file augmented by <strong>Restore Debug Info<\/strong> feature of <strong>dirtyJOE<\/strong><\/li>\n<li><strong>dirtyJOE Opcodes Help<\/strong><\/li>\n<li><strong>JVM Operand Stack Viewer<\/strong><\/li>\n<\/ul>\n<p><a href=\"http:\/\/blog.rewolf.pl\/blog\/wp-content\/uploads\/2014\/02\/java_debug.png\" target=\"_blank\"><img decoding=\"async\" loading=\"lazy\" src=\"http:\/\/blog.rewolf.pl\/blog\/wp-content\/uploads\/2014\/02\/java_debug-1024x598.png\" alt=\"java_debug\" width=\"586\" height=\"342\" class=\"aligncenter size-large wp-image-899\" srcset=\"http:\/\/blog.rewolf.pl\/blog\/wp-content\/uploads\/2014\/02\/java_debug-1024x598.png 1024w, http:\/\/blog.rewolf.pl\/blog\/wp-content\/uploads\/2014\/02\/java_debug-300x175.png 300w, http:\/\/blog.rewolf.pl\/blog\/wp-content\/uploads\/2014\/02\/java_debug.png 1848w\" sizes=\"(max-width: 586px) 100vw, 586px\" \/><\/a><\/p>\n<p style=\"text-align: justify;\">Armed with all those tools I could finally analyse algorithm responsible for serial checking.<\/p>\n<h3><a id=\"obfuscation\">Obfuscation tricks<\/a><\/h3>\n<p style=\"text-align: justify;\">In this paragraph I&#8217;ll describe some obfuscation tricks used by the crackme. I&#8217;ll also write about modifications that I&#8217;ve applied to the .class file before I&#8217;ve started debugging session. Let&#8217;s start with obfuscation:<\/p>\n<p><a id=\"exceptions\"><big><b>Exception records<\/b><\/big><\/a><\/p>\n<p style=\"text-align: justify;\">Exceptions in this crackme are used as a standard method to change execution flow, behaviour is pretty much similar to <strong>SEH <\/strong>based execution path modifications on <strong>x86 Windows<\/strong>, or maybe rather <strong>x64 <\/strong>exceptions, as <strong>JVM <\/strong>uses <em>exception table<\/em> similar to <strong>x64 Windows<\/strong>:<\/p>\n<pre lang=\"dos\">Start | End  | Handler | Type\r\n--------------------------------------------------------------------------------\r\n0002  | 0007 | 022F    | CONSTANT_Class : java\/lang\/IllegalMonitorStateException\r\n0000  | 0007 | 00A4    | any (finally statement)\r\n0007  | 000B | 00EB    | CONSTANT_Class : java\/lang\/IllegalMonitorStateException\r\n0007  | 0015 | 00DE    | any (finally statement)\r\n0017  | 001C | 0244    | any (finally statement)\r\n001C  | 0034 | 029C    | any (finally statement)\r\n0059  | 0062 | 0002    | any (finally statement)\r\n0062  | 007A | 0014    | any (finally statement)\r\n007A  | 007F | 022F    | any (finally statement)\r\n...\r\n<\/pre>\n<p style=\"text-align: justify;\">Exceptions in this crackme are triggered by various types of instructions:<\/p>\n<ul style=\"text-align: justify\">\n<li><strong>athrow <\/strong>&#8211; the simplest one, just throws requested exception, no matter what<\/li>\n<li><strong>monitorexit <\/strong>&#8211; this one is probably the most interesting, as it throws exception only if there wasn&#8217;t any <strong>monitorenter <\/strong>for the requested object. If there were multiple <strong>monitorenter<\/strong>s, then <strong>monitorexit <\/strong>can be called multiple times. Basically <strong>monitorenter <\/strong>increments reference counter, and <strong>monitorexit <\/strong>decrements it, if reference counter goes below 0 it will throw exception.<\/li>\n<li><strong>caload <\/strong>&#8211; load char from array, in case of null array reference, or index out of bounds it throws exception<\/li>\n<li><strong>idiv <\/strong>&#8211; throws exception in case of division by 0<\/li>\n<li><strong>checkcast <\/strong>&#8211; check whether object is of given type, quote from documentation: <em>&#8220;If objectref is null or can be cast to the resolved class, array, or interface type, the operand stack is unchanged; otherwise, the checkcast instruction throws a ClassCastException.&#8221;<\/em><\/li>\n<\/ul>\n<p style=\"text-align: justify;\">I&#8217;m not sure if those are all opcodes used to change execution flow, I think that I might have missed <strong>castore<\/strong>, but it is identical to <strong>caload <\/strong>(just store, not load) and those mentioned here are the most important anyway.<\/p>\n<p><a id=\"jsr\"><big><b>JSR instruction<\/b><\/big><\/a><\/p>\n<p style=\"text-align: justify;\"><strong>JSR <\/strong>(<em>jump subroutine<\/em>) instruction is very similar to <strong>x86 call<\/strong>, it can be used with corresponding <strong>RET <\/strong>opcode to implement some <em>finally<\/em> mechanism of <strong>Java language<\/strong>:<\/p>\n<blockquote><p>The jsr instruction is used with the ret instruction in the implementation of the finally clauses of the Java programming language (see Section 7.13, &#8220;Compiling finally&#8221;). Note that jsr pushes the address onto the operand stack and ret gets it out of a local variable. This asymmetry is intentional.<\/p><\/blockquote>\n<p>In this crackme, <strong>JSR <\/strong>is used to achieve two different goals. First, it is used as an obfuscation:<\/p>\n<pre lang=\"dos\">...\r\n0000007F: \tpop2\r\n...\r\n00000126: \tjsr                 pos.0000007F    ;never return from this jsr\r\n<\/pre>\n<p style=\"text-align: justify;\">Above code is equivalent of <strong>goto<\/strong>, it is also very similar to <strong>x86<\/strong> <em>call\/pop<\/em> or <em>call\/add esp<\/em>. Second usage is <em>more standard<\/em>, because it serves as a call to local functions defined in <strong>main<\/strong> method.<\/p>\n<p><a id=\"goto\"><big><b>GOTO instruction<\/b><\/big><\/a><\/p>\n<p style=\"text-align: justify;\"><strong>GOTO <\/strong>opcode is used as a simple flow obfuscation method, this technique is identical to <strong>jmp<\/strong>-based flow obfuscation used on <strong>x86 <\/strong>platform.<\/p>\n<p><a id=\"antidisasm\"><big><b>Anti-disasm trick<\/b><\/big><\/a><\/p>\n<p style=\"text-align: justify;\">There is quite nice <strong>anti-disasm<\/strong> trick, although I don&#8217;t know which disassembler is a target ;). In a <strong>constant pool<\/strong> of a class file, there is a definition of a long string which contains output of some <em>java disassembler<\/em>. This string is used as a name for some variables, it is also used as an input string in part of the serial check routine. This long string messes up output of decompilers and disassemblers and might confuse some inexperienced reversers.<\/p>\n<p><a id=\"modifications\"><big><b>Modifications applied before debug session<\/b><\/big><\/a><\/p>\n<p style=\"text-align: justify;\">I&#8217;ve done only few small patches for easier debugging:<\/p>\n<ul style=\"text-align: justify;\">\n<li><strong>dirtyJOE <\/strong>-> <strong>Restore Debug Info<\/strong> &#8211; this is inevitable to do <em>bytecode level debugging<\/em>, but it is for free since it is already implemented in <strong>dirtyJOE<\/strong><\/li>\n<li>I&#8217;ve mentioned earlier that there is some long string used as a name for some variables. It doesn&#8217;t look good on disassembly and it doesn&#8217;t help that three different variables has the same name. Those variables differs by type, so I&#8217;ve decided to call them <strong>var_int<\/strong>, <strong>var_char<\/strong> and <strong>var_short<\/strong>. This change can be easily done on <strong>dirtyJOE<\/strong> <em>Constant Pool tab<\/em>, procedure is very simple:\n<ol style=\"text-align: justify;\">\n<li>Add <strong>CONSTANT_Utf8<\/strong> item with the new name for variable, for example <em>&#8220;var_short&#8221;<\/em><\/li>\n<li>Usually there will be also <strong>CONSTANT_NameAndType<\/strong> entry corresponding to each variable from the <em>Fields tab<\/em>, to find the proper one, one need to search for the entry with proper type, for example <em>&#8220;S&#8221;<\/em> (<strong>short<\/strong>)<\/li>\n<li>Edit <strong>Name <\/strong>part in found <strong>CONSTANT_NameAndType<\/strong> entry, just choose new name from the drop down list (<em>&#8220;var_short&#8221;<\/em> should be there)<\/li>\n<li>On the <em>Fields tab<\/em>, set <strong>Name <\/strong>entry to the new name<\/li>\n<\/ol>\n<\/li>\n<li>Last change is related to the class name, as I didn&#8217;t want to constantly fight with this <strong>CON <\/strong>problems. I&#8217;ve changed file name to <em>_CON.class<\/em>, so I had to change class name inside .class file to the <strong>_CON<\/strong> as well. It was very tempting to just change <strong>CONSTANT_Utf8<\/strong> <em>&#8220;CON&#8221;<\/em> entry on constant pool to <em>&#8220;_CON&#8221;<\/em>, but there is small got-cha, <em>&#8220;CON&#8221;<\/em> entry is also used as a string in serial checking routine (situation similar to this long string described earlier), so the proper way was, to add separate <strong>CONSTANT_Utf8<\/strong> entry with <em>&#8220;_CON&#8221;<\/em> name, and set this item as a class name in <strong>CONSTANT_Class<\/strong> entry that currently reference <em>&#8220;CON&#8221;<\/em> string. <\/li>\n<\/ul>\n<p style=\"text-align: justify;\">At this point tools are ready, crackme is ready, all preparations are finished, so it is time to start the analysis of the algorithm.<\/p>\n<h3><a id=\"alg_analysis\">Algorithm analysis<\/a><\/h3>\n<p style=\"text-align: justify;\">At the beginning, crackme pre-calculates some constant char array that is used later. Calculations takes place inside subroutine at address <strong>0x1E1<\/strong> (<em>addresses in Java are relative to the method start<\/em>), so I&#8217;ll be referring to this function as <strong>sr_01E1()<\/strong>. It takes four arguments of <em>char array<\/em> type. This function is also used once again during serial check and I&#8217;ll describe its internals at that point. For now I&#8217;ll just list the arguments of the current call, and generated output:<\/p>\n<pre lang=\"java\">arg1 = \"CON\";\r\narg2 = \"'\\r\\n   7:\\tastore_0\\r\\n   8:\\ticonst_0\\r\\n   9:\\taaload\\r\\n   10:\\tin\";\/\/**\r\narg3 = arg1;\r\narg4 = arg2;\r\n--\r\noutput_c1 = \r\n{ \r\n    0x00D9, 0x00B9, 0x00B0, 0x0040, 0x0040, 0x0040, 0x006E, 0x0074, 0x0012, 0x00C2,\r\n    0x00E6, 0x00E8, 0x00DE, 0x00E4, 0x00CA, 0x00BE, 0x0060, 0x001A, 0x0014, 0x0040,\r\n    0x0040, 0x0040, 0x0070, 0x0074, 0x0012, 0x00D2, 0x00C6, 0x00DE, 0x00DC, 0x00E6,\r\n    0x00E8, 0x00BE, 0x0060, 0x001A, 0x0014, 0x0040, 0x0040, 0x0040, 0x0072, 0x0074,\r\n}; \/\/ **\r\n<\/pre>\n<p style=\"text-align: justify;\">** &#8211; <strong>arg2<\/strong> and <strong>output_c1<\/strong> are cropped to increase readability, they&#8217;re also converted to char array (<strong>toCharArray()<\/strong>)<br \/>\nJust a quick reminder, <strong>char <\/strong>in Java is <strong>16bit<\/strong>, so do not confuse it with <strong>C char<\/strong>. Serial number taken from the command line is converted to char array, and preprocessed to another char array by below code:<\/p>\n<pre lang=\"dos\">000000C3: \tbipush              127\r\n000000C5: \tnewarray            atype:char\r\n000000C7: \tastore_0\r\n000000C8: \ticonst_0\r\n000000C9: \tistore_2\r\n000000CA: \ticonst_0\r\n000000CB: \tistore_3\r\n000000CC: \tgoto                pos.0000024A\r\n...\r\n0000024A: \taload_0\r\n0000024B: \tiload_2\r\n0000024C: \ticonst_4\r\n0000024D: \tidiv\r\n0000024E: \tiload_3\r\n0000024F: \tgetstatic           short _CON.var_short\r\n00000252: \tishl\r\n00000253: \taload               local.04\r\n00000255: \tiload_2\r\n00000256: \tcaload\r\n00000257: \tbipush              36\r\n00000259: \tinvokestatic        int java.lang.Character.digit(char, int)\r\n0000025C: \tixor\r\n0000025D: \tdup\r\n0000025E: \tdup\r\n0000025F: \tgetstatic           int _CON.var_int\r\n00000262: \tiadd\r\n00000263: \tputstatic           int _CON.var_int\r\n00000266: \tistore_3\r\n00000267: \tcastore\r\n00000268: \tiinc                local.02, 1\r\n0000026B: \tgoto                pos.0000024A<\/pre>\n<p style=\"text-align: justify;\">Translating it to Java is rather simple task:<\/p>\n<pre lang=\"java\">    var_short = -8444; \/\/0xFFFFDF04\r\n    index = 0;\r\n    val = 0;\r\n    post_serial array = new char[127];\r\n    while(true)\r\n    {\r\n        val = (val << var_short) ^ Character.digit(serial_array[index], 36);\r\n        var_int += val;\r\n        post_serial_array[index \/ 4] = (char)val;\r\n        index++;\r\n    }\r\n<\/pre>\n<p style=\"text-align: justify;\"><strong>Integer shift<\/strong> operations in Java are cropped to <strong>5 bits<\/strong> (<strong>Long shifts<\/strong> are cropped to <strong>6 bits<\/strong>), which means that shift by <strong>-8444<\/strong> is the same as shift by <strong>4<\/strong>. Above code translates <strong>every 4 characters from serial number to 16bit value<\/strong>. It is using <strong>base36<\/strong> on every single character, so it can be reduced to <strong>base16<\/strong>, as every <strong>16bit <\/strong>value can be represented by <strong>4 base16<\/strong> characters. So, in theory serial number is alphanumeric, but in practice it can be treated as a hexadecimal string. I haven't analysed this encoding further, but it is of course possible to generate proper serial number with all alphanumeric characters. <strong>post_serial_array[]<\/strong> has maximum size set to <strong>127<\/strong>, which means that serial number can by as long as <strong>4*127=508 characters<\/strong>. You may also noticed that there is <strong>var_int<\/strong> variable touched by this loop, this variable is modified by various parts of the crackme, and it's even used as an input for some calculations, but the truth is that it doesn't interfere with serial checking, so the only purpose of this variable is an obfuscation (<strong>var_char<\/strong> has the same purpose but it didn't appeared yet). Below you can see small example of code that is fooling around with <strong>var_int<\/strong>:<\/p>\n<pre lang=\"dos\">000000FA: \tgetstatic           int _CON.var_int    ;place after serial preprocessing\r\n000000FD: \tbipush              -70\r\n000000FF: \tsipush              662\r\n00000102: \timul                ; -70 * 662 = -46340\r\n00000103: \tirem                ; Y = var_int - (var_int \/ (-70 * 662))*(-70 * 662)\r\n00000104: \tdup\r\n00000105: \tdup\r\n00000106: \timul                ; Y*Y\r\n00000107: \ticonst_1\r\n00000108: \tiadd                ; Y*Y + 1\r\n00000109: \tiadd                ; Y*Y + 1 + Y\r\n0000010A: \tdup\r\n0000010B: \tdup2\r\n0000010C: \tputstatic           int _CON.var_int\r\n0000010F: \tgoto                pos.0000000B\r\n<\/pre>\n<p style=\"text-align: justify;\">After serial preprocessing, there is a loop that is using data from <strong>post_serial_array[]<\/strong> and from <strong>output_c1[]<\/strong>:<\/p>\n<pre lang=\"java\">    int c1_idx = 0;\r\n    int sr_idx = 0;\r\n    int[][] local4 = new int[2][53]; \r\n\r\n    do\r\n    {\r\n        local4[(sr_idx + c1_idx) & 1][(sr_idx + c1_idx) \/ 2] += (post_serial_array[sr_idx] * output_c1[c1_idx]);\r\n        \/\/ there is small chunk of code that is missing here but it's not\r\n        \/\/ used by the crackme for the given output_c1 and any serial number\r\n        c1_idx = (var_int + c1_idx) % 89;\r\n        sr_idx = (var_char + sr_idx) % -17;\r\n    }\r\n    while (0 != (sr_idx | c1_idx));\r\n<\/pre>\n<p style=\"text-align: justify;\">It looks complicated as hell, but in fact it is easy. Nevertheless, it took me a while to figure what it really does. Loop is always executed <strong>1513 <\/strong>times, <strong>89 * 17 = 1513<\/strong>, so it seems that it can be represented as a two nested for loops:<\/p>\n<pre lang=\"java\">\r\n    for (int c1_idx = 0; c1_idx < 89; c1_idx++)\r\n        for (int sr_idx = 0; sr_idx < 17; sr_idx++)\r\n            { ... }\r\n<\/pre>\n<p style=\"text-align: justify;\">It tells us, that only <strong>17<\/strong> 16bit values from <strong>post_serial_array[]<\/strong> are used and <strong>89<\/strong> values from <strong>output_c1[]<\/strong> array. To prove that those loops are the same I need to explain those two modulo operations that are used to advance <strong>c1_idx<\/strong> and <strong>sr_idx<\/strong> indexes. <strong>89 <\/strong>and <strong>17 <\/strong>are <strong>prime numbers<\/strong>, <strong>var_int<\/strong> and <strong>var_char<\/strong> must be <strong>coprime <\/strong>to them accordingly (which means, that <strong>var_int != N*89<\/strong> and <strong>var_char != M*17<\/strong>), if this condition is met <strong>var_int % 89<\/strong> (<strong>var_char % 17<\/strong>) is <strong>the generator of additive group modulo 89 (17)<\/strong>. I haven't checked if it is possible to input serial that will generate <strong>var_int<\/strong> (<strong>var_char<\/strong>) that isn't <strong>coprime <\/strong>to <strong>89 <\/strong>(<strong>17<\/strong>), but such analysis is not really required to solve the crackme (I suspect that calculations done over <strong>var_int<\/strong> and <strong>var_char<\/strong> eliminates this case anyway). One thing left from that loop is two dimensional <strong>local4[][]<\/strong> array, its content after execution is as fallow:<\/p>\n<pre lang=\"java\">\/\/ to simplify output: \r\n\/\/ - output_c1[] will be just c[]\r\n\/\/ - post_serial_array[] will be s[]\r\nlocal4[0][0] = s[0]*c[0];\r\nlocal4[1][0] = s[0]*c[1] + s[1]*c[0];\r\nlocal4[0][1] = s[0]*c[2] + s[1]*c[1] + s[2]*c[0];\r\nlocal4[1][1] = s[0]*c[3] + s[1]*c[2] + s[2]*c[1] + s[3]*c[0];\r\nlocal4[0][2] = s[0]*c[4] + s[1]*c[3] + s[2]*c[2] + s[3]*c[1] + s[4]*c[0];\r\nlocal4[1][2] = s[0]*c[5] + s[1]*c[4] + s[2]*c[3] + s[3]*c[2] + s[4]*c[1] + s[5]*c[0];\r\nlocal4[0][3] = s[0]*c[6] + s[1]*c[5] + s[2]*c[4] + s[3]*c[3] + s[4]*c[2] + s[5]*c[1] + s[6]*c[0];\r\nlocal4[1][3] = s[0]*c[7] + s[1]*c[6] + s[2]*c[5] + s[3]*c[4] + s[4]*c[3] + s[5]*c[2] + s[6]*c[1] + s[7]*c[0];\r\n\/\/ ...\r\n<\/pre>\n<p style=\"text-align: justify;\">Full table available here: <a href=\"http:\/\/blog.rewolf.pl\/blog\/wp-content\/uploads\/2014\/02\/multiply.txt\" target=\"_blank\">local4_filling<\/a>. For now I'll leave this array as it is, I'll explain those calculations later. In the next step, <strong>local4[0]<\/strong> and <strong>local4[1]<\/strong> arrays are converted into char arrays by below subroutine:<\/p>\n<pre lang=\"java\">char[] sr_0235(int[] t)\r\n{\r\n    char[] ret = new char[2*t.length];\r\n\t\t\r\n    for (int i = 0; i < t.length; i++)\r\n    {\r\n        ret[2*i + 0] = (char)(t[i] &#038; 0xFFFF);\r\n        ret[2*i + 1] = (char)(t[i] >> 16);\r\n    }\t\t\r\n\t\t\r\n    return ret;\r\n}\r\n<\/pre>\n<p style=\"text-align: justify;\">It just splits every integer value into two char values, nothing complicated here. Let's call those new arrays <strong>t1 <\/strong>and <strong>t2<\/strong>:<\/p>\n<pre lang=\"java\">    char[] t1 = sr_0235(local4[0]);\r\n    char[] t2 = sr_0235(local4[1]);\r\n<\/pre>\n<p style=\"text-align: justify;\"><strong>t2 <\/strong>array is concatenated with <em>\"I\"<\/em> char:<\/p>\n<pre lang=\"dos\">0000028D: \tinvokestatic        java.lang.String java.lang.String.valueOf(char[])\r\n00000290: \tldc                 \"I\"\r\n00000292: \tswap\r\n00000293: \tinvokevirtual       java.lang.String java.lang.String.concat(java.lang.String)\r\n00000296: \tinvokevirtual       char[] java.lang.String.toCharArray()\r\n<\/pre>\n<pre lang=\"java\">\/\/\r\n    t2 = \"I\".concat(String.valueOf(t2)).toCharArray();\r\n\/\/\r\n<\/pre>\n<p style=\"text-align: justify;\">It is now time to get back to <strong>sr_01E1()<\/strong> mentioned at the beginning of this paragraph. It is called with below arguments:<\/p>\n<pre lang=\"java\">arg1 = \"Correct!\".toCharArray();\r\narg2 = \"Incorrect!\".toCharArray();\r\narg3 = t1;\r\narg4 = t2;\r\n<\/pre>\n<p style=\"text-align: justify;\">Those four char arrays are combined into new char array that is constructed by adding elements from each input array at each position: <strong>arg1[i] + arg2[i] + arg3[i] + arg4[i]<\/strong>. If calculated value exceeds char boundaries, then low 16bits are stored as a char at i-th position and high 16bits are <b>carried<\/b> to the (i + 1) element calculation. Overall, it looks like plain <em>big number<\/em> addition. Noticing this fact is a big step towards final solution, but it will be discussed later (of course!). Let's call this new array <strong>sum[]<\/strong>:<\/p>\n<pre lang=\"java\">    char[] sum = sr_01E1(\"Incorrect!\", \"Correct!\", t1, t2);\r\n<\/pre>\n<p style=\"text-align: justify;\">One more thing about <strong>sr_01E1()<\/strong>, it starts with <em>carry value<\/em> set to <strong>5<\/strong>, so <strong>5<\/strong> is like an additional argument added to the four input values. <strong>sum[]<\/strong> array is used as a parameter to yet another subroutine, lets call it <strong>sr_0274()<\/strong>, this subroutine is called twice and operates on <strong>double <\/strong>values (yay!):<\/p>\n<pre lang=\"java\">double sr_0274(char[] t, double d)\t\r\n{\t\r\n    double ret = 0.0;\r\n    for (int i = 0; i < t.length; i++)\r\n    {\t\t\t\t\t\r\n        \/\/ret = (ret + ((t[i] * Math.pow((double)2, (double)(-1048 + (i << 4)))) % d)) % d;\r\n        \/\/splitted for sake of readability\r\n        double pow2 = Math.pow(2, -1048 + (i << 4));\r\n        pow2 %= d;\r\n        ret = (ret + t[i]*pow2) % d;\r\n    }\r\n    return ret;\r\n}\t\r\n<\/pre>\n<p style=\"text-align: justify;\">Above function is called with below arguments:<\/p>\n<pre lang=\"java\">    double da = sr_0274(sum, 1.112536929253537e-308);\r\n    double db = sr_0274(sum, 1.112536929253532e-308);\r\n<\/pre>\n<p style=\"text-align: justify;\">At the end <strong>da <\/strong>is added to <strong>db <\/strong>and compared to <strong>0.0<\/strong> (yes, comparison is made on double type):<\/p>\n<pre lang=\"dos\">0000016B: \tdload               local.04\r\n0000016D: \tdadd\r\n0000016E: \tdconst_0\r\n0000016F: \tdcmpl\r\n<\/pre>\n<p style=\"text-align: justify;\">If <strong>da + db == 0.0<\/strong> it prints <em>\"Correct!\"<\/em>, otherwise it prints <em>\"Incorrect!\"<\/em>.<\/p>\n<h3><a id=\"alg_reversing\">Algorithm reversing<\/a><\/h3>\n<p style=\"text-align: justify;\">Earlier, I've made some statements that fosters reversing, I mean the one about big number addition. As I said, noticing it, was a big progress (so, some of you may already decoded other operations), but lets ignore it for now and analyse other parts of the code. The whole process is called reversing, so I'll start from the end :). <strong>sr_0274()<\/strong> is a good candidate to start, as it is the last piece of serial verification. Printing each iteration as an equation helps with understanding what is really happening:<\/p>\n<pre lang=\"java\">  0: ret = (ret + (t[0]*(2^-1048)) % d) % d\r\n  1: ret = (ret + (t[1]*(2^-1032)) % d) % d\r\n  2: ret = (ret + (t[2]*(2^-1016)) % d) % d\r\n  3: ret = (ret + (t[3]*(2^-1000)) % d) % d\r\n  4: ret = (ret + (t[4]*(2^-984)) % d) % d\r\n  5: ret = (ret + (t[5]*(2^-968)) % d) % d\r\n  6: ret = (ret + (t[6]*(2^-952)) % d) % d\r\n  7: ret = (ret + (t[7]*(2^-936)) % d) % d\r\n...\r\n 99: ret = (ret + (t[99]*(2^536)) % d) % d\r\n100: ret = (ret + (t[100]*(2^552)) % d) % d\r\n101: ret = (ret + (t[101]*(2^568)) % d) % d\r\n102: ret = (ret + (t[102]*(2^584)) % d) % d\r\n103: ret = (ret + (t[103]*(2^600)) % d) % d\r\n104: ret = (ret + (t[104]*(2^616)) % d) % d\r\n105: ret = (ret + (t[105]*(2^632)) % d) % d\r\n<\/pre>\n<p style=\"text-align: justify;\">Analysing this loop on <strong>double <\/strong>values gave me a headache, at some point I've switched to hexadecimal representation (<strong>Double.doubleToLongBits()<\/strong>). I was playing a bit with values in <strong>t[]<\/strong> array and watching how the output is changing. I was also printing partial results like <strong>t[i]*(2^n)<\/strong>, <strong>(t[i]*(2^n))%d<\/strong> etc, virtually everything that can help me in achieving the goal. Constants passed to <strong>sr_0274()<\/strong> are also better looking in hexadecimal form:<\/p>\n<pre lang=\"java\">    A = 1.112536929253537e-308 = 0x0007FFFFFFFFFF7F\r\n    B = 1.112536929253532e-308 = 0x0007FFFFFFFFFF75\r\n<\/pre>\n<p style=\"text-align: justify;\">It is worth noting that both values fits in <strong>52 bits<\/strong>. <strong>Double <\/strong>representation in <strong>Java<\/strong> follows <strong>IEEE 754<\/strong> standard, it defines double as <strong>64bit <\/strong>value, where <strong>1 bit<\/strong> is used as a <strong>sign<\/strong>, <strong>11 bits<\/strong> are used as an <strong>exponent <\/strong>and <strong>mantissa <\/strong>is stored in the remaining <strong>52 bits<\/strong>. So, constants from the crackme are using only <strong>mantissa <\/strong>part, is it anyhow helpful? Actually yes, I've made a nice observation, that simplifies things a lot. <strong>Double modulo<\/strong> operation for values that fits in <strong>mantissa <\/strong>part can be freely changed to <strong>integer modulo<\/strong>! Thinking about previously mentioned big number addition, I've decided to look at a big number modulo algorithms. I've found this article: <a href=\"http:\/\/www.devx.com\/tips\/Tip\/39012\" title=\"Finding Modulus of a Very Large Number with a Normal Number\">http:\/\/www.devx.com\/tips\/Tip\/39012<\/a> and it confirmed my theory, that <strong>sr_0274()<\/strong> might by just big number modulo. Lets try to put together some initial calculations that I've recovered up to this moment:<\/p>\n<pre lang=\"java\">    \/\/ sum[] -> char array produced by sr_01E1(), let's call it just sum\r\n    (sum % A) + (sum % B) = 0;\r\n    \/\/ sum is always above 0, so above can be represented as:\r\n    sum % (A*B) = 0;\r\n    \/\/ and after further transformation\r\n    sum = N*A*B;\r\n<\/pre>\n<p style=\"text-align: justify;\"><strong>sum[]<\/strong> is a result of an addition of a few values: bignumbers made from <em>\"Incorrect!\"<\/em> and <em>\"Correct!\"<\/em> char arrays, const value <strong>5<\/strong> and two bignumbers made from <strong>t1[]<\/strong> and <strong>t2[]<\/strong> char arrays. As I've described earlier,<strong> t2[]<\/strong> is concatenated with <em>\"I\"<\/em> char, to simplify the things, I've to represent this concatenation as a mathematical operation:<\/p>\n<pre lang=\"java\">    t2 = \"I\".concat(String.valueOf(t2)).toCharArray();\r\n\r\n    \"I\" -> 0x49\r\n    concat -> it works like left shift by 16 bits, or multiplication by 0x10000\r\n\r\n    t2 = (t2 * 0x10000) + 0x49\r\n<\/pre>\n<p style=\"text-align: justify;\">Updating calculations:<\/p>\n<pre lang=\"java\">    incor = bignum(\"Incorrect!\")\r\n    cor = bignum(\"Correct!\")\r\n\r\n    sum = incor + cor + 5 + t1 + (t2 * 0x10000) + 0x49\r\n    \/\/ so\r\n    incor + cor + 5 + t1 + (t2 * 0x10000) + 0x49 = N*A*B\r\n<\/pre>\n<p style=\"text-align: justify;\"><strong>t1[]<\/strong> and <strong>t2[]<\/strong> arrays are results of yet unknown operation that strangely resembles bignum multiplication. I'll not go into the details as it would probably took to much space and I'm not sure if it would be clear enough, but the algorithm used in the crackme is standard <a href=\"http:\/\/en.wikipedia.org\/wiki\/Multiplication_algorithm#Long_multiplication\" title=\"Long multiplication\">long multiplication algorithm<\/a>, it is optimized to use chunks of <strong>16bit <\/strong>data, but it's just an implementation detail. <strong>t1 + (t2 * 0x10000)<\/strong> is the final addition for the multiplication algorithm, so calculations can be updated again:<\/p>\n<pre lang=\"java\">    C -> bignum(output_c1)\r\n    Serial -> bignum(post_serial_array)\r\n\r\n    t1 + (t2 * 0x10000) = Serial*C\r\n    \/\/ so\r\n    Serial*C + incor + cor + 5 + 0x49 = N*A*B\r\n<\/pre>\n<p style=\"text-align: justify;\">At this point there are two unknown values in the equation: <strong>Serial <\/strong>and <strong>N<\/strong>. I'll transform current equation to the form that will calculate <strong>Serial number<\/strong>:<\/p>\n<pre lang=\"java\">    Serial*C = N*A*B - incor - cor - 5 - 0x49\r\n\r\n    Serial = (N*A*B - incor - cor - 5 - 0x49) \/ C\r\n<\/pre>\n<p style=\"text-align: justify;\">Both <strong>Serial <\/strong>and <strong>N <\/strong>values has to be Integer numbers, this information allows me to write below equation:<\/p>\n<pre lang=\"java\">    (N*A*B - incor - cor - 5 - 0x49) % C = 0\r\n    \/\/it can be further transformed to the form of standard linear congruence ax = b (mod c)\r\n    N*A*B = incor + cor + 5 + 0x49 (mod C)\r\n    \/\/ a = A*B\r\n    \/\/ x = N\r\n    \/\/ b = incor + cor + 5 + 0x49\r\n    \/\/ c = C\r\n<\/pre>\n<p style=\"text-align: justify;\">Solving <strong>linear congruences<\/strong> requires <strong><a href=\"http:\/\/en.wikipedia.org\/wiki\/Extended_Euclidean_algorithm\" title=\"Extended Euclidean algorithm\">extended Euclidean algorithm<\/a><\/strong>, values used by the crackme are chosen in the way that it is possible to solve it. Final solution:<\/p>\n<pre lang=\"java\">    \/\/ EGCD() - extended Euclidean algorithm\r\n    N = EGCD(A*B, C)\r\n\r\n    Serial = (EGCD(A*B, C)*A*B - incor - cor - 5 - 0x49) \/ C\r\n<\/pre>\n<p style=\"text-align: justify;\">Correct serial number:<\/p>\n<pre lang=\"java\">    \"5876C9436400AD9AC7BA037602CD4261D2C87DB8FAA9F921A93AB2DDFA2C0215\"<\/pre>\n<p style=\"text-align: justify;\">Keygen:<br \/>\n<a href=\"http:\/\/blog.rewolf.pl\/blog\/wp-content\/uploads\/2014\/02\/Crackme3Kgn.java_.txt\">Crackme3Kgn.java<\/a><\/p>\n<h3><a id=\"finalwords\">Final words<\/a><\/h3>\n<p style=\"text-align: justify;\">That was quite long solution, I hope some people made it till the end. It could be even longer, but I've decided to skip some details for clarity. I've never thought that java crackmes can be as tricky as x86, but this crackme proved it. It is also a great example to learn various topics like: java bytecode level analysis and debugging, big numbers algorithms, solving linear congruences. As a bonus, last year dirtyJOE development was mainly driven by this crackme. That's all for now.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Every once in a while I&#8217;m posting solution to some crackme that I consider interesting. By interesting, I mean the solution, so it is not exactly about key generation algorithm but also about technology and tricks that are utilized. Looking at the traffic statistics, it seems that this topic isn&#8217;t exactly the one that people [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[15,7,3],"tags":[],"_links":{"self":[{"href":"http:\/\/blog.rewolf.pl\/blog\/index.php?rest_route=\/wp\/v2\/posts\/856"}],"collection":[{"href":"http:\/\/blog.rewolf.pl\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/blog.rewolf.pl\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/blog.rewolf.pl\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/blog.rewolf.pl\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=856"}],"version-history":[{"count":199,"href":"http:\/\/blog.rewolf.pl\/blog\/index.php?rest_route=\/wp\/v2\/posts\/856\/revisions"}],"predecessor-version":[{"id":1079,"href":"http:\/\/blog.rewolf.pl\/blog\/index.php?rest_route=\/wp\/v2\/posts\/856\/revisions\/1079"}],"wp:attachment":[{"href":"http:\/\/blog.rewolf.pl\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=856"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/blog.rewolf.pl\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=856"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/blog.rewolf.pl\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=856"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}