Repository: zzwlpx/ljd Branch: master Commit: 16d485d0f822 Files: 46 Total size: 217.5 KB Directory structure: gitextract_2i5rc5bq/ ├── .gitignore ├── LICENSE ├── README.md ├── README.md.txt ├── gconfig.py ├── ljd/ │ ├── __init__.py │ ├── ast/ │ │ ├── __init__.py │ │ ├── builder.py │ │ ├── helpers.py │ │ ├── locals.py │ │ ├── mutator.py │ │ ├── nodes.py │ │ ├── slotworks.py │ │ ├── traverse.py │ │ ├── unwarper.py │ │ └── validator.py │ ├── bytecode/ │ │ ├── __init__.py │ │ ├── constants.py │ │ ├── debuginfo.py │ │ ├── helpers.py │ │ ├── instructions.py │ │ └── prototype.py │ ├── lua/ │ │ ├── __init__.py │ │ └── writer.py │ ├── pseudoasm/ │ │ ├── __init__.py │ │ ├── constants.py │ │ ├── instructions.py │ │ ├── prototype.py │ │ └── writer.py │ ├── rawdump/ │ │ ├── __init__.py │ │ ├── code.py │ │ ├── constants.py │ │ ├── debuginfo.py │ │ ├── header.py │ │ ├── parser.py │ │ └── prototype.py │ └── util/ │ ├── __init__.py │ ├── binstream.py │ ├── indentedstream.py │ └── log.py ├── main.py └── test/ ├── breaks.lua ├── expression.lua ├── ifs.lua ├── loop.lua └── primitive.lua ================================================ FILE CONTENTS ================================================ ================================================ FILE: .gitignore ================================================ *.lua[co] *.py[cod] # C extensions *.so # Packages *.egg *.egg-info dist build eggs parts bin var sdist develop-eggs .installed.cfg lib lib64 __pycache__ # Installer logs pip-log.txt # Unit test / coverage reports .coverage .tox nosetests.xml # Translations *.mo # Mr Developer .mr.developer.cfg .project .pydevproject .*.swp ================================================ FILE: LICENSE ================================================ The MIT License (MIT) Copyright (c) 2013 Andrian Nord Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ================================================ FILE: README.md ================================================ 代码在fork https://github.com/NightNord/ljd 的基础上修改的 主要修正了原版本没有将_read_header函数解析过的header中的flag值带进_read_protoypes。 解决了原代码只能解析带调试信息的luajit字节码，而解析不带调试信息字节码时报错的问题。 此版本可解析luajit2.1.0-beta2的字节码文件。如果要解析其它版本的luajit字节码文件，需要将 luajit源码中的opcode，写进instruction.py 和 code.py 使用说明：python main.py "path of luajit-bytecode" 解释文档：http://www.freebuf.com/column/177810.html 参考：https://github.com/NightNord/ljd https://github.com/feicong/lua_re/blob/master/lua/lua_re3.md https://bbs.pediy.com/thread-216800.htm ================================================ FILE: README.md.txt ================================================ LuaJIT raw-bytecode decompiler (LJD) === The original name was _ljwthgnd_ as in _LuaJIT 'What The Hell is Going On' Decompiler_ named under the LuaJIT C sources variable naming convention. __WARNING!__ This code is nor finished, nor tested yet! There is no even slightest warranty that resulting code is even near to the original. Use it at your own risk of the wasted time. __SECOND WARNING!__ And, BTW, this all is a one huge prototype. Because the "release" version should be written into lua itself. Because it's cool to decompile the decompiler - a great test too! Requirements --- Python __3.0+__ from python.org How to use it --- There is no argument parsing right now, so comment out things in the ```main.py``` script and launch it as in ```main.py path/to/file.luac``` TODO --- There is a lot of work to do, in the order of priority 0. Logical subexpressions in while statements: ```lua while x < (xi and 2 or 3) do print ("Hello crazy world!") end ``` Logical subexpressions (the subexpressions used as operands in ariphmetic or comparison operations inside other exrpressions) are currently supported only for ifs. To support them for whiles and repeat untils an expression unwarping logic should be moved at the very beginning. But it won't work without all fixes being done in a loop unwarping logic. So we need to split that and move the fixes before expressions before loops before ifs. That's not that easy... 1. AST Mutations: 1. Use the line information (or common sense if there is no line information) to squash similar expressions into a single expression. 2. Formatting improvements 1. Use the line information (or common sense) to preserve empty lines and break long statements like in the original code. This is mostly done, but only in the "common sense" part. 2. Use method-style calls and definitions for tables. 3. Features not supported: 1. GOTO statement (from lua 5.2). All the required functionality is now in place, but that's rather a low-priority task right now. 2. Local sub-blocks: ```lua do ... end ``` These subblocks are not reflected anyhow directly in the bytecode. The only way to guess them is to watch local variable scopes, which is simple enough in case of non-stripped bytecode and a bit harder otherwise. ================================================ FILE: gconfig.py ================================================ #zzw 20180714 support str encode gFlagDic = {'strEncode' : 'ascii'} ================================================ FILE: ljd/__init__.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # ================================================ FILE: ljd/ast/__init__.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # ================================================ FILE: ljd/ast/builder.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # import ljd.bytecode.instructions as ins from ljd.bytecode.helpers import get_jump_destination from ljd.bytecode.constants import T_NIL, T_FALSE, T_TRUE import ljd.ast.nodes as nodes class _State(): def __init__(self): self.constants = None self.debuginfo = None self.block = None self.blocks = [] self.block_starts = {} def _warp_in_block(self, addr): #print (self.block_starts) #print (addr) block = self.block_starts[addr] block.warpins_count += 1 return block def build(prototype): return _build_function_definition(prototype) def _build_function_definition(prototype): node = nodes.FunctionDefinition() state = _State() state.constants = prototype.constants state.debuginfo = prototype.debuginfo node._upvalues = prototype.constants.upvalue_references node._debuginfo = prototype.debuginfo node._instructions_count = len(prototype.instructions) node.arguments.contents = _build_function_arguments(state, prototype) if prototype.flags.is_variadic: node.arguments.contents.append(nodes.Vararg()) instructions = prototype.instructions #print (len(instructions)) node.statements.contents = _build_function_blocks(state, instructions) return node def _build_function_arguments(state, prototype): arguments = [] count = prototype.arguments_count slot = 0 while slot < count: variable = _build_slot(state, 0, slot) arguments.append(variable) slot += 1 return arguments def _build_function_blocks(state, instructions): _blockenize(state, instructions) _establish_warps(state, instructions) state.blocks[0].warpins_count = 1 for block in state.blocks: addr = block.first_address state.block = block while addr <= block._last_body_addr: instruction = instructions[addr] statement = _build_statement(state, addr, instruction) if statement is not None: line = state.debuginfo.lookup_line_number(addr) setattr(statement, "_addr", addr) setattr(statement, "_line", line) block.contents.append(statement) addr += 1 return state.blocks _JUMP_WARP_INSTRUCTIONS = set(( ins.UCLO.opcode, ins.ISNEXT.opcode, ins.JMP.opcode, ins.FORI.opcode, ins.JFORI.opcode )) _WARP_INSTRUCTIONS = _JUMP_WARP_INSTRUCTIONS | set(( ins.FORL.opcode, ins.IFORL.opcode, ins.JFORL.opcode, ins.ITERL.opcode, ins.IITERL.opcode, ins.JITERL.opcode, ins.LOOP.opcode )) def _blockenize(state, instructions): addr = 1 # Duplicates are possible and ok, but we need to sort them out last_addresses = set() while addr < len(instructions): instruction = instructions[addr] opcode = instruction.opcode if opcode not in _WARP_INSTRUCTIONS: addr += 1 continue if opcode in _JUMP_WARP_INSTRUCTIONS: destination = get_jump_destination(addr, instruction) if opcode != ins.UCLO.opcode or destination != addr + 1: last_addresses.add(destination - 1) last_addresses.add(addr) else: last_addresses.add(addr) addr += 1 last_addresses = sorted(list(last_addresses)) last_addresses.append(len(instructions) - 1) # This could happen if something jumps to the first instruction # We don't need "zero block" with function header, so simply ignore # this if last_addresses[0] == 0: last_addresses.pop(0) previous_last_address = 0 index = 0 for last_address in last_addresses: block = nodes.Block() block.index = index block.first_address = previous_last_address + 1 block.last_address = last_address state.blocks.append(block) state.block_starts[block.first_address] = block previous_last_address = last_address index += 1 def _establish_warps(state, instructions): state.blocks[0].warpins_count = 1 for block in state.blocks[:-1]: state.block = block end_addr = block.last_address + 1 start_addr = max(block.last_address - 1, block.first_address) warp = instructions[start_addr:end_addr] #print (block.first_address) block.warp, shift = _build_warp(state, block.last_address, warp) setattr(block, "_last_body_addr", block.last_address - shift) setattr(block.warp, "_addr", block.last_address - shift + 1) last_block = state.blocks[-1] last_block.warp = nodes.EndWarp() setattr(last_block, "_last_body_addr", last_block.last_address) setattr(last_block.warp, "_addr", last_block.last_address) def _build_warp(state, last_addr, instructions): last = instructions[-1] #print ("%x"%(last.opcode)) if last.opcode in (ins.JMP.opcode, ins.UCLO.opcode, ins.ISNEXT.opcode): return _build_jump_warp(state, last_addr, instructions) elif last.opcode >= ins.ITERL.opcode and last.opcode <= ins.JITERL.opcode: assert len(instructions) == 2 return _build_iterator_warp(state, last_addr, instructions) elif last.opcode >= ins.FORL.opcode and last.opcode <= ins.JFORL.opcode: return _build_numeric_loop_warp(state, last_addr, last) else: return _build_flow_warp(state, last_addr, last) def _build_jump_warp(state, last_addr, instructions): last = instructions[-1] opcode = 256 if len(instructions) == 1 else instructions[-2].opcode if opcode <= ins.ISF.opcode: assert last.opcode != ins.ISNEXT.opcode return _build_conditional_warp(state, last_addr, instructions) else: return _build_unconditional_warp(state, last_addr, last) def _build_conditional_warp(state, last_addr, instructions): condition = instructions[-2] condition_addr = last_addr - 1 warp = nodes.ConditionalWarp() if condition.opcode in (ins.ISTC.opcode, ins.ISFC.opcode): expression = _build_unary_expression(state, condition_addr, condition) setattr(warp, "_slot", condition.A) elif condition.opcode >= ins.IST.opcode: expression = _build_unary_expression(state, condition_addr, condition) setattr(warp, "_slot", condition.CD) else: expression = _build_comparison_expression(state, condition_addr, condition) warp.condition = expression jump = instructions[-1] jump_addr = last_addr destination = get_jump_destination(jump_addr, jump) # A condition is inverted during the preparation phase above warp.false_target = state._warp_in_block(destination) warp.true_target = state._warp_in_block(jump_addr + 1) return warp, 2 def _build_unconditional_warp(state, addr, instruction): warp = nodes.UnconditionalWarp() warp.type = nodes.UnconditionalWarp.T_JUMP opcode = instruction.opcode warp.is_uclo = opcode == ins.UCLO.opcode if warp.is_uclo and instruction.CD == 0: # Not a jump return _build_flow_warp(state, addr, instruction) else: destination = get_jump_destination(addr, instruction) warp.target = state._warp_in_block(destination) return warp, 1 def _build_iterator_warp(state, last_addr, instructions): iterator = instructions[-2] iterator_addr = last_addr - 1 assert iterator.opcode in (ins.ITERC.opcode, ins.ITERN.opcode) warp = nodes.IteratorWarp() base = iterator.A warp.controls.contents = [ _build_slot(state, iterator_addr, base - 3), # generator _build_slot(state, iterator_addr, base - 2), # state _build_slot(state, iterator_addr, base - 1) # control ] last_slot = base + iterator.B - 2 slot = base while slot <= last_slot: variable = _build_slot(state, iterator_addr - 1, slot) warp.variables.contents.append(variable) slot += 1 jump = instructions[-1] jump_addr = last_addr destination = get_jump_destination(jump_addr, jump) warp.way_out = state._warp_in_block(jump_addr + 1) warp.body = state._warp_in_block(destination) return warp, 2 def _build_numeric_loop_warp(state, addr, instruction): warp = nodes.NumericLoopWarp() base = instruction.A warp.index = _build_slot(state, addr, base + 3) warp.controls.contents = [ _build_slot(state, addr, base + 0), # start _build_slot(state, addr, base + 1), # limit _build_slot(state, addr, base + 2) # step ] #print ("%x" % instruction.A) #print ("%x" %instruction.CD) destination = get_jump_destination(addr, instruction) warp.body = state._warp_in_block(destination) warp.way_out = state._warp_in_block(addr + 1) return warp, 1 def _build_flow_warp(state, addr, instruction): warp = nodes.UnconditionalWarp() warp.type = nodes.UnconditionalWarp.T_FLOW warp.target = state._warp_in_block(addr + 1) opcode = instruction.opcode shift = 1 if opcode in (ins.FORI.opcode, ins.UCLO.opcode) else 0 return warp, shift def _build_statement(state, addr, instruction): opcode = instruction.opcode A_type = instruction.A_type # Generic assignments - handle the ASSIGNMENT stuff below if A_type == ins.T_DST or A_type == ins.T_UV: return _build_var_assignment(state, addr, instruction) # ASSIGNMENT starting from MOV and ending at KPRI elif opcode == ins.KNIL.opcode: return _build_knil(state, addr, instruction) # ASSIGNMENT starting from UGET and ending at USETP # SKIP UCL0 is handled below # ASSIGNMENT starting from FNEW and ending at GGET elif opcode == ins.GSET.opcode: return _build_global_assignment(state, addr, instruction) # ASSIGNMENT starting from TGETV and ending at TGETB elif opcode >= ins.TSETV.opcode and opcode <= ins.TSETB.opcode: return _build_table_assignment(state, addr, instruction) elif opcode == ins.TSETM.opcode: return _build_table_mass_assignment(state, addr, instruction) elif opcode >= ins.CALLM.opcode and opcode <= ins.CALLT.opcode: return _build_call(state, addr, instruction) elif opcode == ins.VARG.opcode: return _build_vararg(state, addr, instruction) elif opcode >= ins.RETM.opcode and opcode <= ins.RET1.opcode: return _build_return(state, addr, instruction) else: assert opcode == ins.UCLO.opcode or ( opcode >= ins.LOOP.opcode and opcode <= ins.JLOOP.opcode) # Noop return None def _build_var_assignment(state, addr, instruction): opcode = instruction.opcode assignment = nodes.Assignment() # Unary assignment operators (A = op D) if opcode == ins.MOV.opcode \ or opcode == ins.NOT.opcode \ or opcode == ins.UNM.opcode \ or opcode == ins.LEN.opcode: expression = _build_unary_expression(state, addr, instruction) # Binary assignment operators (A = B op C) elif opcode <= ins.POW.opcode: expression = _build_binary_expression(state, addr, instruction) # Concat assignment type (A = B .. B + 1 .. ... .. C - 1 .. C) elif opcode == ins.CAT.opcode: expression = _build_concat_expression(state, addr, instruction) # Constant assignment operators except KNIL, which is weird anyway elif opcode <= ins.KPRI.opcode: expression = _build_const_expression(state, addr, instruction) elif opcode == ins.UGET.opcode: expression = _build_upvalue(state, addr, instruction.CD) elif opcode == ins.USETV.opcode: expression = _build_slot(state, addr, instruction.CD) elif opcode <= ins.USETP.opcode: expression = _build_const_expression(state, addr, instruction) elif opcode == ins.FNEW.opcode: expression = _build_function(state, instruction.CD) elif opcode == ins.TNEW.opcode: expression = nodes.TableConstructor() elif opcode == ins.TDUP.opcode: expression = _build_table_copy(state, instruction.CD) elif opcode == ins.GGET.opcode: expression = _build_global_variable(state, addr, instruction.CD) else: assert opcode <= ins.TGETB.opcode expression = _build_table_element(state, addr, instruction) assignment.expressions.contents.append(expression) if instruction.A_type == ins.T_DST: destination = _build_slot(state, addr, instruction.A) else: assert instruction.A_type == ins.T_UV destination = _build_upvalue(state, addr, instruction.A) assignment.destinations.contents.append(destination) return assignment def _build_knil(state, addr, instruction): node = _build_range_assignment(state, addr, instruction.A, instruction.CD) node.expressions.contents = [_build_primitive(state, None)] return node def _build_global_assignment(state, addr, instruction): assignment = nodes.Assignment() variable = _build_global_variable(state, addr, instruction.CD) expression = _build_slot(state, addr, instruction.A) assignment.destinations.contents.append(variable) assignment.expressions.contents.append(expression) return assignment def _build_table_assignment(state, addr, instruction): assignment = nodes.Assignment() destination = _build_table_element(state, addr, instruction) expression = _build_slot(state, addr, instruction.A) assignment.destinations.contents.append(destination) assignment.expressions.contents.append(expression) return assignment def _build_table_mass_assignment(state, addr, instruction): assignment = nodes.Assignment() base = instruction.A destination = nodes.TableElement() destination.key = nodes.MULTRES() destination.table = _build_slot(state, addr, base - 1) assignment.destinations.contents = [destination] assignment.expressions.contents = [nodes.MULTRES()] return assignment def _build_call(state, addr, instruction): call = nodes.FunctionCall() call.function = _build_slot(state, addr, instruction.A) call.arguments.contents = _build_call_arguments(state, addr, instruction) if instruction.opcode <= ins.CALL.opcode: if instruction.B == 0: node = nodes.Assignment() node.destinations.contents.append(nodes.MULTRES()) node.expressions.contents.append(call) elif instruction.B == 1: node = call else: from_slot = instruction.A to_slot = instruction.A + instruction.B - 2 node = _build_range_assignment(state, addr, from_slot, to_slot) node.expressions.contents.append(call) else: assert instruction.opcode <= ins.CALLT.opcode node = nodes.Return() node.returns.contents.append(call) return node def _build_vararg(state, addr, instruction): base = instruction.A last_slot = base + instruction.B - 2 if last_slot < base: node = nodes.Assignment() node.destinations.contents.append(nodes.MULTRES()) node.expressions.contents.append(nodes.Vararg()) else: node = _build_range_assignment(state, addr, base, last_slot) node.expressions.contents.append(nodes.Vararg()) return node def _build_return(state, addr, instruction): node = nodes.Return() base = instruction.A last_slot = base + instruction.CD - 1 if instruction.opcode != ins.RETM.opcode: last_slot -= 1 slot = base # Negative count for the RETM is OK while slot <= last_slot: variable = _build_slot(state, addr, slot) node.returns.contents.append(variable) slot += 1 if instruction.opcode == ins.RETM.opcode: node.returns.contents.append(nodes.MULTRES()) return node def _build_call_arguments(state, addr, instruction): base = instruction.A last_argument_slot = base + instruction.CD is_variadic = (instruction.opcode == ins.CALLM.opcode \ or instruction.opcode == ins.CALLMT.opcode) if not is_variadic: last_argument_slot -= 1 arguments = [] slot = base + 1 while slot <= last_argument_slot: argument = _build_slot(state, addr, slot) arguments.append(argument) slot += 1 if is_variadic: arguments.append(nodes.MULTRES()) return arguments def _build_range_assignment(state, addr, from_slot, to_slot): assignment = nodes.Assignment() slot = from_slot assert from_slot <= to_slot while slot <= to_slot: destination = _build_slot(state, addr, slot) assignment.destinations.contents.append(destination) slot += 1 return assignment _BINARY_OPERATOR_MAP = [None] * 255 _BINARY_OPERATOR_MAP[ins.ADDVN.opcode] = nodes.BinaryOperator.T_ADD _BINARY_OPERATOR_MAP[ins.SUBVN.opcode] = nodes.BinaryOperator.T_SUBTRACT _BINARY_OPERATOR_MAP[ins.MULVN.opcode] = nodes.BinaryOperator.T_MULTIPLY _BINARY_OPERATOR_MAP[ins.DIVVN.opcode] = nodes.BinaryOperator.T_DIVISION _BINARY_OPERATOR_MAP[ins.MODVN.opcode] = nodes.BinaryOperator.T_MOD def _build_binary_expression(state, addr, instruction): operator = nodes.BinaryOperator() opcode = instruction.opcode if opcode == ins.POW.opcode: operator.type = nodes.BinaryOperator.T_POW else: map_index = opcode - ins.ADDVN.opcode map_index %= 5 map_index += ins.ADDVN.opcode operator.type = _BINARY_OPERATOR_MAP[map_index] if instruction.B_type == ins.T_VAR and instruction.CD_type == ins.T_VAR: operator.left = _build_slot(state, addr, instruction.B) operator.right = _build_slot(state, addr, instruction.CD) elif instruction.B_type == ins.T_VAR: operator.left = _build_slot(state, addr, instruction.B) operator.right = _build_numeric_constant(state, instruction.CD) else: assert instruction.CD_type == ins.T_VAR operator.right = _build_slot(state, addr, instruction.B) operator.left = _build_numeric_constant(state, instruction.CD) return operator def _build_concat_expression(state, addr, instruction): operator = nodes.BinaryOperator() operator.type = nodes.BinaryOperator.T_CONCAT slot = instruction.B operator.left = _build_slot(state, addr, slot) operator.right = _build_slot(state, addr, slot + 1) slot += 2 while slot <= instruction.CD: upper_operator = nodes.BinaryOperator() upper_operator.left = operator upper_operator.right = _build_slot(state, addr, slot) upper_operator.type = nodes.BinaryOperator.T_CONCAT operator = upper_operator slot += 1 return operator def _build_const_expression(state, addr, instruction): CD_type = instruction.CD_type if CD_type == ins.T_STR: return _build_string_constant(state, instruction.CD) elif CD_type == ins.T_CDT: return _build_cdata_constant(state, instruction.CD) elif CD_type == ins.T_SLIT: value = instruction.CD if value & 0x8000: value = -0x10000 + value return _build_literal(state, value) elif CD_type == ins.T_LIT: return _build_literal(state, instruction.CD) elif CD_type == ins.T_NUM: return _build_numeric_constant(state, instruction.CD) else: assert CD_type == ins.T_PRI return _build_primitive(state, instruction.CD) def _build_table_element(state, addr, instruction): node = nodes.TableElement() node.table = _build_slot(state, addr, instruction.B) if instruction.CD_type == ins.T_VAR: node.key = _build_slot(state, addr, instruction.CD) else: node.key = _build_const_expression(state, addr, instruction) return node def _build_function(state, slot): prototype = state.constants.complex_constants[slot] return _build_function_definition(prototype) def _build_table_copy(state, slot): node = nodes.TableConstructor() table = state.constants.complex_constants[slot] i = 0 for value in table.array: record = nodes.ArrayRecord() record.value = _build_table_record_item(value) node.array.contents.append(record) i += 1 for key, value in table.dictionary: record = nodes.TableRecord() record.key = _build_table_record_item(key) record.value = _build_table_record_item(value) node.records.contents.append(record) return node def _build_table_record_item(value): if value is None: item = nodes.Primitive() item.type = nodes.Primitive.T_NIL elif value is True: item = nodes.Primitive() item.type = nodes.Primitive.T_TRUE elif value is False: item = nodes.Primitive() item.type = nodes.Primitive.T_FALSE elif isinstance(value, int): item = nodes.Constant() item.value = value item.type = nodes.Constant.T_INTEGER elif isinstance(value, float): item = nodes.Constant() item.value = value item.type = nodes.Constant.T_FLOAT elif isinstance(value, str): item = nodes.Constant() item.value = value item.type = nodes.Constant.T_STRING return item _COMPARISON_MAP = [None] * 255 # Mind the inversion - comparison operators are affecting JMP to the next block # So in the normal code a comparison will be inverted _COMPARISON_MAP[ins.ISLT.opcode] = nodes.BinaryOperator.T_GREATER_OR_EQUAL _COMPARISON_MAP[ins.ISGE.opcode] = nodes.BinaryOperator.T_LESS_THEN _COMPARISON_MAP[ins.ISLE.opcode] = nodes.BinaryOperator.T_GREATER_THEN _COMPARISON_MAP[ins.ISGT.opcode] = nodes.BinaryOperator.T_LESS_OR_EQUAL _COMPARISON_MAP[ins.ISEQV.opcode] = nodes.BinaryOperator.T_NOT_EQUAL _COMPARISON_MAP[ins.ISNEV.opcode] = nodes.BinaryOperator.T_EQUAL _COMPARISON_MAP[ins.ISEQS.opcode] = nodes.BinaryOperator.T_NOT_EQUAL _COMPARISON_MAP[ins.ISNES.opcode] = nodes.BinaryOperator.T_EQUAL _COMPARISON_MAP[ins.ISEQN.opcode] = nodes.BinaryOperator.T_NOT_EQUAL _COMPARISON_MAP[ins.ISNEN.opcode] = nodes.BinaryOperator.T_EQUAL _COMPARISON_MAP[ins.ISEQP.opcode] = nodes.BinaryOperator.T_NOT_EQUAL _COMPARISON_MAP[ins.ISNEP.opcode] = nodes.BinaryOperator.T_EQUAL def _build_comparison_expression(state, addr, instruction): operator = nodes.BinaryOperator() operator.left = _build_slot(state, addr, instruction.A) opcode = instruction.opcode if opcode == ins.ISEQS.opcode or opcode == ins.ISNES.opcode: operator.right = _build_string_constant(state, instruction.CD) elif opcode == ins.ISEQN.opcode or opcode == ins.ISNEN.opcode: operator.right = _build_numeric_constant(state, instruction.CD) elif opcode == ins.ISEQP.opcode or opcode == ins.ISNEP.opcode: operator.right = _build_primitive(state, instruction.CD) else: operator.right = _build_slot(state, addr, instruction.CD) operator.type = _COMPARISON_MAP[instruction.opcode] assert operator.type is not None return operator def _build_unary_expression(state, addr, instruction): opcode = instruction.opcode variable = _build_slot(state, addr, instruction.CD) # Mind the inversion if opcode == ins.ISFC.opcode \ or opcode == ins.ISF.opcode \ or opcode == ins.MOV.opcode: return variable operator = nodes.UnaryOperator() operator.operand = variable if opcode == ins.ISTC.opcode \ or opcode == ins.IST.opcode \ or opcode == ins.NOT.opcode: operator.type = nodes.UnaryOperator.T_NOT elif opcode == ins.UNM.opcode: operator.type = nodes.UnaryOperator.T_MINUS else: assert opcode == ins.LEN.opcode operator.type = nodes.UnaryOperator.T_LENGTH_OPERATOR return operator def _build_slot(state, addr, slot): return _build_identifier(state, addr, slot, nodes.Identifier.T_LOCAL) def _build_upvalue(state, addr, slot): return _build_identifier(state, addr, slot, nodes.Identifier.T_UPVALUE) def _build_identifier(state, addr, slot, want_type): node = nodes.Identifier() setattr(node, "_addr", addr) node.slot = slot node.type = nodes.Identifier.T_SLOT if want_type == nodes.Identifier.T_UPVALUE: name = state.debuginfo.lookup_upvalue_name(slot) if name is not None: node.name = name node.type = want_type return node def _build_global_variable(state, addr, slot): node = nodes.TableElement() node.table = nodes.Identifier() node.table.type = nodes.Identifier.T_BUILTIN node.table.name = "_env" node.key = _build_string_constant(state, slot) return node def _build_string_constant(state, index): node = nodes.Constant() node.type = nodes.Constant.T_STRING node.value = state.constants.complex_constants[index] return node def _build_cdata_constant(state, index): node = nodes.Constant() node.type = nodes.Constant.T_CDATA node.value = state.constants.complex_constants[index] return node def _build_numeric_constant(state, index): number = state.constants.numeric_constants[index] node = nodes.Constant() node.value = number if isinstance(number, int): node.type = nodes.Constant.T_INTEGER else: node.type = nodes.Constant.T_FLOAT return node def _build_primitive(state, value): node = nodes.Primitive() if value is True or value == T_TRUE: node.type = nodes.Primitive.T_TRUE elif value is False or value == T_FALSE: node.type = nodes.Primitive.T_FALSE else: assert value is None or value == T_NIL node.type = nodes.Primitive.T_NIL return node def _build_literal(state, value): node = nodes.Constant() node.value = value node.type = nodes.Constant.T_INTEGER return node ================================================ FILE: ljd/ast/helpers.py ================================================ import ljd.ast.nodes as nodes import ljd.ast.traverse as traverse def insert_table_record(constructor, key, value): array = constructor.array.contents records = constructor.records.contents if isinstance(key, nodes.MULTRES): assert len(records) == 0 \ or isinstance(records[-1], nodes.TableRecord) records.append(value) return while isinstance(key, nodes.Constant) \ and key.type == key.T_INTEGER \ and key.value >= 0: index = key.value if index == 1 and len(array) == 0: record = nodes.ArrayRecord() record.value = nodes.Primitive() record.value.type = nodes.Primitive.T_NIL array.append(record) if (index > len(array)): break record = nodes.ArrayRecord() record.value = value if len(array) == 0 or index == len(array): array.append(record) else: array[index] = record return record = nodes.TableRecord() record.key = key record.value = value if len(records) == 0: records.append(record) return last = records[-1] if isinstance(last, (nodes.FunctionCall, nodes.Vararg)): records.insert(-1, record) else: records.append(record) def has_same_table(node, table): class Checker(traverse.Visitor): def __init__(self, table): self.found = False self.table = table def visit_table_element(self, node): if is_equal(self.table, node): self.found = True def _visit(self, node): if not self.found: traverse.Visitor._visit(self, node) def _visit_list(self, nodes_list): if not self.found: traverse.Visitor._visit_list(self, nodes_list) checker = Checker(table) traverse.traverse(checker, node) return checker.found def is_equal(a, b): if type(a) != type(b): return False if isinstance(a, nodes.Identifier): return a.type == b.type and a.slot == b.slot elif isinstance(a, nodes.TableElement): return is_equal(a.table, b.table) \ and is_equal(a.key, b.key) else: assert isinstance(a, nodes.Constant) return a.type == b.type and a.value == b.value ================================================ FILE: ljd/ast/locals.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # import ljd.ast.nodes as nodes import ljd.ast.traverse as traverse def mark_locals(ast): traverse.traverse(_LocalsMarker(), ast) def mark_local_definitions(ast): traverse.traverse(_LocalDefinitionsMarker(), ast) class _LocalsMarker(traverse.Visitor): class _State(): def __init__(self): self.pending_slots = {} self.debuginfo = None self.addr = -1 def __init__(self): self._states = [] # ## def _push_state(self): self._states.append(_LocalsMarker._State()) def _pop_state(self): self._states.pop() def _state(self): return self._states[-1] def _process_slots(self, addr): debuginfo = self._state().debuginfo cleanup = [] for slot, nodes in self._state().pending_slots.items(): varinfo = debuginfo.lookup_local_name(addr, slot) if varinfo is None: continue cleanup.append(slot) if varinfo.type == varinfo.T_INTERNAL: continue for node in nodes: node.name = varinfo.name node.type = node.T_LOCAL #zzw.20180712 setattr(node, "_varinfo", varinfo) for slot in cleanup: del self._state().pending_slots[slot] def _reset_slot(self, slot): self._state().pending_slots.pop(slot, None) def _reset_all(self, slots): for slot in slots: if isinstance(slot, nodes.Identifier): self._reset_slot(slot.slot) # ## def visit_function_definition(self, node): self._push_state() self._state().debuginfo = node._debuginfo def leave_function_definition(self, node): addr = node._instructions_count self._process_slots(addr) self._pop_state() # ## def visit_variables_list(self, node): # Last chance for a local = local + 1 type assignments self._process_slots(self._state().addr) self._reset_all(node.contents) def visit_identifiers_list(self, node): self._reset_all(node.contents) def visit_numeric_loop_warp(self, node): self._reset_slot(node.index.slot) def visit_identifier(self, node): if node.type == nodes.Identifier.T_SLOT: queue = self._state().pending_slots slots = queue.setdefault(node.slot, []) slots.append(node) # ## def _process_worthy_node(self, node): addr = getattr(node, "_addr", None) if not isinstance(node, nodes.Identifier) and addr is not None: assert self._state().addr <= addr self._state().addr = addr self._process_slots(addr) # We need to process slots twice as it could be the last # statement in the function/block and it could be anassignment # as well so we need to process slots before the reset def _leave_node(self, handler, node): traverse.Visitor._leave_node(self, handler, node) self._process_worthy_node(node) def _visit_node(self, handler, node): self._process_worthy_node(node) traverse.Visitor._visit_node(self, handler, node) class _LocalDefinitionsMarker(traverse.Visitor): class _State(): def __init__(self): self.known_locals = [None] * 255 self.addr = 0 def __init__(self): self._states = [] def _push_state(self): self._states.append(_LocalDefinitionsMarker._State()) def _pop_state(self): self._states.pop() def _state(self): return self._states[-1] def _update_known_locals(self, local, addr): varinfo = self._state().known_locals[local.slot] #zzw 20180712 self._state().known_locals[local.slot] = getattr(local, "_varinfo", None)#local._varinfo if varinfo is None: return False if varinfo.end_addr <= addr: return False return True # ## def visit_function_definition(self, node): self._push_state() for local in node.arguments.contents: if not isinstance(local, nodes.Vararg): self._update_known_locals(local, 1) def leave_function_definition(self, node): self._pop_state() def visit_iterator_for(self, node): addr = node._addr for local in node.identifiers.contents: if local.type == nodes.Identifier.T_LOCAL: self._update_known_locals(local, addr) def visit_numeric_for(self, node): addr = node._addr if node.variable.type == nodes.Identifier.T_LOCAL: self._update_known_locals(node.variable, addr) # ## def visit_assignment(self, node): dst = node.destinations.contents[0] addr = self._state().addr if not isinstance(dst, nodes.Identifier): return if dst.type != nodes.Identifier.T_LOCAL: return known_slot = self._update_known_locals(dst, addr) for slot in node.destinations.contents[1:]: if not isinstance(slot, nodes.Identifier): return if slot.type != nodes.Identifier.T_LOCAL: return also_known = self._update_known_locals(slot, addr) assert known_slot == also_known if not known_slot: node.type = nodes.Assignment.T_LOCAL_DEFINITION def _visit(self, node): node_addr = getattr(node, "_addr", -1) if node_addr >= 0: self._state().addr = node_addr traverse.Visitor._visit(self, node) ================================================ FILE: ljd/ast/mutator.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # import copy from ljd.ast.helpers import * class SimpleLoopWarpSwapper(traverse.Visitor): def visit_statements_list(self, node): blocks = node.contents fixed = [] index_shift = 0 for i, block in enumerate(node.contents): warp = block.warp fixed.append(block) block.index += index_shift if isinstance(warp, nodes.IteratorWarp): self._swap_iterator_warps(blocks, block) continue if isinstance(warp, nodes.NumericLoopWarp): self._swap_numeric_loop_warps(blocks, block) continue if isinstance(warp, nodes.UnconditionalWarp) \ and warp.is_uclo: assert block != node.contents[-1] next_block = node.contents[i + 1] self._fix_uclo_return(block, next_block) if not isinstance(warp, nodes.ConditionalWarp): continue if warp.true_target != warp.false_target: continue slot = getattr(warp, "_slot", -1) if slot < 0: continue next_index = block.index - index_shift + 1 assert block.warp.false_target.index == next_index new_block = self._create_dummy_block(block, slot) fixed.append(new_block) index_shift += 1 node.contents = fixed def _create_dummy_block(self, block, slot): new_block = nodes.Block() new_block.first_address = block.last_address new_block.last_address = new_block.first_address new_block.index = block.index + 1 new_block.warpins_count = 1 new_block.warp = nodes.UnconditionalWarp() new_block.warp.type = nodes.UnconditionalWarp.T_FLOW new_block.warp.target = block.warp.false_target statement = nodes.Assignment() identifier = nodes.Identifier() identifier.type = nodes.Identifier.T_SLOT identifier.slot = slot statement.destinations.contents.append(identifier) statement.expressions.contents.append(copy.copy(identifier)) new_block.contents.append(statement) block.warp.true_target = new_block return new_block def _fix_uclo_return(self, block, next_block): warp = block.warp target = warp.target if len(target.contents) != 1: return statement = target.contents[0] if not isinstance(statement, nodes.Return): return block.contents.append(statement) statement._addr = block.last_address target.contents = [] warp.type = nodes.UnconditionalWarp.T_FLOW warp.target = next_block def _swap_iterator_warps(self, blocks, end): warp = end.warp index = blocks.index(warp.body) assert index > 0 start = blocks[index - 1] assert isinstance(start.warp, nodes.UnconditionalWarp) assert start.warp.type == nodes.UnconditionalWarp.T_JUMP assert start.warp.target == end end_addr = end.warp._addr start_addr = start.warp._addr new_end_warp = start.warp new_end_warp._addr = end_addr new_start_warp = end.warp new_start_warp._addr = start_addr end.warp = new_end_warp start.warp = new_start_warp new_end_warp.target = start def _swap_numeric_loop_warps(self, blocks, end): warp = end.warp index = blocks.index(warp.body) assert index > 0 start = blocks[index - 1] assert isinstance(start.warp, nodes.UnconditionalWarp) assert start.warp.type == nodes.UnconditionalWarp.T_FLOW assert start.warp.target == warp.body end_addr = end.warp._addr start_addr = start.warp._addr new_end_warp = start.warp new_end_warp._addr = end_addr new_start_warp = end.warp new_start_warp._addr = start_addr end.warp = new_end_warp start.warp = new_start_warp new_end_warp.type = nodes.UnconditionalWarp.T_JUMP new_end_warp.target = start class MutatorVisitor(traverse.Visitor): # ## def leave_if(self, node): if len(node.else_block.contents) != 1: return subif = node.else_block.contents[0] if not isinstance(subif, nodes.If): return elseif = nodes.ElseIf() elseif.expression = subif.expression elseif.then_block = subif.then_block node.elseifs.append(elseif) node.elseifs += subif.elseifs node.else_block = subif.else_block def visit_statements_list(self, node): patched = [] i = -1 while i < len(node.contents) - 1: i += 1 statement = node.contents[i] patched.append(statement) if not isinstance(statement, nodes.Assignment): continue src = statement.expressions.contents[0] if not isinstance(src, nodes.TableConstructor): continue assert len(statement.destinations.contents) == 1 dst = statement.destinations.contents[0] i += self._fill_constructor(dst, src, node.contents[i + 1:]) node.contents = patched def _fill_constructor(self, table, constructor, statements): consumed = 0 for statement in statements: if not isinstance(statement, nodes.Assignment): break if len(statement.destinations.contents) > 1: break dst = statement.destinations.contents[0] if not isinstance(dst, nodes.TableElement): break if not is_equal(dst.table, table): break assert len(statement.expressions.contents) == 1 src = statement.expressions.contents[0] if has_same_table(src, table): break insert_table_record(constructor, dst.key, src) consumed += 1 return consumed def pre_pass(ast): traverse.traverse(SimpleLoopWarpSwapper(), ast) return ast def primary_pass(ast): traverse.traverse(MutatorVisitor(), ast) return ast ================================================ FILE: ljd/ast/nodes.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # # We should visit stuff in it's execution order. That's important class FunctionDefinition(): def __init__(self): self.arguments = IdentifiersList() self.statements = StatementsList() self._upvalues = None self._debuginfo = None self._instructions_count = 0 def _accept(self, visitor): visitor._visit_node(visitor.visit_function_definition, self) visitor._visit(self.arguments) visitor._visit(self.statements) visitor._leave_node(visitor.leave_function_definition, self) class TableConstructor(): def __init__(self): self.array = RecordsList() self.records = RecordsList() def _accept(self, visitor): visitor._visit_node(visitor.visit_table_constructor, self) visitor._visit(self.array) visitor._visit(self.records) visitor._leave_node(visitor.leave_table_constructor, self) class ArrayRecord(): def __init__(self): self.value = None def _accept(self, visitor): visitor._visit_node(visitor.visit_array_record, self) visitor._visit(self.value) visitor._leave_node(visitor.leave_array_record, self) class TableRecord(): def __init__(self): self.key = None self.value = None def _accept(self, visitor): visitor._visit_node(visitor.visit_table_record, self) visitor._visit(self.key) visitor._visit(self.value) visitor._leave_node(visitor.leave_table_record, self) class Assignment(): T_LOCAL_DEFINITION = 0 T_NORMAL = 1 def __init__(self): self.expressions = ExpressionsList() self.destinations = VariablesList() self.type = -1 def _accept(self, visitor): visitor._visit_node(visitor.visit_assignment, self) visitor._visit(self.expressions) visitor._visit(self.destinations) visitor._leave_node(visitor.leave_assignment, self) class BinaryOperator(): T_LOGICAL_OR = 0 # left or right T_LOGICAL_AND = 10 # left and right T_LESS_THEN = 20 # left < right T_GREATER_THEN = 21 # left > right T_LESS_OR_EQUAL = 22 # left <= right T_GREATER_OR_EQUAL = 23 # left >= right T_NOT_EQUAL = 24 # left ~= right T_EQUAL = 25 # left == right T_CONCAT = 30 # left .. right T_ADD = 40 # left + right T_SUBTRACT = 41 # left - right T_MULTIPLY = 50 # left * right T_DIVISION = 51 # left / right T_MOD = 52 # left % right T_POW = 70 # left ^ right def __init__(self): self.type = -1 self.left = None self.right = None def _accept(self, visitor): visitor._visit_node(visitor.visit_binary_operator, self) visitor._visit(self.left) visitor._visit(self.right) visitor._leave_node(visitor.leave_binary_operator, self) class UnaryOperator(): T_NOT = 60 # not operand T_LENGTH_OPERATOR = 61 # #operand T_MINUS = 62 # -operand def __init__(self): self.type = -1 self.operand = None def _accept(self, visitor): visitor._visit_node(visitor.visit_unary_operator, self) visitor._visit(self.operand) visitor._leave_node(visitor.leave_unary_operator, self) class StatementsList(): def __init__(self): self.contents = [] def _accept(self, visitor): visitor._visit_node(visitor.visit_statements_list, self) visitor._visit_list(self.contents) visitor._leave_node(visitor.leave_statements_list, self) class IdentifiersList(): def __init__(self): self.contents = [] def _accept(self, visitor): visitor._visit_node(visitor.visit_identifiers_list, self) visitor._visit_list(self.contents) visitor._leave_node(visitor.leave_identifiers_list, self) class RecordsList(): def __init__(self): self.contents = [] def _accept(self, visitor): visitor._visit_node(visitor.visit_records_list, self) visitor._visit_list(self.contents) visitor._leave_node(visitor.leave_records_list, self) class VariablesList(): def __init__(self): self.contents = [] def _accept(self, visitor): visitor._visit_node(visitor.visit_variables_list, self) visitor._visit_list(self.contents) visitor._leave_node(visitor.leave_variables_list, self) class ExpressionsList(): def __init__(self): self.contents = [] def _accept(self, visitor): visitor._visit_node(visitor.visit_expressions_list, self) visitor._visit_list(self.contents) visitor._leave_node(visitor.leave_expressions_list, self) # Called Name in the Lua 5.1 reference class Identifier(): T_SLOT = 0 T_LOCAL = 1 T_UPVALUE = 2 T_BUILTIN = 3 def __init__(self): self.name = None self.type = -1 self.slot = -1 def _accept(self, visitor): visitor._visit_node(visitor.visit_identifier, self) visitor._leave_node(visitor.leave_identifier, self) # helper vararg/varreturn class MULTRES(): def _accept(self, visitor): visitor._visit_node(visitor.visit_multres, self) visitor._leave_node(visitor.leave_multres, self) class TableElement(): def __init__(self): self.table = None self.key = None def _accept(self, visitor): visitor._visit_node(visitor.visit_table_element, self) visitor._visit(self.key) visitor._visit(self.table) visitor._leave_node(visitor.leave_table_element, self) class Vararg(): def _accept(self, visitor): visitor._visit_node(visitor.visit_vararg, self) visitor._leave_node(visitor.leave_vararg, self) class FunctionCall(): def __init__(self): self.function = None self.arguments = ExpressionsList() def _accept(self, visitor): visitor._visit_node(visitor.visit_function_call, self) visitor._visit(self.arguments) visitor._visit(self.function) visitor._leave_node(visitor.leave_function_call, self) class If(): def __init__(self): self.expression = None self.then_block = StatementsList() self.elseifs = [] self.else_block = StatementsList() def _accept(self, visitor): visitor._visit_node(visitor.visit_if, self) visitor._visit(self.expression) visitor._visit(self.then_block) visitor._visit_list(self.elseifs) visitor._visit(self.else_block) visitor._leave_node(visitor.leave_if, self) class ElseIf(): def __init__(self): self.expression = None self.then_block = StatementsList() def _accept(self, visitor): visitor._visit_node(visitor.visit_elseif, self) visitor._visit(self.expression) visitor._visit(self.then_block) visitor._leave_node(visitor.leave_elseif, self) # ## class Block(): def __init__(self): self.index = -1 self.warp = None self.contents = [] self.first_address = 0 self.last_address = 0 self.warpins_count = 0 def _accept(self, visitor): visitor._visit_node(visitor.visit_block, self) visitor._visit_list(self.contents) visitor._visit(self.warp) visitor._leave_node(visitor.leave_block, self) class UnconditionalWarp(): T_JUMP = 0 T_FLOW = 1 def __init__(self): self.type = -1 self.target = None self.is_uclo = False def _accept(self, visitor): visitor._visit_node(visitor.visit_unconditional_warp, self) # DO NOT VISIT self.target - warps are not part of the tree visitor._leave_node(visitor.leave_unconditional_warp, self) class ConditionalWarp(): def __init__(self): self.condition = None self.true_target = None self.false_target = None def _accept(self, visitor): visitor._visit_node(visitor.visit_conditional_warp, self) visitor._visit(self.condition) # DO NOT VISIT self.true_target - warps are not part of the tree # DO NOT VISIT self.false_target - warps are not part of the tree visitor._leave_node(visitor.leave_conditional_warp, self) class IteratorWarp(): def __init__(self): self.variables = VariablesList() self.controls = ExpressionsList() self.body = None self.way_out = None def _accept(self, visitor): visitor._visit_node(visitor.visit_iterator_warp, self) visitor._visit(self.variables) visitor._visit(self.controls) # DO NOT VISIT self.body - warps are not part of the tree # DO NOT VISIT self.way_out - warps are not part of the tree visitor._leave_node(visitor.leave_iterator_warp, self) class NumericLoopWarp(): def __init__(self): self.index = Identifier() self.controls = ExpressionsList() self.body = None self.way_out = None def _accept(self, visitor): visitor._visit_node(visitor.visit_numeric_loop_warp, self) visitor._visit(self.index) visitor._visit(self.controls) # DO NOT VISIT self.body - warps are not part of the tree # DO NOT VISIT self.way_out - warps are not part of the tree visitor._leave_node(visitor.leave_numeric_loop_warp, self) class EndWarp(): def _accept(self, visitor): visitor._visit_node(visitor.visit_end_warp, self) visitor._leave_node(visitor.leave_end_warp, self) # ## class Return(): def __init__(self): self.returns = ExpressionsList() def _accept(self, visitor): visitor._visit_node(visitor.visit_return, self) visitor._visit(self.returns) visitor._leave_node(visitor.leave_return, self) class Break(): def _accept(self, visitor): visitor._visit_node(visitor.visit_break, self) visitor._leave_node(visitor.leave_break, self) class While(): def __init__(self): self.expression = None self.statements = StatementsList() def _accept(self, visitor): visitor._visit_node(visitor.visit_while, self) visitor._visit(self.expression) visitor._visit(self.statements) visitor._leave_node(visitor.leave_while, self) class RepeatUntil(): def __init__(self): self.expression = None self.statements = StatementsList() def _accept(self, visitor): visitor._visit_node(visitor.visit_repeat_until, self) visitor._visit(self.statements) visitor._visit(self.expression) visitor._leave_node(visitor.leave_repeat_until, self) class NumericFor(): def __init__(self): self.variable = None self.expressions = ExpressionsList() self.statements = StatementsList() def _accept(self, visitor): visitor._visit_node(visitor.visit_numeric_for, self) visitor._visit(self.variable) visitor._visit(self.expressions) visitor._visit(self.statements) visitor._leave_node(visitor.leave_numeric_for, self) class IteratorFor(): def __init__(self): self.expressions = ExpressionsList() self.identifiers = VariablesList() self.statements = StatementsList() def _accept(self, visitor): visitor._visit_node(visitor.visit_iterator_for, self) visitor._visit(self.expressions) visitor._visit(self.identifiers) visitor._visit(self.statements) visitor._leave_node(visitor.leave_iterator_for, self) class Constant(): T_INTEGER = 0 T_FLOAT = 1 T_STRING = 2 T_CDATA = 3 def __init__(self): self.type = -1 self.value = None def _accept(self, visitor): visitor._visit_node(visitor.visit_constant, self) visitor._leave_node(visitor.leave_constant, self) class Primitive(): T_NIL = 0 T_TRUE = 1 T_FALSE = 2 def __init__(self): self.type = -1 def _accept(self, visitor): visitor._visit_node(visitor.visit_primitive, self) visitor._leave_node(visitor.leave_primitive, self) ================================================ FILE: ljd/ast/slotworks.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # import ljd.ast.nodes as nodes import ljd.ast.traverse as traverse from ljd.ast.helpers import insert_table_record def eliminate_temporary(ast): _eliminate_multres(ast) slots, unused = _collect_slots(ast) _eliminate_temporary(slots) # _remove_unused(unused) _cleanup_invalid_nodes(ast) return ast def _eliminate_temporary(slots): simple = [] massive = [] tables = [] iterators = [] for info in slots: assignment = info.assignment if not isinstance(assignment, nodes.Assignment): assert isinstance(assignment, (nodes.IteratorWarp, nodes.NumericLoopWarp, nodes.FunctionDefinition)) src = info.references[1].identifier simple.append((info.references, src)) continue assert len(assignment.expressions.contents) == 1 is_massive = len(assignment.destinations.contents) > 1 if is_massive: _fill_massive_refs(info, simple, massive, iterators) else: _fill_simple_refs(info, simple, tables) _eliminate_simple_cases(simple) _eliminate_into_table_constructors(tables) _eliminate_mass_assignments(massive) _eliminate_iterators(iterators) def _fill_massive_refs(info, simple, massive, iterators): ref = info.references[1] holder = _get_holder(ref.path) src = info.assignment.expressions.contents[0] assert isinstance(src, (nodes.FunctionCall, nodes.Vararg, nodes.Primitive)) if isinstance(holder, nodes.Assignment): dst = holder.destinations.contents[0] assert len(info.references) == 2 orig = info.references[0].identifier assignment = ref.path[-3] assert isinstance(assignment, nodes.Assignment) massive.append((orig, info.assignment, assignment, dst)) elif isinstance(holder, nodes.IteratorWarp): assert len(info.references) == 2 iterators.append((info.assignment, src, holder)) elif isinstance(src, nodes.Primitive) and src.type == src.T_NIL: assert len(info.references) == 2 # Create a new primitive, so it won't mess with the # writer's ignore list src = nodes.Primitive() src.type = nodes.Primitive.T_NIL simple.append((info, ref, src)) def _fill_simple_refs(info, simple, tables): src = info.assignment.expressions.contents[0] if isinstance(src, nodes.FunctionCall) and len(info.references) > 3: return src_is_table = isinstance(src, nodes.TableConstructor) for ref in info.references[1:]: holder = _get_holder(ref.path) is_element = isinstance(holder, nodes.TableElement) path_index = ref.path.index(holder) statement = _get_holder(ref.path[:path_index]) statement_is_assignment = isinstance(statement, nodes.Assignment) if statement_is_assignment: is_dst = statement.destinations.contents[0] == holder else: is_dst = False # Could be more then one reference here if src_is_table and is_element and is_dst: assert holder.table == ref.identifier tables.append((info, ref)) else: simple.append((info, ref, None)) LIST_TYPES = (nodes.VariablesList, nodes.IdentifiersList, nodes.ExpressionsList, nodes.StatementsList) def _get_holder(path): for node in reversed(path[:-1]): if not isinstance(node, LIST_TYPES): return node return None def _eliminate_simple_cases(simple): for info, ref, src in simple: holder = ref.path[-2] dst = ref.identifier if src is None: src = info.assignment.expressions.contents[0] _mark_invalidated(info.assignment) if isinstance(holder, LIST_TYPES): nodes = holder.contents found = _replace_node_in_list(nodes, dst, src) else: found = _replace_node(holder, dst, src) if found == None: print("err: slotworks.py assert found") def _eliminate_into_table_constructors(tables): for info, ref in tables: constructor = info.assignment.expressions.contents[0] table_element = ref.path[-2] assignment = ref.path[-4] assert isinstance(assignment, nodes.Assignment) assert len(assignment.expressions.contents) == 1 _mark_invalidated(assignment) key = table_element.key value = assignment.expressions.contents[0] insert_table_record(constructor, key, value) def _eliminate_mass_assignments(massive): for identifier, assignment, base_assignment, globalvar in massive: destinations = assignment.destinations.contents found = _replace_node_in_list(destinations, identifier, globalvar) _mark_invalidated(base_assignment) assert found def _replace_node(holder, original, replacement): for key, value in holder.__dict__.items(): if value == original: setattr(holder, key, replacement) return True return False def _replace_node_in_list(nodes, original, replacement): try: index = nodes.index(original) except ValueError: return False nodes[index] = replacement return True def _eliminate_iterators(iterators): processed_warps = set() for assignment, src, warp in iterators: if warp in processed_warps: continue for i, slot in enumerate(assignment.destinations.contents): if hasattr(warp.controls.contents[i], 'slot') and hasattr(slot, 'slot'): #assert warp.controls.contents[i].slot == slot.slot if warp.controls.contents[i].slot != slot.slot: print ("err: slotworks.py assert warp.controls.contents[i].slot == slot.slot") warp.controls.contents = [src] processed_warps.add(warp) _mark_invalidated(assignment) def _mark_invalidated(node): setattr(node, "_invalidated", True) def _is_invalidated(node): return getattr(node, "_invalidated", False) def _remove_unused(unused): pass def _collect_slots(ast): collector = _SlotsCollector() traverse.traverse(collector, ast) return collector.slots, collector.unused def _eliminate_multres(ast): traverse.traverse(_MultresEliminator(), ast) _cleanup_invalid_nodes(ast) class _MultresEliminator(traverse.Visitor): def __init__(self): self._last_multres_value = None def leave_assignment(self, node): src = node.expressions.contents[0] dst = node.destinations.contents[0] if isinstance(dst, nodes.MULTRES): assert len(node.destinations.contents) == 1 assert len(node.expressions.contents) == 1 assert isinstance(src, (nodes.FunctionCall, nodes.Vararg)) assert self._last_multres_value is None self._last_multres_value = src _mark_invalidated(node) else: for i, src in enumerate(node.expressions.contents): if isinstance(src, nodes.MULTRES): break else: return assert self._last_multres_value is not None node.expressions.contents[i] = self._last_multres_value self._last_multres_value = None def _process_multres_in_list(self, nodes_list): for i, node in enumerate(nodes_list): if isinstance(node, nodes.MULTRES): break else: return assert self._last_multres_value is not None nodes_list[i] = self._last_multres_value self._last_multres_value = None def visit_function_call(self, node): self._process_multres_in_list(node.arguments.contents) def visit_return(self, node): self._process_multres_in_list(node.returns.contents) class _SlotReference(): def __init__(self): self.path = [] self.identifier = None class _SlotInfo(): def __init__(self): self.slot = 0 self.assignment = None self.references = [] self.termination = None self.function = None class _SlotsCollector(traverse.Visitor): class _State(): def __init__(self): self.known_slots = {} self.function = None # ## def __init__(self): self._states = [] self._path = [] self._skip = None self.slots = [] self.unused = [] self._push_state() # ## def _state(self): return self._states[-1] def _push_state(self): self._states.append(_SlotsCollector._State()) def _pop_state(self): self._states.pop() def _commit_info(self, info): assert len(info.references) > 0 if len(info.references) == 1: self.unused.append(info) else: self.slots.append(info) def _commit_slot(self, slot, node): info = self._state().known_slots.get(slot) if info is None: return info.termination = node del self._state().known_slots[slot] self._commit_info(info) def _register_slot(self, slot, node): self._commit_slot(slot, node) info = _SlotInfo() info.slot = slot info.assignment = node info.function = self._state().function self._state().known_slots[slot] = info def _register_all_slots(self, node, slots): for slot in slots: if not isinstance(slot, nodes.Identifier): continue if slot.type != nodes.Identifier.T_SLOT: continue self._register_slot(slot.slot, node) def _commit_all_slots(self, slots, node): for slot in slots: if not isinstance(slot, nodes.Identifier): continue self._commit_slot(slot.slot, node) def _register_slot_reference(self, slot, node): info = self._state().known_slots.get(slot) if info is None: return reference = _SlotReference() reference.identifier = node # Copy the list, but not contents reference.path = self._path[:] info.references.append(reference) # ## def visit_assignment(self, node): self._visit(node.expressions) self._skip = node.expressions self._register_all_slots(node, node.destinations.contents) def leave_assignment(self, node): self._skip = None def visit_identifier(self, node): if node.type == nodes.Identifier.T_SLOT: self._register_slot_reference(node.slot, node) # ## def visit_function_definition(self, node): self._push_state() self._state().function = node def leave_function_definition(self, node): self._pop_state() def leave_block(self, node): for info in self._state().known_slots.values(): self._commit_info(info) self._state().known_slots = {} def visit_iterator_warp(self, node): self._commit_all_slots(node.variables.contents, node) def visit_numeric_loop_warp(self, node): self._commit_slot(node.index.slot, node) # ## def _visit_node(self, handler, node): self._path.append(node) traverse.Visitor._visit_node(self, handler, node) def _leave_node(self, handler, node): self._path.pop() traverse.Visitor._leave_node(self, handler, node) def _visit(self, node): if self._skip == node: return traverse.Visitor._visit(self, node) def _cleanup_invalid_nodes(ast): traverse.traverse(_TreeCleanup(), ast) class _TreeCleanup(traverse.Visitor): def visit_block(self, node): patched = [] for subnode in node.contents: if not _is_invalidated(subnode): patched.append(subnode) node.contents = patched ================================================ FILE: ljd/ast/traverse.py ================================================ class Visitor(): def __init__(self): pass # ## def visit_function_definition(self, node): pass def leave_function_definition(self, node): pass # ## def visit_table_constructor(self, node): pass def leave_table_constructor(self, node): pass def visit_table_record(self, node): pass def leave_table_record(self, node): pass def visit_array_record(self, node): pass def leave_array_record(self, node): pass # ## def visit_assignment(self, node): pass def leave_assignment(self, node): pass # ## def visit_binary_operator(self, node): pass def leave_binary_operator(self, node): pass def visit_unary_operator(self, node): pass def leave_unary_operator(self, node): pass # ## def visit_statements_list(self, node): pass def leave_statements_list(self, node): pass def visit_identifiers_list(self, node): pass def leave_identifiers_list(self, node): pass def visit_records_list(self, node): pass def leave_records_list(self, node): pass def visit_variables_list(self, node): pass def leave_variables_list(self, node): pass def visit_expressions_list(self, node): pass def leave_expressions_list(self, node): pass # ## def visit_identifier(self, node): pass def leave_identifier(self, node): pass def visit_multres(self, node): pass def leave_multres(self, node): pass def visit_table_element(self, node): pass def leave_table_element(self, node): pass def visit_vararg(self, node): pass def leave_vararg(self, node): pass def visit_function_call(self, node): pass def leave_function_call(self, node): pass # ## def visit_if(self, node): pass def leave_if(self, node): pass def visit_elseif(self, node): pass def leave_elseif(self, node): pass # ## def visit_block(self, node): pass def leave_block(self, node): pass def visit_unconditional_warp(self, node): pass def leave_unconditional_warp(self, node): pass def visit_conditional_warp(self, node): pass def leave_conditional_warp(self, node): pass def visit_iterator_warp(self, node): pass def leave_iterator_warp(self, node): pass def visit_numeric_loop_warp(self, node): pass def leave_numeric_loop_warp(self, node): pass def visit_end_warp(self, node): pass def leave_end_warp(self, node): pass # ## def visit_return(self, node): pass def leave_return(self, node): pass def visit_break(self, node): pass def leave_break(self, node): pass # ## def visit_while(self, node): pass def leave_while(self, node): pass def visit_repeat_until(self, node): pass def leave_repeat_until(self, node): pass def visit_numeric_for(self, node): pass def leave_numeric_for(self, node): pass def visit_iterator_for(self, node): pass def leave_iterator_for(self, node): pass # ## def visit_constant(self, node): pass def leave_constant(self, node): pass def visit_primitive(self, node): pass def leave_primitive(self, node): pass # ## def _visit_node(self, handler, node): handler(node) def _leave_node(self, handler, node): handler(node) # ## def _visit(self, node): assert node is not None node._accept(self) def _visit_list(self, nodes_list): assert isinstance(nodes_list, list) for node in nodes_list: self._visit(node) def traverse(visitor, node): if isinstance(node, list): visitor._visit_list(node) else: visitor._visit(node) ================================================ FILE: ljd/ast/unwarper.py ================================================ import copy import ljd.ast.nodes as nodes import ljd.ast.traverse as traverse import ljd.ast.slotworks as slotworks binop = nodes.BinaryOperator # ## # ## REMEMBER # ## # ## Block indices are unreliable while you are mangling them! # ## # ## P.S. Probably they should not be named indices... But they ARE used as indices during other phases. Sometimes. # ## # ## class _StatementsCollector(traverse.Visitor): def __init__(self): self.result = [] def visit_statements_list(self, node): if len(node.contents) > 0: self.result.append(node) def unwarp(node): # There could be many negative jumps within while conditions, so # filter them first _run_step(_unwarp_loops, node, repeat_until=False) _run_step(_unwarp_loops, node, repeat_until=True) _run_step(_unwarp_expressions, node) _run_step(_unwarp_ifs, node) _glue_flows(node) def _run_step(step, node, **kargs): for statements in _gather_statements_lists(node): statements.contents = step(statements.contents, **kargs) # Fix block indices in case anything was moved for statements in _gather_statements_lists(node): for i, block in enumerate(statements.contents): block.index = i def _gather_statements_lists(node): collector = _StatementsCollector() traverse.traverse(collector, node) return collector.result def _glue_flows(node): for statements in _gather_statements_lists(node): blocks = statements.contents if hasattr(blocks[-1], 'warp'): assert isinstance(blocks[-1].warp, nodes.EndWarp) for i, block in enumerate(blocks[:-1]): warp = block.warp assert _is_flow(warp) target = warp.target assert target == blocks[i + 1] target.contents = block.contents + target.contents block.contents = [] statements.contents = blocks[-1].contents # ## # ## IFs AND EXPRESSIONs PROCESSING # ## def _unwarp_expressions(blocks): pack = [] pack_set = set() start_index = 0 while start_index < len(blocks) - 1: start = blocks[start_index] warp = start.warp if isinstance(warp, nodes.UnconditionalWarp): if warp.type == nodes.UnconditionalWarp.T_FLOW: start_index += 1 continue body, end, end_index = _extract_if_body(start_index, blocks, None) if body is None: raise NotImplementedError("GOTO statements are not" " supported") expressions = _find_expressions(start, body, end) assert pack_set.isdisjoint(expressions) expressions_set = set(expressions) assert len(expressions_set) == len(expressions) if len(expressions) == 0: start_index += 1 continue pack += list(reversed(expressions)) pack_set |= expressions_set endest_end = _find_endest_end(expressions) if endest_end != end: end_index = blocks.index(endest_end) start_index = end_index return _unwarp_expressions_pack(blocks, pack) def _find_endest_end(expressions): endest_end = expressions[0][1] for _start, exp_end, _slot, _slot_type in expressions[1:]: if exp_end.index > endest_end.index: endest_end = exp_end return endest_end def _unwarp_ifs(blocks, top_end=None, topmost_end=None): boundaries = [] start_index = 0 while start_index < len(blocks) - 1: start = blocks[start_index] warp = start.warp if isinstance(warp, nodes.UnconditionalWarp): if warp.type == nodes.UnconditionalWarp.T_FLOW: start_index += 1 continue body, end, end_index = _extract_if_body(start_index, blocks, topmost_end) if body is None: raise NotImplementedError("GOTO statements are not" " supported") is_end = isinstance(body[-1].warp, nodes.EndWarp) _unwarp_if_statement(start, body, end, end) if is_end: _set_end(start) else: _set_flow_to(start, end) boundaries.append((start_index, end_index - 1)) start_index = end_index return _remove_processed_blocks(blocks, boundaries) def _extract_if_body(start_index, blocks, topmost_end): end = _find_branching_end(blocks[start_index:], topmost_end) try: end_index = blocks.index(end) except ValueError: if end == topmost_end: end_index = len(blocks) else: return None, None, None body = blocks[start_index + 1:end_index] return body, end, end_index def _unwarp_expressions_pack(blocks, pack): replacements = {} for start, end, slot, slot_type in reversed(pack): end = replacements.get(end, end) start_index = blocks.index(start) end_index = blocks.index(end) body = blocks[start_index + 1:end_index] _unwarp_logical_expression(start, end, body) statements = start.contents + end.contents if slot_type == nodes.Identifier.T_SLOT: min_i = len(start.contents) split_i = _split_by_slot_use(statements, min_i, end.warp, slot) else: split_i = len(start.contents) max_i = len(statements) if split_i > max_i: end.contents = start.contents + end.contents start.contents = [] blocks = blocks[:start_index] + blocks[end_index:] _replace_targets(blocks, start, end) replacements[start] = end slotworks.eliminate_temporary(end) _set_flow_to(start, end) else: blocks = blocks[:start_index + 1] + blocks[end_index:] start.contents = statements[:split_i] end.contents = statements[split_i:] # We need to kill the start's warp before slot # elimination or it could result in a cycled AST. _set_flow_to(start, end) slotworks.eliminate_temporary(start) return blocks def _split_by_slot_use(statements, min_i, warp, slot): known_slots = set([slot]) split_i = min_i for i, statement in enumerate(statements): if isinstance(statement, nodes.Assignment): sets = _extract_destination_slots(statement) if i < min_i: known_slots |= sets else: known_slots -= sets if len(known_slots) == 0: break split_i = i + 1 if split_i < len(statements): return split_i if isinstance(warp, nodes.ConditionalWarp): known_slots -= _gather_slots(warp) if len(known_slots) == 0: split_i += 1 return split_i def _extract_destination_slots(statement): sets = set() for node in statement.destinations.contents: if not isinstance(node, nodes.Identifier): continue if node.type == nodes.Identifier.T_SLOT: sets.add(node.slot) return sets def _gather_slots(node): class Collector(traverse.Visitor): def __init__(self): self.slots = set() def visit_identifier(self, node): if node.type == nodes.Identifier.T_SLOT: self.slots.add(node.slot) collector = Collector() traverse.traverse(collector, node) return collector.slots def _find_expressions(start, body, end): # Explicitly allow the local a = x ~= "b" case slot, slot_type = _get_simple_local_assignment_slot(start, body, end) if slot >= 0: return [(start, end, slot, slot_type)] expressions = [] # We have something at the end, but not the true/false? i = 0 extbody = [start] + body is_local = False sure_expression = False while i < len(extbody): block = extbody[i] subs = _find_subexpressions(block, body[i:]) if len(subs) != 0: endest_end = _find_endest_end(subs) new_i = extbody.index(endest_end) # Loop? No way! if new_i <= i: return expressions # It should end with a conditional warp if that's # really a subexpression-as-operand end_warp = endest_end.warp if not isinstance(end_warp, nodes.ConditionalWarp): return expressions expressions = subs + expressions i = new_i continue if isinstance(block.warp, nodes.ConditionalWarp): condition = block.warp.condition is_end = block.warp.false_target == end is_binop = isinstance(condition, nodes.BinaryOperator) block_slot = getattr(block.warp, "_slot", slot) if is_end: if is_binop: return expressions elif slot < 0 and block_slot >= 0: slot = block_slot slot_type = nodes.Identifier.T_SLOT sure_expression = True elif slot != block_slot: return expressions else: sure_expression = True elif isinstance(block.warp, nodes.UnconditionalWarp): if block == start and len(block.contents) == 0: return [] if len(block.contents) == 0: i += 1 continue if block != start and len(block.contents) > 1: return expressions assignment = block.contents[-1] if not isinstance(assignment, nodes.Assignment): if block == start: i += 1 continue return expressions destinations = assignment.destinations.contents if len(destinations) != 1: if block == start: i += 1 continue return expressions dst = destinations[0] if not isinstance(dst, nodes.Identifier): if block == start: i += 1 continue return expressions if isinstance(block.warp, nodes.ConditionalWarp): if block == start: i += 1 continue return expressions if slot < 0: # If all encounters are locals, which means # that the first encounter is a local if dst.type == nodes.Identifier.T_LOCAL: is_local = True elif dst.type == nodes.Identifier.T_UPVALUE: return [] slot = dst.slot slot_type = dst.type elif slot == dst.slot: slot_type = dst.type if dst.type == nodes.Identifier.T_UPVALUE: return [] else: assert block != start return [] i += 1 if slot < 0: return [] true, _false, body = _get_terminators(body) if true is not None: sure_expression = True if len(expressions) > 0: sure_expression = True if not sure_expression and is_local: return expressions return expressions + [(start, end, slot, slot_type)] def _find_subexpressions(start, body): try: body, end, _end_index = _extract_if_body(0, [start] + body, None) except ValueError: # a warp target is not in a list return [] if body is None: return [] return _find_expressions(start, body, end) def _get_simple_local_assignment_slot(start, body, end): if len(body) != 2: return -1, None true, _false, body = _get_terminators(body) if true is None: return -1, None else: slot = true.contents[0].destinations.contents[0] return slot.slot, slot.type def _find_expression_slot(body): slot = None for block in reversed(body): if len(block.contents) != 1: continue slot = block.contents[0].destinations.contents[0] break return slot def _unwarp_logical_expression(start, end, body): slot = _find_expression_slot(body) assert slot is not None true, false, body = _get_terminators(body) expression = _compile_expression([start] + body, end, true, false) dst = copy.deepcopy(slot) assignment = nodes.Assignment() assignment.destinations.contents.append(dst) assignment.expressions.contents.append(expression) start.contents.append(assignment) def _compile_expression(body, end, true, false): parts = _unwarp_expression(body, end, true, false) if len(parts) < 3: assert len(parts) == 1 return parts[0] explicit_parts = _make_explicit_subexpressions(parts) return _assemble_expression(explicit_parts) # # The logical expressions: # # There are terminators: a true, a false and an end # # For an if case the true will be a "then" clause and the false - an "else" or # "after-the-if" clause. The end is required for a single-variable (unary) # components and is only used at during topmost phase. # # The last block in expression is always "double terminal" - both ends are # pointing at terminators. It's rather useless so we just append it's condition # (inverted if needed - it's easy to see if true end targets the false # terminator) at the end of processing. # # Then we need to pack all other blocks into subexpressions. Subexpressions # always end with a _terminal block_, i.e. the block which warp points to a # terminator. Idea is that we can guess the operator only right of a # terminal block, because we can check if the block's warp condition is # inverted or not. # # If that's an "OR" clause then it will jump out of the current expression if # the condition is true, so the condition is inverted and the false branch is # pointing at the way out (at the TRUE terminator - because the result of the # expression level will be true). (because in the bytecode there # is only one JMP, so a ConditionalWarp's true branch is actually a fake and # always points to the next block - in the bytecode a "positive" jump # will be represented by a normal negative jump with inverted condition). # # Otherwise, if that's an "AND" clause then it will jump out of the current # expression level if a condition is false, so the condition is not inverted # and a false branch points to the false. # # This way we can understand which operators go just right of terminal blocks. # Everything in-between these block is considered a subexpression. And just # because we don't know where exactly the subexpression ends we are using # greedy approach and trying to pack into a subexpression as much blocks as # possible, including any terminal blocks pointing at the same terminator # with the same inversion status (that is - we are always using the # rightmost block if there are consequitive similar terminal blocks, ignoring # all the blocks to the left). # # Then comes the trick: the subexpression is a component of this expression and # we know the operator to the right of it. We can guess now what will # be evaluated if the subexpression evaluates to "false" and what - if it's # "true". If the operator is "AND" then the subexpression failure will cause # the expression failure too, i.e. the "FALSE" target remains the same and the # true terminator is set to the next block (after the "AND"). # # If the operator is "OR" the the subexpression success will cause the success # of the expression, so the "TRUE" target remains the same, but the false # target is set to the next block (after the "OR"). # # Now we have a subexpression and both TRUE and FALSE terminators for it. # Recurse and repeat. # def _unwarp_expression(body, end, true, false): parts = [] if true is not None: terminator_index = min(true.index, false.index) if end is not None: terminator_index = min(end.index, terminator_index) else: assert end is not None terminator_index = end.index terminators = set((true, false, end)) subexpression_start = 0 i = 0 while i < len(body) - 1: block = body[i] warp = block.warp target = _get_target(warp) # # A chance for # (foo and (bar and y or z)) or x # type expressions, because the first "foo and ... )) or" part # will be broken by the "or z))" part in the code below. # # So we are going to intercept subexpressions by it's start # instead of an end, but only if we are already at the # subexpression start (so nothing formally changes, but creates # a bit more correct execution order) # if target.index < terminator_index: if i != subexpression_start: i += 1 continue target_index = body.index(target) last_block = body[target_index - 1] last_block_target = _get_target(last_block.warp) if last_block_target.index < terminator_index: i += 1 continue assert last_block_target in terminators subexpression = body[i:target_index] else: assert target in terminators while i < len(body) - 2: next_block = body[i + 1] next_target = _get_target(next_block.warp) if next_target != target: break next_inv = _is_inverted(next_block.warp, true, end) this_inv = _is_inverted(warp, true, end) # Special hack for unary expressions (x, not x)... if next_inv != this_inv: break block = next_block warp = next_block.warp i += 1 subexpression = body[subexpression_start:i + 1] last_block = subexpression[-1] last_block_index = body.index(last_block) next_block = body[last_block_index + 1] operator = _get_operator(last_block, true, end) subexpression = _compile_subexpression(subexpression, operator, last_block, next_block, true, end) parts.append(subexpression) parts.append(operator) i = last_block_index + 1 subexpression_start = i last = body[-1] if isinstance(last.warp, nodes.ConditionalWarp): if _is_inverted(last.warp, true, end): last = _invert(last.warp.condition) else: last = last.warp.condition parts.append(last) else: assert isinstance(last.warp, (nodes.EndWarp, nodes.UnconditionalWarp)) src = _get_last_assignment_source(last) if src is None: src = nodes.Primitive() if last.warp.target == true: src.type = nodes.Primitive.T_TRUE else: src.type = nodes.Primitive.T_FALSE parts.append(src) return parts def _get_target(warp, allow_end=False): if isinstance(warp, nodes.ConditionalWarp): return warp.false_target else: if allow_end and isinstance(warp, nodes.EndWarp): return getattr(warp, "_target", None) assert isinstance(warp, nodes.UnconditionalWarp) return warp.target def _set_target(warp, target): if isinstance(warp, nodes.ConditionalWarp): warp.false_target = target else: assert isinstance(warp, nodes.UnconditionalWarp) warp.target = target def _get_operator(block, true, end): if isinstance(block.warp, nodes.UnconditionalWarp): src = _get_last_assignment_source(block) if isinstance(src, nodes.Constant): is_true = src.value != 0 elif isinstance(src, nodes.BinaryOperator): is_true = True elif isinstance(src, nodes.Primitive): is_true = src.type == nodes.Primitive.T_TRUE elif isinstance(src, nodes.Identifier): is_true = True else: #assert src is None is_true = block.warp.target == true if is_true: return binop.T_LOGICAL_OR else: return binop.T_LOGICAL_AND else: is_inverted = _is_inverted(block.warp, true, end) if is_inverted: return binop.T_LOGICAL_OR else: return binop.T_LOGICAL_AND def _get_last_assignment_source(block): if len(block.contents) == 0: return None assignment = block.contents[-1] assert isinstance(assignment, nodes.Assignment) return assignment.expressions.contents[0] def _get_and_remove_last_assignment_source(block): assignment = block.contents.pop() assert isinstance(assignment, nodes.Assignment) return assignment.expressions.contents[0] def _compile_subexpression(subexpression, operator, block, next_block, true, end): warp = block.warp if len(subexpression) == 1: if isinstance(warp, nodes.UnconditionalWarp): return _get_and_remove_last_assignment_source(block) elif _is_inverted(warp, true, end): return _invert(warp.condition) else: return warp.condition else: if isinstance(warp, nodes.UnconditionalWarp): if operator == binop.T_LOGICAL_OR: subtrue = warp.target subfalse = next_block else: subtrue = next_block subfalse = warp.target else: if operator == binop.T_LOGICAL_OR: subtrue = warp.false_target subfalse = warp.true_target else: subtrue = warp.true_target subfalse = warp.false_target return _unwarp_expression(subexpression, None, subtrue, subfalse) def _is_inverted(warp, true, end): if isinstance(warp, nodes.UnconditionalWarp): return warp.target == end if warp.false_target == true: return True elif warp.false_target == end: assert not isinstance(warp.condition, nodes.BinaryOperator) if not isinstance(warp.condition, nodes.UnaryOperator): return False return warp.condition.type == nodes.UnaryOperator.T_NOT return False _NEGATION_MAP = [None] * 100 _NEGATION_MAP[binop.T_LESS_THEN] = binop.T_GREATER_OR_EQUAL _NEGATION_MAP[binop.T_GREATER_THEN] = binop.T_LESS_OR_EQUAL _NEGATION_MAP[binop.T_LESS_OR_EQUAL] = binop.T_GREATER_THEN _NEGATION_MAP[binop.T_GREATER_OR_EQUAL] = binop.T_LESS_THEN _NEGATION_MAP[binop.T_NOT_EQUAL] = binop.T_EQUAL _NEGATION_MAP[binop.T_EQUAL] = binop.T_NOT_EQUAL def _invert(expression): if isinstance(expression, nodes.UnaryOperator): return expression.operand if not isinstance(expression, nodes.BinaryOperator): node = nodes.UnaryOperator() node.type = nodes.UnaryOperator.T_NOT node.operand = expression return node # Just in case expression = copy.deepcopy(expression) new_type = _NEGATION_MAP[expression.type] assert new_type is not None expression.type = new_type return expression def _get_terminators(body): if len(body) < 2: return None, None, body last = body[-1] if len(last.contents) != 1: return None, None, body assignment = last.contents[0] if not isinstance(assignment, nodes.Assignment): return None, None, body src = assignment.expressions.contents[0] if not isinstance(src, nodes.Primitive) or src.type != src.T_TRUE: return None, None, body prev = body[-2] if len(prev.contents) != 1: return None, None, body src = prev.contents[0].expressions.contents[0] if not isinstance(src, nodes.Primitive) or src.type != src.T_FALSE: return None, None, body return last, prev, body[:-2] def _assemble_expression(parts): if not isinstance(parts, list): return parts node = nodes.BinaryOperator() node.left = _assemble_expression(parts[-3]) node.type = parts[-2] assert isinstance(node.type, int) node.right = _assemble_expression(parts[-1]) i = len(parts) - 4 while i > 0: operator = parts[i] component = parts[i - 1] upper_node = nodes.BinaryOperator() upper_node.right = node upper_node.left = _assemble_expression(component) upper_node.type = operator node = upper_node i -= 2 return node # Split the topmost expression into smaller subexpressions at each # operator change to simplify the assembly phase def _make_explicit_subexpressions(parts): patched = [] i = 0 last_operator = parts[1] subexpression_start = -1 while i < len(parts) - 1: component = parts[i] operator = parts[i + 1] if operator < last_operator: subexpression_start = i last_operator = operator elif subexpression_start > 0: if operator > last_operator \ and (i - subexpression_start) % 2 != 0: subexpression = parts[subexpression_start:i] patched.append(subexpression) subexpression_start = -1 else: patched += [component, operator] i += 2 if subexpression_start >= 0: patched.append(parts[subexpression_start:]) else: patched.append(parts[-1]) return patched def _unwarp_if_statement(start, body, end, topmost_end): expression, body, false = _extract_if_expression(start, body, end, topmost_end) node = nodes.If() node.expression = expression # has an else branch if false != end and false != topmost_end: else_start = false else_start_index = body.index(else_start) then_body = body[:else_start_index] then_warp_out = then_body[-1].warp assert _is_jump(then_warp_out) assert then_warp_out.target in (end, topmost_end) else_body = body[else_start_index:] else_warp_out = else_body[-1].warp if isinstance(else_warp_out, nodes.UnconditionalWarp): if else_warp_out.type == nodes.UnconditionalWarp.T_JUMP: assert else_warp_out.target in (end, topmost_end) else: assert else_warp_out.target == end else: print ("err: unwarper.py assert isinstance(else_warp_out, nodes.EndWarp), Block indices are unreliable while you are mangling them! P.S. Probably they should not be named indices... But they ARE used as indices during other phases. Sometimes.") _set_end(then_body[-1]) then_blocks = _unwarp_ifs(then_body, then_body[-1], topmost_end) node.then_block.contents = then_blocks _set_end(else_body[-1]) else_blocks = _unwarp_ifs(else_body, else_body[-1], topmost_end) node.else_block.contents = else_blocks else: warp_out = body[-1].warp if not isinstance(warp_out, nodes.EndWarp): assert isinstance(warp_out, nodes.UnconditionalWarp) assert warp_out.target in (end, topmost_end) _set_end(body[-1]) then_blocks = _unwarp_ifs(body, body[-1], topmost_end) node.then_block.contents = then_blocks start.contents.append(node) def _extract_if_expression(start, body, end, topmost_end): for i, block in enumerate(body): if len(block.contents) != 0: break assert i < len(body) expression = [start] + body[:i] body = body[i:] falses = set() for i, block in enumerate(body[:-1]): if not isinstance(block.warp, nodes.UnconditionalWarp): continue if block.warp.type != nodes.UnconditionalWarp.T_JUMP: continue if block.warp.target != end and block.warp.target != topmost_end: continue falses.add(body[i + 1]) falses.add(end) if topmost_end is not None: falses.add(topmost_end) false, end_i = _search_expression_end(expression, falses) assert end_i >= 0 body = expression[end_i:] + body expression = expression[:end_i] assert len(expression) > 0 true = body[0] expression = _compile_expression(expression, None, true, false) return expression, body, false def _search_expression_end(expression, falses): expression_end = -1 false = None for i, block in enumerate(expression): target = _get_target(block.warp) if target not in falses: continue if false is None or target == false: false = target expression_end = i + 1 else: break assert false is not None return false, expression_end def _find_branching_end(blocks, topmost_end): end = blocks[0] for block in blocks: warp = block.warp target = _get_target(warp, allow_end=True) if isinstance(warp, nodes.EndWarp) and target is None: assert block == end return block if isinstance(warp, nodes.UnconditionalWarp) and target == end: return end if target.index > end.index: end = target return end def _remove_processed_blocks(blocks, boundaries): remains = [] last_end_index = -1 for start, end in boundaries: if start == end: up_to_index = start else: up_to_index = start + 1 remains += blocks[last_end_index + 1:up_to_index] last_end_index = end remains += blocks[last_end_index + 1:] return remains # ## # ## LOOPS PROCESSING # ## def _unwarp_loops(blocks, repeat_until): loops = _find_all_loops(blocks, repeat_until) if len(loops) == 0: return blocks fixed = _cleanup_breaks_and_if_ends(loops, blocks) for start, end in fixed: start_index = blocks.index(start) end_index = blocks.index(end) if repeat_until: body = blocks[start_index:end_index] else: body = blocks[start_index + 1:end_index] loop = _unwarp_loop(start, end, body) body = loop.statements.contents block = nodes.Block() block.first_address = body[0].first_address block.last_address = body[-1].last_address block.index = start.index + 1 block.contents.append(loop) block.warp = nodes.UnconditionalWarp() block.warp.type = nodes.UnconditionalWarp.T_FLOW block.warp.target = end _replace_targets(blocks, body[0], block) _set_end(body[-1]) _unwarp_breaks(start, body, end) blocks = blocks[:start_index + 1] + [block] + blocks[end_index:] return blocks def _cleanup_breaks_and_if_ends(loops, blocks): outer_start_index = -1 outer_end = None current_start_index = -1 current_end = None fixed = [] for start, end in loops: if start.index in (outer_start_index, current_start_index): end_i = blocks.index(end) last_in_body = blocks[end_i - 1] warp = last_in_body.warp assert isinstance(warp, nodes.UnconditionalWarp) assert warp.target == start # Break if start.index == outer_start_index: assert outer_end is not None outer_end_i = blocks.index(outer_end) warp.target = blocks[outer_end_i - 1] assert blocks[outer_end_i - 2] != end else: assert current_end is not None assert start.index == current_start_index current_end_i = blocks.index(current_end) last = blocks[current_end_i - 1] if last == end: last = _create_next_block(end) last.warp = end.warp _set_flow_to(end, last) blocks.insert(current_end_i, last) warp.target = last else: fixed.append((start, end)) if current_end is not None \ and current_start_index < start.index \ and current_end.index >= end.index: outer_start_index = current_start_index outer_end = current_end else: outer_start_index = -1 outer_end = None current_start_index = start.index current_end = end return fixed def _replace_targets(blocks, original, replacement): for block in blocks: warp = block.warp if isinstance(warp, nodes.UnconditionalWarp): if warp.target == original: warp.target = replacement elif isinstance(warp, nodes.ConditionalWarp): if warp.true_target == original: warp.true_target = replacement if warp.false_target == original: warp.false_target = replacement elif isinstance(warp, nodes.EndWarp): pass else: if warp.way_out == original: warp.way_out = replacement if warp.body == original: warp.body = replacement def _unwarp_loop(start, end, body): if len(body) > 0: last = body[-1] else: last = start if isinstance(start.warp, nodes.IteratorWarp): assert isinstance(last.warp, nodes.UnconditionalWarp) assert last.warp.target == start loop = nodes.IteratorFor() loop.statements.contents = body loop.identifiers = start.warp.variables loop.expressions = start.warp.controls loop._addr = body[0].first_address _set_flow_to(start, body[0]) elif isinstance(start.warp, nodes.NumericLoopWarp): assert isinstance(last.warp, nodes.UnconditionalWarp) assert last.warp.target == start loop = nodes.NumericFor() loop.statements.contents = body loop.variable = start.warp.index loop.expressions = start.warp.controls loop._addr = body[0].first_address _set_flow_to(start, body[0]) # While (including "while true" and "repeat until false" which will be # while true) elif isinstance(last.warp, nodes.UnconditionalWarp): assert last.warp.target == start # while true if _is_flow(start.warp): loop = nodes.While() loop.expression = nodes.Primitive() loop.expression.type = nodes.Primitive.T_TRUE loop.statements.contents = body else: # There shouldn't be many problems similar to ifs, as # we are processing loops in the order from innermost # to outermost for i, block in enumerate(body): assert len(block.contents) == 0 if _is_flow(block.warp): break assert i < len(body) expression = [start] + body[:i] body = body[i:] # Sometimes expression may decide to jump to the # outer loop start instead _fix_expression(expression, start, end) true = body[0] false = end expression = _compile_expression(expression, None, true, false) # If something jumps to the start (instead of the end) # - that's a nested if loop = nodes.While() loop.expression = expression loop.statements.contents = body _fix_nested_ifs(body, start) _set_flow_to(start, body[0]) # Repeat until else: assert isinstance(last.warp, nodes.ConditionalWarp) assert last.warp.false_target == start i = len(body) - 1 while i >= 0: block = body[i] warp = block.warp if _is_flow(warp): i += 1 break if len(block.contents) != 0: break i -= 1 expression = body[i:] body = body[:i + 1] assert len(expression) > 0 first = expression[0] if _is_jump(first.warp): # Don't append to the body - it already has it expression.pop(0) body[-1].contents.append(nodes.Break()) false = body[0] # Don't use end as it could be broken by a previous # repeat until pass true = expression[-1].warp.true_target loop = nodes.RepeatUntil() loop.expression = _compile_expression(expression, None, true, false) start_copy = copy.copy(start) start.contents = [] if len(body) > 1: _set_flow_to(start_copy, body[1]) else: _set_end(start_copy) _set_flow_to(start, start_copy) body[0] = start_copy loop.statements.contents = body return loop def _create_next_block(original): block = nodes.Block() block.first_address = original.last_address + 1 block.last_address = block.first_address block.index = original.index + 1 block.warpins_count = original.warpins_count return block def _set_flow_to(block, target): block.warp = nodes.UnconditionalWarp() block.warp.type = nodes.UnconditionalWarp.T_FLOW block.warp.target = target def _set_end(block): target = None if block.warp is not None: target = _get_target(block.warp, allow_end=True) block.warp = nodes.EndWarp() setattr(block.warp, "_target", target) def _is_flow(warp): return isinstance(warp, nodes.UnconditionalWarp) \ and warp.type == nodes.UnconditionalWarp.T_FLOW def _is_jump(warp): return isinstance(warp, nodes.UnconditionalWarp) \ and warp.type == nodes.UnconditionalWarp.T_JUMP def _fix_nested_ifs(blocks, start): # We can't point both targets of a conditional warp to the # same block. We will have to create a new block last = _create_next_block(blocks[-1]) if isinstance(blocks[-1].warp, nodes.ConditionalWarp): blocks[-1].warp.false_target = last else: _set_flow_to(blocks[-1], last) blocks.append(last) _set_end(last) for block in blocks[:-1]: target = _get_target(block.warp) if target == start: _set_target(block.warp, last) def _fix_expression(blocks, start, end): for block in blocks: if len(block.contents) != 0: break target = _get_target(block.warp) if target.index < start.index: _set_target(block.warp, end) def _gather_possible_ends(block): warp = block.warp ends = set((block,)) while _is_jump(warp): block = warp.target warp = block.warp ends.add(block) return ends BREAK_INFINITE = 0 BREAK_ONE_USE = 1 def _unwarp_breaks(start, blocks, next_block): blocks_set = set([start] + blocks) ends = _gather_possible_ends(next_block) breaks = set() patched = [] for i, block in enumerate(blocks): warp = block.warp if not isinstance(warp, nodes.UnconditionalWarp): patched.append(block) continue target = _get_target(warp) if target in blocks_set: patched.append(block) continue assert target in ends, "GOTO statements are not supported" if block.warpins_count != 0: new_block = _create_next_block(block) new_block.warpins_count = block.warpins_count _set_flow_to(block, new_block) patched.append(block) patched.append(new_block) block = new_block else: patched.append(block) block.contents.append(nodes.Break()) if i + 1 == len(blocks): _set_end(block) else: _set_flow_to(block, blocks[i + 1]) breaks.add(block) blocks[:] = patched if len(breaks) == 0: return breaks_stack = [] warpsout = [] pending_break = None for i, block in enumerate(reversed(blocks)): if block in breaks: pending_break = None if block.warpins_count == 0: breaks_stack.append((BREAK_ONE_USE, block)) else: breaks_stack.append((BREAK_INFINITE, block)) continue warp = block.warp if not isinstance(warp, nodes.ConditionalWarp): if _is_flow(warp): pending_break = None continue target = _get_target(warp) if target in blocks_set: continue assert target in ends, "GOTO statements are not supported" if pending_break is None: assert len(breaks_stack) > 0 top_break = breaks_stack[-1] _set_target(warp, top_break[1]) if top_break[0] == BREAK_ONE_USE: pending_break = breaks_stack.pop() warpsout = [] else: warpsout.append(block) else: _set_target(warp, pending_break[1]) warpsout.append(block) if len(block.contents) > 0: pending_break = None while len(breaks_stack) > 0 and breaks_stack[-1][0] == BREAK_INFINITE: breaks_stack.pop() # And pray for the best... while len(warpsout) > 0 and len(breaks_stack) > 0: _set_target(warpsout.pop().warp, breaks_stack.pop()[1]) # # We don't need any complex checks here. # # Just search for any negative jump - that's a loop and what it's jumping to is # a loop start. # def _find_all_loops(blocks, repeat_until): # Duplicates are NOT possible loops = [] i = 0 while i < len(blocks): block = blocks[i] warp = block.warp if isinstance(warp, nodes.UnconditionalWarp): if warp.type == nodes.UnconditionalWarp.T_FLOW: i += 1 continue if warp.target.index <= block.index: assert not repeat_until assert i < len(blocks) - 1 loops.append((warp.target, blocks[i + 1])) elif repeat_until and isinstance(warp, nodes.ConditionalWarp): if warp.false_target.index > block.index: i += 1 continue start = warp.false_target first = block end = block last_i = i # Find the end of the expression while i < len(blocks): block = blocks[i] warp = block.warp if block != first and len(block.contents) != 0: break if isinstance(warp, nodes.EndWarp): break # Go up to a first negative jump of an # another loop target = _get_target(warp) if target.index < block.index: if target == start: start = target end = block last_i = i else: break i += 1 # And then rollback to the last known negative jump # of our loop i = last_i # There always should be at least one return block end_index = blocks.index(end) end = blocks[end_index + 1] loops.append((start, end)) i += 1 # Reverse the order so inner "while" loops are processed before # outer loops return list(reversed(sorted(loops, key=lambda x: x[0].index))) def _get_previous_block(block, blocks): block_index = blocks.index(block) assert block_index > 0 return blocks[block_index - 1] ================================================ FILE: ljd/ast/validator.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # import ljd.ast.nodes as nodes import ljd.ast.traverse as traverse class TypeRestriction(): def __init__(self, default, specific): if isinstance(default, dict) and specific == {}: specific = default default = None self.default = default self.specific = specific def check(self, node): try: typespec = self.specific[node] except KeyError: typespec = self.default #assert typespec, "Unknown node: {0}".format(node) #assert isinstance(node, typespec),"Invalid node type: {0} should be: {1}".format(type(node), typespec) STATEMENT_TYPES = ( nodes.Assignment, nodes.If, nodes.IteratorFor, nodes.NumericFor, nodes.RepeatUntil, nodes.Return, nodes.Break, nodes.FunctionCall, nodes.While ) EXPRESSION_TYPES = ( nodes.FunctionCall, nodes.Primitive, nodes.Constant, nodes.Identifier, nodes.FunctionDefinition, nodes.TableConstructor, nodes.Vararg, nodes.BinaryOperator, nodes.UnaryOperator, nodes.MULTRES, nodes.TableElement, ) VARIABLE_TYPES = ( nodes.Identifier, nodes.TableElement, nodes.MULTRES # It's not valid here, but it is a hack anyway... ) WARP_TYPES = ( nodes.UnconditionalWarp, nodes.ConditionalWarp, nodes.IteratorWarp, nodes.NumericLoopWarp, nodes.EndWarp ) class Visitor(traverse.Visitor): def __init__(self, warped=True): # Restrictions for the upmost level self.restrictions = [None] self.warped = warped def _set_restrictions(self, default, specific={}): self.restrictions[-1] = TypeRestriction(default, specific) # ## def visit_function_definition(self, node): self._set_restrictions(nodes.Block, { node.arguments: nodes.IdentifiersList, node.statements: nodes.StatementsList }) # ## def visit_table_constructor(self, node): self._set_restrictions(nodes.RecordsList) def visit_array_record(self, node): self._set_restrictions(EXPRESSION_TYPES) def visit_table_record(self, node): self._set_restrictions(EXPRESSION_TYPES) # ## def visit_assignment(self, node): self._set_restrictions({ node.destinations: nodes.VariablesList, node.expressions: nodes.ExpressionsList }) if not isinstance(node.destinations.contents[0], nodes.Identifier): return if node.destinations.contents[0].type != nodes.Identifier.T_LOCAL: return # Don't test type flag here # ## def visit_binary_operator(self, node): self._set_restrictions(EXPRESSION_TYPES) assert node.type == nodes.BinaryOperator.T_LOGICAL_OR \ or node.type == nodes.BinaryOperator.T_LOGICAL_AND \ \ or node.type == nodes.BinaryOperator.T_LESS_THEN \ or node.type == nodes.BinaryOperator.T_GREATER_THEN \ or node.type == nodes.BinaryOperator.T_LESS_OR_EQUAL \ or node.type == nodes.BinaryOperator.T_GREATER_OR_EQUAL \ \ or node.type == nodes.BinaryOperator.T_NOT_EQUAL \ or node.type == nodes.BinaryOperator.T_EQUAL \ \ or node.type == nodes.BinaryOperator.T_CONCAT \ \ or node.type == nodes.BinaryOperator.T_ADD \ or node.type == nodes.BinaryOperator.T_SUBTRACT \ \ or node.type == nodes.BinaryOperator.T_MULTIPLY \ or node.type == nodes.BinaryOperator.T_DIVISION \ or node.type == nodes.BinaryOperator.T_MOD \ \ or node.type == nodes.BinaryOperator.T_POW def visit_unary_operator(self, node): self._set_restrictions(EXPRESSION_TYPES) assert node.type == nodes.UnaryOperator.T_NOT \ or node.type == nodes.UnaryOperator.T_LENGTH_OPERATOR \ or node.type == nodes.UnaryOperator.T_MINUS # ## def visit_statements_list(self, node): if self.warped: types = nodes.Block else: types = STATEMENT_TYPES self._set_restrictions(types) def visit_identifiers_list(self, node): # HACK self._set_restrictions((nodes.Identifier, nodes.Vararg)) def visit_records_list(self, node): self._set_restrictions((nodes.TableRecord, nodes.ArrayRecord, nodes.FunctionCall, nodes.Vararg)) if len(node.contents) == 0: return is_array = isinstance(node.contents[0], nodes.ArrayRecord) for i, x in enumerate(node.contents): if is_array: assert isinstance(x, nodes.ArrayRecord) elif not isinstance(x, nodes.TableRecord): assert i == (len(node.contents) - 1) def visit_variables_list(self, node): self._set_restrictions(VARIABLE_TYPES) def visit_expressions_list(self, node): self._set_restrictions(EXPRESSION_TYPES) # ## def visit_identifier(self, node): assert node.type == nodes.Identifier.T_SLOT \ or node.type == nodes.Identifier.T_BUILTIN \ or node.type == nodes.Identifier.T_UPVALUE \ or (node.name is not None \ and node.type == nodes.Identifier.T_LOCAL) assert node.type == nodes.Identifier.T_BUILTIN or node.slot >= 0 def visit_table_element(self, node): self._set_restrictions(EXPRESSION_TYPES) def visit_function_call(self, node): self._set_restrictions({ node.function: VARIABLE_TYPES, node.arguments: nodes.ExpressionsList }) # ## def visit_if(self, node): self._set_restrictions(nodes.ElseIf, { node.expression: EXPRESSION_TYPES, node.then_block: nodes.StatementsList, node.else_block: nodes.StatementsList }) def visit_elseif(self, node): self._set_restrictions({ node.expression: EXPRESSION_TYPES, node.then_block: nodes.StatementsList }) # ## def visit_block(self, node): self._set_restrictions(STATEMENT_TYPES, { node.warp: WARP_TYPES }) assert node.index >= 0 assert node.first_address >= 0 \ and node.first_address <= node.last_address # if false produce a statements without warps in # assert node.warpins_count > 0 def visit_unconditional_warp(self, node): assert node.target is not None assert node.type == nodes.UnconditionalWarp.T_JUMP \ or node.type == nodes.UnconditionalWarp.T_FLOW def visit_conditional_warp(self, node): self._set_restrictions({ node.condition: EXPRESSION_TYPES }) assert node.true_target is not None assert node.false_target is not None # It might happen in case of if blabla or true stuff # or in case of a = a and foo(a) or a type expression # assert node.true_target != node.false_target # assert node.true_target.index != node.false_target.index def visit_iterator_warp(self, node): assert node.body is not None assert node.way_out is not None assert node.way_out.index > node.body.index self._set_restrictions(nodes.Block, { node.variables: nodes.VariablesList, node.controls: nodes.ExpressionsList }) def visit_numeric_loop_warp(self, node): assert node.body is not None assert node.way_out is not None self._set_restrictions(nodes.Block, { node.index: nodes.Identifier, node.controls: nodes.ExpressionsList }) # ## def visit_return(self, node): self._set_restrictions(nodes.ExpressionsList) # ## def visit_while(self, node): self._set_restrictions({ node.expression: EXPRESSION_TYPES, node.statements: nodes.StatementsList }) def visit_repeat_until(self, node): self._set_restrictions({ node.expression: EXPRESSION_TYPES, node.statements: nodes.StatementsList }) def visit_numeric_for(self, node): self._set_restrictions({ node.expressions: nodes.ExpressionsList, node.statements: nodes.StatementsList, node.variable: VARIABLE_TYPES }) def visit_iterator_for(self, node): self._set_restrictions({ node.expressions: nodes.ExpressionsList, node.identifiers: nodes.VariablesList, node.statements: nodes.StatementsList }) # ## def visit_constant(self, node): assert node.type == nodes.Constant.T_CDATA \ or node.type == nodes.Constant.T_FLOAT \ or node.type == nodes.Constant.T_INTEGER \ or node.type == nodes.Constant.T_STRING def visit_primitive(self, node): assert node.type == nodes.Primitive.T_NIL \ or node.type == nodes.Primitive.T_TRUE \ or node.type == nodes.Primitive.T_FALSE # ## def _visit(self, node): restrictions = self.restrictions[-1] if restrictions is not None: restrictions.check(node) # Add layer for the child node self.restrictions.append(None) traverse.Visitor._visit(self, node) # And pop it back self.restrictions.pop() def validate(ast, warped=True): visitor = Visitor(warped) traverse.traverse(visitor, ast) ================================================ FILE: ljd/bytecode/__init__.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # ================================================ FILE: ljd/bytecode/constants.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # T_NIL = 0 T_FALSE = 1 T_TRUE = 2 class Table(): def __init__(self): self.array = [] # Use a list so we can keep the original items order in the # table self.dictionary = [] class Constants(): def __init__(self): self.upvalue_references = [] self.numeric_constants = [] self.complex_constants = [] ================================================ FILE: ljd/bytecode/debuginfo.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # class VariableInfo(): T_VISIBILE = 0 T_INTERNAL = 1 def __init__(self): self.start_addr = 0 self.end_addr = 0 self.type = -1 self.name = "" class DebugInformation(): def __init__(self): self.addr_to_line_map = [] self.upvalue_variable_names = [] self.variable_info = [] def lookup_line_number(self, addr): try: return self.addr_to_line_map[addr] except IndexError: return 0 def lookup_local_name(self, addr, slot): for info in self.variable_info: if info.start_addr > addr: break if info.end_addr <= addr: continue elif slot == 0: return info else: slot -= 1 return None def lookup_upvalue_name(self, slot): try: return self.upvalue_variable_names[slot] except IndexError: return None ================================================ FILE: ljd/bytecode/helpers.py ================================================ def get_jump_destination(addr, instruction): return addr + instruction.CD + 1 def set_jump_destination(addr, instruction, value): instruction.CD = value - addr - 1 ================================================ FILE: ljd/bytecode/instructions.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # # # Almost direct wiki-to-code from # http://wiki.luajit.org/Bytecode-2.0 # # What the hell is an upvalue? # It is value from the upper prototype, i.e. a variable stored in a closure. # What the hell is the MULTRES thing? # The magical VM's variable that counts the CALL* or VARG returns. # I.e. it's used to speed-up things like foo(var1, var2, bar(var3, var4)) or # # function foo(...) # bla-bla # bar(...) # # Argument types T_VAR = 0 # variable slot number T_DST = 1 # variable slot number, used as a destination T_BS = 2 # base slot number, read-write T_RBS = 3 # base slot number, read-only T_UV = 4 # upvalue number (slot number, but specific to upvalues) T_LIT = 5 # literal T_SLIT = 6 # signed literal T_PRI = 7 # primitive type (0 = nil, 1 = false, 2 = true) T_NUM = 8 # numeric constant, index into constant table T_STR = 9 # string constant, negated index into constant table T_TAB = 10 # template table, negated index into constant table T_FUN = 11 # function prototype, negated index into constant table T_CDT = 12 # cdata constant, negated index into constant table T_JMP = 13 # branch target, relative to next instruction, biased with 0x8000 class _Instruction(): def __init__(self, definition): for key, value in definition.__dict__.items(): setattr(self, key, value) if self.A_type is not None: self.A = 0 if self.B_type is not None: self.B = 0 if self.CD_type is not None: self.CD = 0 class _IDef(): _LAST_OPCODE = 0 def __init__(self, name, A_type, B_type, CD_type, description): self.name = name self.opcode = _IDef._LAST_OPCODE self.A_type = A_type self.B_type = B_type self.CD_type = CD_type self.description = description self.args_count = (self.A_type is not None) \ + (self.B_type is not None) \ + (self.CD_type is not None) _IDef._LAST_OPCODE += 1 def __call__(self): return _Instruction(self) # Names and order are in sync with luaJIT bytecode for ease of changing # class = name A B C description # Comparison ops ISLT = _IDef("ISLT", T_VAR, None, T_VAR, "if {A} < {D}") ISGE = _IDef("ISGE", T_VAR, None, T_VAR, "if {A} >= {D}") ISLE = _IDef("ISLE", T_VAR, None, T_VAR, "if {A} <= {D}") ISGT = _IDef("ISGT", T_VAR, None, T_VAR, "if {A} > {D}") ISEQV = _IDef("ISEQV", T_VAR, None, T_VAR, "if {A} == {D}") ISNEV = _IDef("ISNEV", T_VAR, None, T_VAR, "if {A} ~= {D}") ISEQS = _IDef("ISEQS", T_VAR, None, T_STR, "if {A} == {D}") ISNES = _IDef("ISNES", T_VAR, None, T_STR, "if {A} ~= {D}") ISEQN = _IDef("ISEQN", T_VAR, None, T_NUM, "if {A} == {D}") ISNEN = _IDef("ISNEN", T_VAR, None, T_NUM, "if {A} ~= {D}") ISEQP = _IDef("ISEQP", T_VAR, None, T_PRI, "if {A} == {D}") ISNEP = _IDef("ISNEP", T_VAR, None, T_PRI, "if {A} ~= {D}") # Unary test and copy ops ISTC = _IDef("ISTC", T_DST, None, T_VAR, "{A} = {D}; if {D}") ISFC = _IDef("ISFC", T_DST, None, T_VAR, "{A} = {D}; if not {D}") IST = _IDef("IST", None, None, T_VAR, "if {D}") ISF = _IDef("ISF", None, None, T_VAR, "if not {D}") ISTYPE = _IDef("ISTYPE", T_VAR, None, T_LIT, "ISTYPE unknow") ISNUM = _IDef("ISNUM", T_VAR, None, T_LIT, "ISNUM unknow") # Unary ops MOV = _IDef("MOV", T_DST, None, T_VAR, "{A} = {D}") NOT = _IDef("NOT", T_DST, None, T_VAR, "{A} = not {D}") UNM = _IDef("UNM", T_DST, None, T_VAR, "{A} = -{D}") LEN = _IDef("LEN", T_DST, None, T_VAR, "{A} = #{D}") # Binary ops ADDVN = _IDef("ADDVN", T_DST, T_VAR, T_NUM, "{A} = {B} + {C}") SUBVN = _IDef("SUBVN", T_DST, T_VAR, T_NUM, "{A} = {B} - {C}") MULVN = _IDef("MULVN", T_DST, T_VAR, T_NUM, "{A} = {B} * {C}") DIVVN = _IDef("DIVVN", T_DST, T_VAR, T_NUM, "{A} = {B} / {C}") MODVN = _IDef("MODVN", T_DST, T_VAR, T_NUM, "{A} = {B} % {C}") ADDNV = _IDef("ADDNV", T_DST, T_VAR, T_NUM, "{A} = {C} + {B}") SUBNV = _IDef("SUBNV", T_DST, T_VAR, T_NUM, "{A} = {C} - {B}") MULNV = _IDef("MULNV", T_DST, T_VAR, T_NUM, "{A} = {C} * {B}") DIVNV = _IDef("DIVNV", T_DST, T_VAR, T_NUM, "{A} = {C} / {B}") MODNV = _IDef("MODNV", T_DST, T_VAR, T_NUM, "{A} = {C} % {B}") ADDVV = _IDef("ADDVV", T_DST, T_VAR, T_VAR, "{A} = {B} + {C}") SUBVV = _IDef("SUBVV", T_DST, T_VAR, T_VAR, "{A} = {B} - {C}") MULVV = _IDef("MULVV", T_DST, T_VAR, T_VAR, "{A} = {B} * {C}") DIVVV = _IDef("DIVVV", T_DST, T_VAR, T_VAR, "{A} = {B} / {C}") MODVV = _IDef("MODVV", T_DST, T_VAR, T_VAR, "{A} = {B} % {C}") POW = _IDef("POW", T_DST, T_VAR, T_VAR, "{A} = {B} ^ {C} (pow)") CAT = _IDef("CAT", T_DST, T_RBS, T_RBS, "{A} = {concat_from_B_to_C}") # Constant ops. KSTR = _IDef("KSTR", T_DST, None, T_STR, "{A} = {D}") KCDATA = _IDef("KCDATA", T_DST, None, T_CDT, "{A} = {D}") KSHORT = _IDef("KSHORT", T_DST, None, T_SLIT, "{A} = {D}") KNUM = _IDef("KNUM", T_DST, None, T_NUM, "{A} = {D}") KPRI = _IDef("KPRI", T_DST, None, T_PRI, "{A} = {D}") KNIL = _IDef("KNIL", T_BS, None, T_BS, "{from_A_to_D} = nil") # Upvalue and function ops. UGET = _IDef("UGET", T_DST, None, T_UV, "{A} = {D}") USETV = _IDef("USETV", T_UV, None, T_VAR, "{A} = {D}") USETS = _IDef("USETS", T_UV, None, T_STR, "{A} = {D}") USETN = _IDef("USETN", T_UV, None, T_NUM, "{A} = {D}") USETP = _IDef("USETP", T_UV, None, T_PRI, "{A} = {D}") UCLO = _IDef("UCLO", T_RBS, None, T_JMP, "nil uvs >= {A}; goto {D}") FNEW = _IDef("FNEW", T_DST, None, T_FUN, "{A} = function {D}") # Table ops. TNEW = _IDef("TNEW", T_DST, None, T_LIT, "{A} = new table(" " array: {D_array}," " dict: {D_dict})") TDUP = _IDef("TDUP", T_DST, None, T_TAB, "{A} = copy {D}") GGET = _IDef("GGET", T_DST, None, T_STR, "{A} = _env[{D}]") GSET = _IDef("GSET", T_VAR, None, T_STR, "_env[{D}] = {A}") TGETV = _IDef("TGETV", T_DST, T_VAR, T_VAR, "{A} = {B}[{C}]") TGETS = _IDef("TGETS", T_DST, T_VAR, T_STR, "{A} = {B}.{C}") TGETB = _IDef("TGETB", T_DST, T_VAR, T_LIT, "{A} = {B}[{C}]") TGETR = _IDef("TGETR", T_DST, T_VAR, T_VAR, "unkown TGETR") TSETV = _IDef("TSETV", T_VAR, T_VAR, T_VAR, "{B}[{C}] = {A}") TSETS = _IDef("TSETS", T_VAR, T_VAR, T_STR, "{B}.{C} = {A}") TSETB = _IDef("TSETB", T_VAR, T_VAR, T_LIT, "{B}[{C}] = {A}") TSETM = _IDef("TSETM", T_BS, None, T_NUM, "for i = 0, MULTRES, 1 do" " {A_minus_one}[{D_low} + i] = slot({A} + i)") TSETR = _IDef("TSETR", T_VAR, T_VAR, T_VAR, "unkow TSETR") # Calls and vararg handling. T = tail call. CALLM = _IDef("CALLM", T_BS, T_LIT, T_LIT, "{from_A_x_B_minus_two} = {A}({from_A_plus_one_x_C}, ...MULTRES)") CALL = _IDef("CALL", T_BS, T_LIT, T_LIT, "{from_A_x_B_minus_two} = {A}({from_A_plus_one_x_C_minus_one})") CALLMT = _IDef("CALLMT", T_BS, None, T_LIT, "return {A}({from_A_plus_one_x_D}, ...MULTRES)") CALLT = _IDef("CALLT", T_BS, None, T_LIT, "return {A}({from_A_plus_one_x_D_minus_one})") ITERC = _IDef("ITERC", T_BS, T_LIT, T_LIT, "{A}, {A_plus_one}, {A_plus_two} =" " {A_minus_three}, {A_minus_two}, {A_minus_one};" " {from_A_x_B_minus_two} =" " {A_minus_three}({A_minus_two}, {A_minus_one})") ITERN = _IDef("ITERN", T_BS, T_LIT, T_LIT, "{A}, {A_plus_one}, {A_plus_two} =" " {A_minus_three}, {A_minus_two}, {A_minus_one};" " {from_A_x_B_minus_two} =" " {A_minus_three}({A_minus_two}, {A_minus_one})") VARG = _IDef("VARG", T_BS, T_LIT, T_LIT, "{from_A_x_B_minus_two} = ...") ISNEXT = _IDef("ISNEXT", T_BS, None, T_JMP, "Verify ITERN at {D}; goto {D}") # Returns. RETM = _IDef("RETM", T_BS, None, T_LIT, "return {from_A_x_D_minus_one}, ...MULTRES") RET = _IDef("RET", T_RBS, None, T_LIT, "return {from_A_x_D_minus_two}") RET0 = _IDef("RET0", T_RBS, None, T_LIT, "return") RET1 = _IDef("RET1", T_RBS, None, T_LIT, "return {A}") # Loops and branches. I/J = interp/JIT, I/C/L = init/call/loop. FORI = _IDef("FORI", T_BS, None, T_JMP, "for {A_plus_three} = {A},{A_plus_one},{A_plus_two}" " else goto {D}") JFORI = _IDef("JFORI", T_BS, None, T_JMP, "for {A_plus_three} = {A},{A_plus_one},{A_plus_two}" " else goto {D}") FORL = _IDef("FORL", T_BS, None, T_JMP, "{A} = {A} + {A_plus_two};" " if cmp({A}, sign {A_plus_two}, {A_plus_one}) goto {D}") IFORL = _IDef("IFORL", T_BS, None, T_JMP, "{A} = {A} + {A_plus_two};" " if cmp({A}, sign {A_plus_two}, {A_plus_one}) goto {D}") JFORL = _IDef("JFORL", T_BS, None, T_JMP, "{A} = {A} + {A_plus_two};" " if cmp({A}, sign {A_plus_two}, {A_plus_one}) goto {D}") ITERL = _IDef("ITERL", T_BS, None, T_JMP, "{A_minus_one} = {A}; if {A} != nil goto {D}") IITERL = _IDef("IITERL", T_BS, None, T_JMP, "{A_minus_one} = {A}; if {A} != nil goto {D}") JITERL = _IDef("JITERL", T_BS, None, T_LIT, "{A_minus_one} = {A}; if {A} != nil goto {D}") LOOP = _IDef("LOOP", T_RBS, None, T_JMP, "Noop") ILOOP = _IDef("ILOOP", T_RBS, None, T_JMP, "Noop") JLOOP = _IDef("JLOOP", T_RBS, None, T_LIT, "Noop") JMP = _IDef("JMP", T_RBS, None, T_JMP, " goto {D}") # Function headers. I/J = interp/JIT, F/V/C = fixarg/vararg/C func. # Shouldn't be ever seen - they are not stored in raw dump? FUNCF = _IDef("FUNCF", T_RBS, None, None, "Fixed-arg function with frame size {A}") IFUNCF = _IDef("IFUNCF", T_RBS, None, None, "Interpreted fixed-arg function with frame size {A}") JFUNCF = _IDef("JFUNCF", T_RBS, None, T_LIT, "JIT compiled fixed-arg function with frame size {A}") FUNCV = _IDef("FUNCV", T_RBS, None, None, "Var-arg function with frame size {A}") IFUNCV = _IDef("IFUNCV", T_RBS, None, None, "Interpreted var-arg function with frame size {A}") JFUNCV = _IDef("JFUNCV", T_RBS, None, T_LIT, "JIT compiled var-arg function with frame size {A}") FUNCC = _IDef("FUNCC", T_RBS, None, None, "C function with frame size {A}") FUNCCW = _IDef("FUNCCW", T_RBS, None, None, "Wrapped C function with frame size {A}") UNKNW = _IDef("UNKNW", T_LIT, T_LIT, T_LIT, "Unknown instruction") ================================================ FILE: ljd/bytecode/prototype.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # import ljd.bytecode.constants as constants import ljd.bytecode.debuginfo as debug class Flags(): def __init(self): self.has_sub_prototypes = False self.is_variadic = False self.has_ffi = False self.has_jit = True self.has_iloop = False class Prototype(): def __init__(self): self.flags = Flags() self.arguments_count = 0 self.framesize = 0 self.first_line_number = 0 self.lines_count = 0 self.instructions = [] self.constants = constants.Constants() self.debuginfo = debug.DebugInformation() ================================================ FILE: ljd/lua/__init__.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # ================================================ FILE: ljd/lua/writer.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # import re import ljd.ast.nodes as nodes import ljd.ast.traverse as traverse CMD_START_STATEMENT = 0 CMD_END_STATEMENT = 1 CMD_END_LINE = 3 CMD_START_BLOCK = 4 CMD_END_BLOCK = 5 CMD_WRITE = 6 OPERATOR_TYPES = (nodes.BinaryOperator, nodes.UnaryOperator) STATEMENT_NONE = -1 STATEMENT_ASSIGNMENT = 0 STATEMENT_FUNCTION_CALL = 1 STATEMENT_RETURN = 2 STATEMENT_BREAK = 3 STATEMENT_IF = 4 STATEMENT_ITERATOR_FOR = 5 STATEMENT_NUMERIC_FOR = 6 STATEMENT_REPEAT_UNTIL = 7 STATEMENT_WHILE = 8 STATEMENT_FUNCTION = 9 VALID_IDENTIFIER = re.compile(r'^\w[\w\d]*$') class _State(): def __init__(self): self.current_statement = STATEMENT_NONE self.function_name = None self.function_local = False class Visitor(traverse.Visitor): def __init__(self): traverse.Visitor.__init__(self) self.print_queue = [] self._visited_nodes = [set()] self._states = [_State()] # ## def _start_statement(self, statement): assert self._state().current_statement == STATEMENT_NONE self._state().current_statement = statement self.print_queue.append((CMD_START_STATEMENT, statement)) def _end_statement(self, statement): assert statement == self._state().current_statement self._state().current_statement = STATEMENT_NONE self.print_queue.append((CMD_END_STATEMENT, statement)) def _end_line(self): self.print_queue.append((CMD_END_LINE,)) def _start_block(self): self.print_queue.append((CMD_START_BLOCK,)) def _end_block(self): self.print_queue.append((CMD_END_BLOCK,)) def _write(self, fmt, *args, **kargs): self.print_queue.append((CMD_WRITE, fmt, args, kargs)) def _state(self): return self._states[-1] def _push_state(self): return self._states.append(_State()) def _pop_state(self): return self._states.pop() # ## def visit_function_definition(self, node): is_statement = self._state().function_name is not None if is_statement: self._start_statement(STATEMENT_FUNCTION) if self._state().function_local: self._write("local ") self._write("function ") self._visit(self._state().function_name) self._write("(") self._state().function_name = None else: self._write("function (") self._visit(node.arguments) self._write(")") self._end_line() self._visit(node.statements) self._write("end") if is_statement: self._end_statement(STATEMENT_FUNCTION) # ## def visit_table_constructor(self, node): self._write("{") if len(node.records.contents) + len(node.array.contents) > 0: self._end_line() self._start_block() array = node.array.contents records = node.records.contents all_records = nodes.RecordsList() all_records.contents = array + records if len(array) > 0: first = array[0].value all_records.contents.pop(0) if not isinstance(first, nodes.Primitive) \ or first.type != first.T_NIL: record = nodes.TableRecord() record.key = nodes.Constant() record.key.type = nodes.Constant.T_INTEGER record.key.value = 0 record.value = first all_records.contents.insert(0, record) self._visit(all_records) self._skip(node.array) self._skip(node.records) self._end_block() else: self._skip(node.array) self._skip(node.records) self._write("}") def visit_table_record(self, node): if self._is_valid_name(node.key): self._write(node.key.value) self._skip(node.key) self._write(" = ") else: self._write("[") self._visit(node.key) self._write("] = ") self._visit(node.value) # visit_array_record is a passthough # ## def visit_assignment(self, node): is_local = node.type == node.T_LOCAL_DEFINITION dsts = node.destinations.contents srcs = node.expressions.contents if len(dsts) == 1 and len(srcs) == 1: dst = dsts[0] src = srcs[0] src_is_function = isinstance(src, nodes.FunctionDefinition) dst_is_simple = self._is_variable(dst) if src_is_function and dst_is_simple: self._state().function_name = dst self._state().function_local = is_local self._visit(src) self._skip(node.destinations) self._skip(node.expressions) return if is_local: self._write("local ") self._start_statement(STATEMENT_ASSIGNMENT) self._visit(node.destinations) self._write(" = ") self._visit(node.expressions) self._end_statement(STATEMENT_ASSIGNMENT) def _is_variable(self, node): if isinstance(node, nodes.Identifier): return True return self._is_global(node) def _is_global(self, node): if isinstance(node, nodes.TableElement): return self._is_builtin(node.table) return False def _is_builtin(self, node): if not isinstance(node, nodes.Identifier): return False return node.type == nodes.Identifier.T_BUILTIN def _is_valid_name(self, key): if not isinstance(key, nodes.Constant) or key.type != key.T_STRING: return False return VALID_IDENTIFIER.match(key.value) # ## def visit_binary_operator(self, node): is_left_op = isinstance(node.left, OPERATOR_TYPES) is_right_op = isinstance(node.right, OPERATOR_TYPES) # If the subexpressions are less in order then this expression, # they should go with parentheses left_parentheses = False right_parentheses = False binop = nodes.BinaryOperator if is_left_op: if node.type <= binop.T_LOGICAL_AND: left_parentheses = ( node.left.type <= node.type or (node.left.type <= binop.T_LOGICAL_AND and node.type <= binop.T_LOGICAL_AND) ) and node.left.type != node.type else: left_parentheses = node.left.type < node.type if is_right_op: if node.type <= nodes.BinaryOperator.T_LOGICAL_AND: right_parentheses = ( node.right.type <= node.type or (node.right.type <= binop.T_LOGICAL_AND and node.type <= binop.T_LOGICAL_AND) ) and node.right.type != node.type else: right_parentheses = node.right.type < node.type if left_parentheses: self._write("(") self._visit(node.left) if left_parentheses: self._write(")") if node.type == nodes.BinaryOperator.T_LOGICAL_OR: self._write(" or ") elif node.type == nodes.BinaryOperator.T_LOGICAL_AND: self._write(" and ") elif node.type == nodes.BinaryOperator.T_LESS_THEN: self._write(" < ") elif node.type == nodes.BinaryOperator.T_GREATER_THEN: self._write(" > ") elif node.type == nodes.BinaryOperator.T_LESS_OR_EQUAL: self._write(" <= ") elif node.type == nodes.BinaryOperator.T_GREATER_OR_EQUAL: self._write(" >= ") elif node.type == nodes.BinaryOperator.T_NOT_EQUAL: self._write(" ~= ") elif node.type == nodes.BinaryOperator.T_EQUAL: self._write(" == ") elif node.type == nodes.BinaryOperator.T_CONCAT: self._write(" .. ") elif node.type == nodes.BinaryOperator.T_ADD: self._write(" + ") elif node.type == nodes.BinaryOperator.T_SUBTRACT: self._write(" - ") elif node.type == nodes.BinaryOperator.T_DIVISION: self._write("/") elif node.type == nodes.BinaryOperator.T_MULTIPLY: self._write("*") elif node.type == nodes.BinaryOperator.T_MOD: self._write("%") else: assert node.type == nodes.BinaryOperator.T_POW self._write("^") if right_parentheses: self._write("(") self._visit(node.right) if right_parentheses: self._write(")") def visit_unary_operator(self, node): if node.type == nodes.UnaryOperator.T_LENGTH_OPERATOR: self._write("#") elif node.type == nodes.UnaryOperator.T_MINUS: self._write("-") elif node.type == nodes.UnaryOperator.T_NOT: self._write("not ") has_subexp = isinstance(node.operand, OPERATOR_TYPES) need_parentheses = has_subexp and node.operand.type < node.type if need_parentheses: self._write("(") self._visit(node.operand) if need_parentheses: self._write(")") # ## def visit_statements_list(self, node): if len(self._states) > 1: self._start_block() self._push_state() def leave_statements_list(self, node): self._pop_state() if len(self._states) > 1: self._end_block() def _visit_comma_separated_list(self, node): if node.contents == []: return for subnode in node.contents[:-1]: self._visit(subnode) self._write(", ") self._visit(node.contents[-1]) visit_identifiers_list = _visit_comma_separated_list def visit_records_list(self, node): if node.contents == []: return for subnode in node.contents[:-1]: self._visit(subnode) self._write(",") self._end_line() self._visit(node.contents[-1]) self._end_line() visit_variables_list = _visit_comma_separated_list visit_expressions_list = _visit_comma_separated_list # ## def visit_identifier(self, node): if node.type == nodes.Identifier.T_SLOT: self._write("slot{0}", node.slot) else: self._write(node.name) def visit_multres(self, node): self._write("MULTRES") def visit_table_element(self, node): key = node.key base = node.table is_valid_name = self._is_valid_name(key) if self._is_global(node): assert is_valid_name self._skip(base) self._skip(key) self._write(key.value) return base_is_constructor = isinstance(base, nodes.TableConstructor) if not base_is_constructor and is_valid_name: self._visit(base) self._write(".") self._write(key.value) self._skip(key) else: if base_is_constructor: self._write("(") self._visit(base) if base_is_constructor: self._write(")") self._write("[") self._visit(key) self._write("]") def visit_vararg(self, node): self._write("...") def visit_function_call(self, node): is_statement = self._state().current_statement == STATEMENT_NONE if is_statement: self._start_statement(STATEMENT_FUNCTION_CALL) # We are going to modify this list so we can remove the first # argument args = node.arguments.contents is_method = False if len(args) > 0 and isinstance(node.function, nodes.TableElement): table = node.function.table first_arg = node.arguments.contents[0] if self._is_valid_name(node.function.key): is_method = table == first_arg if is_method: self._visit(node.function.table) self._write(":") self._write(node.function.key.value) self._skip(node.function) args.pop(0) self._write("(") self._visit(node.arguments) self._write(")") self._skip(node.arguments) else: self._visit(node.function) self._write("(") self._visit(node.arguments) self._write(")") if is_statement: self._end_statement(STATEMENT_FUNCTION_CALL) # ## def visit_if(self, node): self._start_statement(STATEMENT_IF) self._write("if ") self._visit(node.expression) self._write(" then") self._end_line() self._visit(node.then_block) self._visit_list(node.elseifs) if len(node.else_block.contents) > 0: self._write("else") self._end_line() self._visit(node.else_block) else: self._skip(node.else_block) self._write("end") self._end_statement(STATEMENT_IF) def visit_elseif(self, node): self._write("elseif ") self._visit(node.expression) self._write(" then") self._end_line() self._visit(node.then_block) # ## def visit_block(self, node): self._write("--- BLOCK #{0} {1}-{2}, warpins: {3} ---", node.index, node.first_address, node.last_address, node.warpins_count) self._end_line() self._visit_list(node.contents) self._write("--- END OF BLOCK #{0} ---", node.index) self._end_line() self._end_line() self._visit(node.warp) self._end_line() self._end_line() def visit_unconditional_warp(self, node): if node.type == nodes.UnconditionalWarp.T_FLOW: self._write("FLOW") elif node.type == nodes.UnconditionalWarp.T_JUMP: self._write("UNCONDITIONAL JUMP") self._write("; TARGET BLOCK #{0}", node.target.index) self._end_line() def visit_conditional_warp(self, node): if hasattr(node, "_slot"): self._write("slot" + str(node._slot)) self._write(" = ") self._write("if ") self._visit(node.condition) self._write(" then") self._end_line() self._write("JUMP TO BLOCK #{0}", node.true_target.index) self._end_line() self._write("else") self._end_line() self._write("JUMP TO BLOCK #{0}", node.false_target.index) self._end_line() self._write("end") self._end_line() def visit_iterator_warp(self, node): self._write("for ") self._visit(node.variables) self._write(" in ") self._visit(node.controls) self._end_line() self._write("LOOP BLOCK #{0}", node.body.index) self._end_line() self._write("GO OUT TO BLOCK #{0}", node.way_out.index) self._end_line() def visit_numeric_loop_warp(self, node): self._write("for ") self._visit(node.index) self._write("=") self._visit(node.controls) self._end_line() self._write("LOOP BLOCK #{0}", node.body.index) self._end_line() self._write("GO OUT TO BLOCK #{0}", node.way_out.index) # ## def visit_return(self, node): self._start_statement(STATEMENT_RETURN) self._write("return ") self._visit(node.returns) self._end_statement(STATEMENT_RETURN) def visit_break(self, node): self._start_statement(STATEMENT_BREAK) self._write("break") self._end_statement(STATEMENT_BREAK) # ## def visit_while(self, node): self._start_statement(STATEMENT_WHILE) self._write("while ") self._visit(node.expression) self._write(" do") self._end_line() self._visit(node.statements) self._write("end") self._end_statement(STATEMENT_WHILE) def visit_repeat_until(self, node): self._start_statement(STATEMENT_REPEAT_UNTIL) self._write("repeat") self._end_line() self._visit(node.statements) self._write("until ") self._visit(node.expression) self._end_statement(STATEMENT_REPEAT_UNTIL) def visit_numeric_for(self, node): self._start_statement(STATEMENT_NUMERIC_FOR) self._write("for ") self._visit(node.variable) self._write(" = ") self._visit(node.expressions) self._write(" do") self._end_line() self._visit(node.statements) self._write("end") self._end_statement(STATEMENT_NUMERIC_FOR) def visit_iterator_for(self, node): self._start_statement(STATEMENT_ITERATOR_FOR) self._write("for ") self._visit(node.identifiers) self._write(" in ") self._visit(node.expressions) self._write(" do") self._end_line() self._visit(node.statements) self._write("end") self._end_statement(STATEMENT_ITERATOR_FOR) # ## def visit_constant(self, node): if node.type != nodes.Constant.T_STRING: self._write(node.value) return lines = node.value.count("\n") if lines > 2: self._write("[[") self._write("\n") self._write(node.value) self._write("]]") else: text = node.value text = text.replace("\\", "\\\\") text = text.replace("\t", "\\t") text = text.replace("\n", "\\n") text = text.replace("\r", "\\r") text = text.replace("\"", "\\\"") self._write('"' + text + '"') def visit_primitive(self, node): if node.type == nodes.Primitive.T_FALSE: self._write("false") elif node.type == nodes.Primitive.T_TRUE: self._write("true") else: self._write("nil") def _skip(self, node): self._visited_nodes[-1].add(node) def _visit(self, node): assert node is not None if node in self._visited_nodes[-1]: return self._visited_nodes[-1].add(node) # TODO: add check # "It looks like you forgot about some node changes..." self._visited_nodes.append(set()) traverse.Visitor._visit(self, node) self._visited_nodes.pop() def write(fd, ast): assert isinstance(ast, nodes.FunctionDefinition) visitor = Visitor() traverse.traverse(visitor, ast.statements) _process_queue(fd, visitor.print_queue) def _get_next_significant(queue, i): i += 1 while i < len(queue): cmd = queue[i] if cmd[0] not in (CMD_END_LINE, CMD_WRITE): break i += 1 if i < len(queue): return queue[i] else: return (CMD_END_BLOCK,) def _process_queue(fd, queue): indent = 0 line_broken = True for i, cmd in enumerate(queue): assert isinstance(cmd, tuple) if cmd[0] == CMD_START_STATEMENT: # assert line_broken pass elif cmd[0] == CMD_END_STATEMENT: fd.write("\n") line_broken = True next_cmd = _get_next_significant(queue, i) if next_cmd[0] not in (CMD_END_BLOCK, CMD_START_BLOCK): assert next_cmd[0] == CMD_START_STATEMENT if next_cmd[1] != cmd[1] \ or cmd[1] >= STATEMENT_IF \ or next_cmd[1] >= STATEMENT_IF: fd.write("\n") elif cmd[0] == CMD_END_LINE: fd.write("\n") line_broken = True elif cmd[0] == CMD_START_BLOCK: indent += 1 elif cmd[0] == CMD_END_BLOCK: indent -= 1 assert indent >= 0 else: assert cmd[0] == CMD_WRITE if line_broken: fd.write(indent * '\t') line_broken = False _id, fmt, args, kargs = cmd if len(args) + len(kargs) > 0: text = fmt.format(*args, **kargs) elif isinstance(fmt, str): text = fmt else: text = str(fmt) fd.write(text) ================================================ FILE: ljd/pseudoasm/__init__.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # ================================================ FILE: ljd/pseudoasm/constants.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # import ljd.bytecode.constants def write_tables(writer, prototype): i = 0 for element in prototype.constants.complex_constants: if isinstance(element, ljd.bytecode.constants.Table): _write_table(writer, i, element) i += 1 def _write_table(writer, index, table): writer.stream.open_block("ktable#{0} = [", index) i = 0 for element in table.array: if i != 0 or element is not None: text = _translate_element(element) writer.stream.write_line("#{0}: {1},", i, text) i += 1 for key, value in table.dictionary: key = _translate_element(key) value = _translate_element(value) writer.stream.write_line("[{0}] = {1},", key, value) writer.stream.close_block("]") writer.stream.write_line() writer.stream.write_line() def _translate_element(element): if element is True: return "true" elif element is False: return "false" elif element is None: return "nil" elif isinstance(element, bytes): return '"' + element.decode("utf8") + '"' else: return str(element) ================================================ FILE: ljd/pseudoasm/instructions.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # import ljd.bytecode.instructions as ins from ljd.bytecode.constants import T_NIL, T_FALSE, T_TRUE import ljd.pseudoasm.prototype _FORMAT = "{addr:3}\t[{line:3}]\t{name:<5}\t{a:3}\t{b}\t{c}\t; {description}" _DESCRIPTION_HANDLERS = [None] * 255 class _State(): def __init__(self, writer, prototype, instructions): for key, value in writer.__dict__.items(): setattr(self, key, value) self.prototype = prototype self.instructions = instructions def write(writer, prototype): global _MAP # skip the first function header addr = 1 instructions = prototype.instructions writer = _State(writer, prototype, instructions) while addr < len(instructions): instruction = instructions[addr] line = prototype.debuginfo.lookup_line_number(addr) if instruction.opcode == ins.FNEW.opcode: _write_function(writer, addr, line, instruction) else: _write_instruction(writer, addr, line, instruction) addr += 1 def _write_instruction(writer, addr, line, instruction): description = _translate_description(writer, addr, line, instruction) writer.stream.write_multiline(_FORMAT, addr=addr, line=line, name=instruction.name, a=instruction.A if instruction.A_type is not None else "", b=instruction.B if instruction.B_type is not None else "", c=instruction.CD if instruction.CD_type is not None else "", description=description ) def _write_function(writer, addr, line, instruction): prototype = writer.prototype.constants.complex_constants[instruction.CD] description = ljd.pseudoasm.prototype.format_header(writer, prototype) writer.stream.open_block(_FORMAT, addr=addr, line=line, name="FNEW", a=instruction.A, b="", c=instruction.CD, description=description ) writer.stream.write_line() ljd.pseudoasm.prototype.write_body(writer, prototype) writer.stream.close_block() def _translate_description(writer, addr, line, instruction): global _DESCRIPTION_HANDLERS handler = _DESCRIPTION_HANDLERS[instruction.opcode] description = instruction.description return handler(writer, description, addr, line, instruction) def _translate(writer, addr, value, attr_type): prototype = writer.prototype if attr_type == ins.T_DST or attr_type == ins.T_BS: return "slot" + str(value) if attr_type == ins.T_VAR: name = _lookup_variable_name(writer, addr, value) if name is not None: return name else: return "slot" + str(value) elif attr_type == ins.T_UV: name = prototype.debuginfo.lookup_upvalue_name(value) return "uv" + str(value) + '"' + name + '"' elif attr_type == ins.T_PRI: if value is None or value == T_NIL: return "nil" elif value is True or value == T_TRUE: return "true" else: assert value is False or value == T_FALSE return "false" elif attr_type == ins.T_NUM: return str(prototype.constants.numeric_constants[value]) elif attr_type == ins.T_STR: binary = prototype.constants.complex_constants[value] return '"' + binary + '"' elif attr_type == ins.T_TAB: return "table#k" + str(value) elif attr_type == ins.T_CDT: return str(prototype.constants.complex_constants[value]) elif attr_type == ins.T_JMP: return str(1 + addr + value) elif attr_type == ins.T_LIT or attr_type == ins.T_SLIT: return str(value) elif attr_type == ins.T_BS or attr_type == ins.T_RBS: return "r" + str(value) else: return " " # "r" + str(value) def _lookup_variable_name(writer, addr, slot): while True: result = _lookup_variable_name_step(writer, addr, slot) if isinstance(result, tuple): addr = result[0] slot = result[1] continue return result def _lookup_variable_name_step(writer, addr, slot): info = writer.prototype.debuginfo.lookup_local_name(addr, slot) if info is not None: name = info.name if name[0] == '<': name = "slot" + str(slot) + name return name instructions = writer.instructions knil_opcode = ins.KNIL.opcode constants = writer.prototype.constants.complex_constants while addr > 1: addr -= 1 instruction = instructions[addr] if instruction.A_type == ins.T_BS: if slot >= instruction.A \ and (instruction.opcode == knil_opcode \ or slot <= instruction.CD): return None continue if instruction.A_type != ins.T_DST or instruction.A != slot: continue if instruction.opcode == ins.MOV.opcode: # Retry with new addr and slot return addr, instruction.CD if instruction.opcode == ins.GGET.opcode: binary = constants[instruction.CD] return binary # field or method if instruction.opcode == ins.TGETS.opcode: table_slot = instruction.B table = _lookup_variable_name_step(writer, addr, table_slot) if table is None: table = "" binary = constants[instruction.CD] return table + "." + binary if instruction.opcode == ins.UGET.opcode: uv = instruction.CD name = writer.prototype.debuginfo.lookup_upvalue_name(uv) return "uv" + str(uv) + '"' + name + '"' return None return None def _translate_standard(writer, addr, line, instruction): A = None B = None CD = None if instruction.A_type is not None: A = _translate(writer, addr, instruction.A, instruction.A_type) if instruction.B_type is not None: B = _translate(writer, addr, instruction.B, instruction.B_type) if instruction.CD_type is not None: CD = _translate(writer, addr, instruction.CD, instruction.CD_type) return A, B, CD def _translate_normal(writer, description, addr, line, instruction): A, B, CD = _translate_standard(writer, addr, line, instruction) return description.format(A=A, B=B, C=CD, D=CD) def _translate_concat(writer, description, addr, line, instruction): A = _translate(writer, addr, instruction.A, instruction.A_type) args = [] start = instruction.B end = instruction.CD + 1 while start != end: var = _translate(writer, addr, start, ins.T_VAR) args.append(var) start += 1 return description.format(A=A, concat_from_B_to_C=" .. ".join(args)) def _translate_nil(writer, description, addr, line, instruction): args = [] start = instruction.A end = instruction.CD + 1 while start != end: var = _translate(writer, addr, start, ins.T_VAR) args.append(var) start += 1 return description.format(from_A_to_D=", ".join(args)) def _translate_table_str_op(writer, description, addr, line, instruction): A, B, CD = _translate_standard(writer, addr, line, instruction) C = CD[1:-1] return description.format(A=A, B=B, C=C) def _translate_new_table(writer, description, addr, line, instruction): A = _translate(writer, addr, instruction.A, instruction.A_type) size = instruction.CD array_size = size & 0b0000011111111111 dict_size = 2 ** (size >> 11) return description.format( A=A, D_array=array_size, D_dict=dict_size, ) def _translate_mass_set(writer, description, addr, line, instruction): base = instruction.A table_var = _translate(writer, addr, base - 1, ins.T_VAR) first = instruction.CD return description.format( A_minus_one=table_var, A=base, D_low=first, ) def _translate_varg_call(writer, description, addr, line, instruction): base = instruction.A argn = instruction.CD retn = instruction.B - 1 args = [] returns = [] i = 0 while i < argn: args.append(_translate(writer, addr, base + i + 1, ins.T_VAR)) i += 1 i = 0 while i < retn: returns.append(_translate(writer, addr, base + i, ins.T_DST)) i += 1 func_var = _translate(writer, addr, base, ins.T_VAR) return description.format( A=func_var, from_A_x_B_minus_two=", ".join(returns) if retn >= 0 else "MULTRES", from_A_plus_one_x_C=", ".join(args) ) def _translate_call(writer, description, addr, line, instruction): base = instruction.A argn = instruction.CD - 1 retn = instruction.B - 1 args = [] returns = [] i = 0 while i < argn: args.append(_translate(writer, addr, base + i + 1, ins.T_VAR)) i += 1 i = 0 while i < retn: returns.append(_translate(writer, addr, base + i, ins.T_DST)) i += 1 func_var = _translate(writer, addr, base, ins.T_VAR) return description.format( A=func_var, from_A_x_B_minus_two=", ".join(returns) if retn >= 0 else "MULTRES", from_A_plus_one_x_C_minus_one=", ".join(args) ) def _translate_varg_tailcall(writer, description, addr, line, instruction): base = instruction.A argn = instruction.CD - 1 args = [] i = 0 while i < argn: args.append(_translate(writer, addr, base + i + 1, ins.T_VAR)) i += 1 func_var = _translate(writer, addr, base, ins.T_VAR) return description.format( A=func_var, from_A_plus_one_x_D=", ".join(args) ) def _translate_tailcall(writer, description, addr, line, instruction): base = instruction.A argn = instruction.CD - 1 args = [] i = 0 while i < argn: args.append(_translate(writer, addr, base + i + 1, ins.T_VAR)) i += 1 func_var = _translate(writer, addr, base, ins.T_VAR) return description.format( A=func_var, from_A_plus_one_x_D_minus_one=", ".join(args) ) def _translate_iterator(writer, description, addr, line, instruction): base = instruction.A A = _translate(writer, addr, instruction.A, ins.T_DST) A_plus_one = _translate(writer, addr, instruction.A + 1, ins.T_DST) A_plus_two = _translate(writer, addr, instruction.A + 2, ins.T_DST) A_minus_three = _translate(writer, addr, instruction.A - 3, ins.T_VAR) A_minus_two = _translate(writer, addr, instruction.A - 2, ins.T_VAR) A_minus_one = _translate(writer, addr, instruction.A - 1, ins.T_VAR) retn = instruction.B - 1 returns = [] i = 0 while i < retn: returns.append(_translate(writer, addr, base + i, ins.T_DST)) i += 1 return description.format( A=A, A_plus_one=A_plus_one, A_plus_two=A_plus_two, A_minus_three=A_minus_three, A_minus_two=A_minus_two, A_minus_one=A_minus_one, from_A_x_B_minus_two=", ".join(returns) ) def _translate_vararg(writer, description, addr, line, instruction): returns = [] base = instruction.A count = instruction.B - 2 if count < 0: return description.format(from_A_x_B_minus_two="MULTRES") i = 0 while i <= count: returns.append(_translate(writer, addr, base + i, ins.T_DST)) i += 1 return description.format( from_A_x_B_minus_two=", ".join(returns) ) def _translate_return_mult(writer, description, addr, line, instruction): returns = [] base = instruction.A count = instruction.CD - 1 i = 0 while i < count: returns.append(_translate(writer, addr, base + i, ins.T_VAR)) i += 1 return description.format( from_A_x_D_minus_one=", ".join(returns) ) def _translate_return_many(writer, description, addr, line, instruction): returns = [] base = instruction.A count = instruction.CD - 2 i = 0 while i < count: returns.append(_translate(writer, addr, base + i, ins.T_VAR)) i += 1 return description.format( from_A_x_D_minus_two=", ".join(returns) ) def _translate_return_one(writer, description, addr, line, instruction): A = _translate(writer, addr, instruction.A, ins.T_VAR) return description.format(A=A) def _translate_for_init(writer, description, addr, line, instruction): idx = _translate(writer, addr, instruction.A, ins.T_BS) stop = _translate(writer, addr, instruction.A + 1, ins.T_BS) step = _translate(writer, addr, instruction.A + 2, ins.T_BS) ext_idx = _translate(writer, addr, instruction.A + 3, ins.T_VAR) return description.format( A=idx, A_plus_one=stop, A_plus_two=step, A_plus_three=ext_idx, D=_translate(writer, addr, instruction.CD, ins.T_JMP) ) def _translate_numeric_loop(writer, description, addr, line, instruction): stop = _translate(writer, addr, instruction.A + 1, ins.T_VAR) step = _translate(writer, addr, instruction.A + 2, ins.T_VAR) ext_idx = _translate(writer, addr, instruction.A + 3, ins.T_VAR) # ext_idx isn't correct var here, but for the visualisation sake we will # omit all the stuff with the internal idx var return description.format( A=ext_idx, A_plus_one=stop, A_plus_two=step, D=_translate(writer, addr, instruction.CD, ins.T_JMP) ) def _translate_iter_loop(writer, description, addr, line, instruction): A_minus_one = _translate(writer, addr, instruction.A - 1, ins.T_VAR) A = _translate(writer, addr, instruction.A, ins.T_VAR) return description.format( A_minus_one=A_minus_one, A=A, D=_translate(writer, addr, instruction.CD, ins.T_JMP) ) _HANDLERS_MAP = ( # Comparison ops (ins.ISLT.opcode, _translate_normal), (ins.ISGE.opcode, _translate_normal), (ins.ISLE.opcode, _translate_normal), (ins.ISGT.opcode, _translate_normal), (ins.ISEQV.opcode, _translate_normal), (ins.ISNEV.opcode, _translate_normal), (ins.ISEQS.opcode, _translate_normal), (ins.ISNES.opcode, _translate_normal), (ins.ISEQN.opcode, _translate_normal), (ins.ISNEN.opcode, _translate_normal), (ins.ISEQP.opcode, _translate_normal), (ins.ISNEP.opcode, _translate_normal), # Unary test and copy ops (ins.ISTC.opcode, _translate_normal), (ins.ISFC.opcode, _translate_normal), (ins.IST.opcode, _translate_normal), (ins.ISF.opcode, _translate_normal), # Unary ops (ins.MOV.opcode, _translate_normal), (ins.NOT.opcode, _translate_normal), (ins.UNM.opcode, _translate_normal), (ins.LEN.opcode, _translate_normal), # Binary ops (ins.ADDVN.opcode, _translate_normal), (ins.SUBVN.opcode, _translate_normal), (ins.MULVN.opcode, _translate_normal), (ins.DIVVN.opcode, _translate_normal), (ins.MODVN.opcode, _translate_normal), (ins.ADDNV.opcode, _translate_normal), (ins.SUBNV.opcode, _translate_normal), (ins.MULNV.opcode, _translate_normal), (ins.DIVNV.opcode, _translate_normal), (ins.MODNV.opcode, _translate_normal), (ins.ADDVV.opcode, _translate_normal), (ins.SUBVV.opcode, _translate_normal), (ins.MULVV.opcode, _translate_normal), (ins.DIVVV.opcode, _translate_normal), (ins.MODVV.opcode, _translate_normal), (ins.POW.opcode, _translate_normal), (ins.CAT.opcode, _translate_concat), # Constant ops (ins.KSTR.opcode, _translate_normal), (ins.KCDATA.opcode, _translate_normal), (ins.KSHORT.opcode, _translate_normal), (ins.KNUM.opcode, _translate_normal), (ins.KPRI.opcode, _translate_normal), (ins.KNIL.opcode, _translate_nil), # Upvalue and function ops (ins.UGET.opcode, _translate_normal), (ins.USETV.opcode, _translate_normal), (ins.USETS.opcode, _translate_normal), (ins.USETN.opcode, _translate_normal), (ins.USETP.opcode, _translate_normal), (ins.UCLO.opcode, _translate_normal), (ins.FNEW.opcode, _translate_normal), # Table ops (ins.TNEW.opcode, _translate_new_table), (ins.TDUP.opcode, _translate_normal), (ins.GGET.opcode, _translate_normal), (ins.GSET.opcode, _translate_normal), (ins.TGETV.opcode, _translate_normal), (ins.TGETS.opcode, _translate_table_str_op), (ins.TGETB.opcode, _translate_normal), (ins.TSETV.opcode, _translate_normal), (ins.TSETS.opcode, _translate_table_str_op), (ins.TSETB.opcode, _translate_normal), (ins.TSETM.opcode, _translate_mass_set), # Calls and vararg handling (ins.CALLM.opcode, _translate_varg_call), (ins.CALL.opcode, _translate_call), (ins.CALLMT.opcode, _translate_varg_tailcall), (ins.CALLT.opcode, _translate_tailcall), (ins.ITERC.opcode, _translate_iterator), (ins.ITERN.opcode, _translate_iterator), (ins.VARG.opcode, _translate_vararg), (ins.ISNEXT.opcode, _translate_normal), # Returns (ins.RETM.opcode, _translate_return_mult), (ins.RET.opcode, _translate_return_many), (ins.RET0.opcode, _translate_normal), (ins.RET1.opcode, _translate_return_one), # Loops and branches (ins.FORI.opcode, _translate_for_init), (ins.JFORI.opcode, _translate_for_init), (ins.FORL.opcode, _translate_numeric_loop), (ins.IFORL.opcode, _translate_numeric_loop), (ins.JFORL.opcode, _translate_numeric_loop), (ins.ITERL.opcode, _translate_iter_loop), (ins.IITERL.opcode, _translate_iter_loop), (ins.JITERL.opcode, _translate_iter_loop), (ins.LOOP.opcode, _translate_normal), (ins.ILOOP.opcode, _translate_normal), (ins.JLOOP.opcode, _translate_normal), (ins.JMP.opcode, _translate_normal), # Function headers (ins.FUNCF.opcode, _translate_normal), (ins.IFUNCF.opcode, _translate_normal), (ins.JFUNCF.opcode, _translate_normal), (ins.FUNCV.opcode, _translate_normal), (ins.IFUNCV.opcode, _translate_normal), (ins.JFUNCV.opcode, _translate_normal), (ins.FUNCC.opcode, _translate_normal), (ins.FUNCCW.opcode, _translate_normal) ) def _init(): global _HANDLERS_MAP, _DESCRIPTION_HANDLERS for opcode, handler in _HANDLERS_MAP: _DESCRIPTION_HANDLERS[opcode] = handler del globals()["_init"] del globals()["_HANDLERS_MAP"] _init() ================================================ FILE: ljd/pseudoasm/prototype.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # import ljd.pseudoasm.constants import ljd.pseudoasm.instructions def write(writer, prototype): _write_header(writer, prototype) write_body(writer, prototype) writer.stream.close_block("") def _write_header(writer, prototype): writer.stream.open_block("main {0}", format_header(writer, prototype)) def format_header(writer, prototype): return "{s}:{start}-{end}: {argn}{varg} args," \ " {uvs} upvalues, {slots} slots".format( s=writer.source, start=prototype.first_line_number, end=prototype.first_line_number + prototype.lines_count, argn=prototype.arguments_count, varg="+" if prototype.flags.is_variadic else "", uvs=len(prototype.constants.upvalue_references), slots=prototype.framesize ) def write_body(writer, prototype): writer.stream.write_line(";;;; constant tables ;;;;") ljd.pseudoasm.constants.write_tables(writer, prototype) writer.stream.write_line(";;;; instructions ;;;;") ljd.pseudoasm.instructions.write(writer, prototype) ================================================ FILE: ljd/pseudoasm/writer.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # import ljd.util.indentedstream import ljd.pseudoasm.prototype class _State(): def __init__(self): self.flags = None self.stream = None self.source = None def write(fd, header, prototype): writer = _State() writer.stream = ljd.util.indentedstream.IndentedStream(fd) writer.flags = header.flags writer.source = "N/A" if header.flags.is_stripped else header.name _write_header(writer, header) ljd.pseudoasm.prototype.write(writer, prototype) def _write_header(writer, header): writer.stream.write_multiline(""" ; ; Disassemble of {origin} ; ; Source file: {source} ; ; Flags: ; Stripped: {stripped} ; Endianness: {endianness} ; FFI: {ffi} ; """, origin=header.origin, source=writer.source, stripped="Yes" if header.flags.is_stripped else "No", endianness="Big" if header.flags.is_big_endian else "Little", ffi="Present" if header.flags.has_ffi else "Not present") ================================================ FILE: ljd/rawdump/__init__.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # ================================================ FILE: ljd/rawdump/code.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # from ljd.util.log import errprint import ljd.bytecode.instructions as instructions _OPCODES = ( # Comparison ops (0x00, instructions.ISLT), # @UndefinedVariable (0x01, instructions.ISGE), # @UndefinedVariable (0x02, instructions.ISLE), # @UndefinedVariable (0x03, instructions.ISGT), # @UndefinedVariable (0x04, instructions.ISEQV), # @UndefinedVariable (0x05, instructions.ISNEV), # @UndefinedVariable (0x06, instructions.ISEQS), # @UndefinedVariable (0x07, instructions.ISNES), # @UndefinedVariable (0x08, instructions.ISEQN), # @UndefinedVariable (0x09, instructions.ISNEN), # @UndefinedVariable (0x0A, instructions.ISEQP), # @UndefinedVariable (0x0B, instructions.ISNEP), # @UndefinedVariable # Unary test and copy ops (0x0C, instructions.ISTC), # @UndefinedVariable (0x0D, instructions.ISFC), # @UndefinedVariable (0x0E, instructions.IST), # @UndefinedVariable (0x0F, instructions.ISF), # @UndefinedVariable (0x0E, instructions.ISTYPE), # @UndefinedVariable (0x0F, instructions.ISNUM), # @UndefinedVariable # Unary ops (0x10, instructions.MOV), # @UndefinedVariable (0x11, instructions.NOT), # @UndefinedVariable (0x12, instructions.UNM), # @UndefinedVariable (0x13, instructions.LEN), # @UndefinedVariable # Binary ops (0x14, instructions.ADDVN), # @UndefinedVariable (0x15, instructions.SUBVN), # @UndefinedVariable (0x16, instructions.MULVN), # @UndefinedVariable (0x17, instructions.DIVVN), # @UndefinedVariable (0x18, instructions.MODVN), # @UndefinedVariable (0x19, instructions.ADDNV), # @UndefinedVariable (0x1A, instructions.SUBNV), # @UndefinedVariable (0x1B, instructions.MULNV), # @UndefinedVariable (0x1C, instructions.DIVNV), # @UndefinedVariable (0x1D, instructions.MODNV), # @UndefinedVariable (0x1E, instructions.ADDVV), # @UndefinedVariable (0x1F, instructions.SUBVV), # @UndefinedVariable (0x20, instructions.MULVV), # @UndefinedVariable (0x21, instructions.DIVVV), # @UndefinedVariable (0x22, instructions.MODVV), # @UndefinedVariable (0x23, instructions.POW), # @UndefinedVariable (0x24, instructions.CAT), # @UndefinedVariable # Constant ops (0x25, instructions.KSTR), # @UndefinedVariable (0x26, instructions.KCDATA), # @UndefinedVariable (0x27, instructions.KSHORT), # @UndefinedVariable (0x28, instructions.KNUM), # @UndefinedVariable (0x29, instructions.KPRI), # @UndefinedVariable (0x2A, instructions.KNIL), # @UndefinedVariable # Upvalue and function ops (0x2B, instructions.UGET), # @UndefinedVariable (0x2C, instructions.USETV), # @UndefinedVariable (0x2D, instructions.USETS), # @UndefinedVariable (0x2E, instructions.USETN), # @UndefinedVariable (0x2F, instructions.USETP), # @UndefinedVariable (0x30, instructions.UCLO), # @UndefinedVariable (0x31, instructions.FNEW), # @UndefinedVariable # Table ops (0x32, instructions.TNEW), # @UndefinedVariable (0x33, instructions.TDUP), # @UndefinedVariable (0x34, instructions.GGET), # @UndefinedVariable (0x35, instructions.GSET), # @UndefinedVariable (0x36, instructions.TGETV), # @UndefinedVariable (0x37, instructions.TGETS), # @UndefinedVariable (0x38, instructions.TGETB), # @UndefinedVariable (0x38, instructions.TGETR), # @UndefinedVariable (0x39, instructions.TSETV), # @UndefinedVariable (0x3A, instructions.TSETS), # @UndefinedVariable (0x3B, instructions.TSETB), # @UndefinedVariable (0x3C, instructions.TSETM), # @UndefinedVariable (0x3B, instructions.TSETR), # @UndefinedVariable # Calls and vararg handling (0x3D, instructions.CALLM), # @UndefinedVariable (0x3E, instructions.CALL), # @UndefinedVariable (0x3F, instructions.CALLMT), # @UndefinedVariable (0x40, instructions.CALLT), # @UndefinedVariable (0x41, instructions.ITERC), # @UndefinedVariable (0x42, instructions.ITERN), # @UndefinedVariable (0x43, instructions.VARG), # @UndefinedVariable (0x44, instructions.ISNEXT), # @UndefinedVariable # Returns (0x45, instructions.RETM), # @UndefinedVariable (0x46, instructions.RET), # @UndefinedVariable (0x47, instructions.RET0), # @UndefinedVariable (0x48, instructions.RET1), # @UndefinedVariable # Loops and branches (0x49, instructions.FORI), # @UndefinedVariable (0x4A, instructions.JFORI), # @UndefinedVariable (0x4B, instructions.FORL), # @UndefinedVariable (0x4C, instructions.IFORL), # @UndefinedVariable (0x4D, instructions.JFORL), # @UndefinedVariable (0x4E, instructions.ITERL), # @UndefinedVariable (0x4F, instructions.IITERL), # @UndefinedVariable (0x50, instructions.JITERL), # @UndefinedVariable (0x51, instructions.LOOP), # @UndefinedVariable (0x52, instructions.ILOOP), # @UndefinedVariable (0x53, instructions.JLOOP), # @UndefinedVariable (0x54, instructions.JMP), # @UndefinedVariable # Function headers (0x55, instructions.FUNCF), # @UndefinedVariable (0x56, instructions.IFUNCF), # @UndefinedVariable (0x57, instructions.JFUNCF), # @UndefinedVariable (0x58, instructions.FUNCV), # @UndefinedVariable (0x59, instructions.IFUNCV), # @UndefinedVariable (0x5A, instructions.JFUNCV), # @UndefinedVariable (0x5B, instructions.FUNCC), # @UndefinedVariable (0x5C, instructions.FUNCCW) # @UndefinedVariable ) _MAP = [None] * 256 def read(parser): global _MAP codeword = parser.stream.read_uint(4) opcode = codeword & 0xFF instruction_class = _MAP[opcode] if instruction_class is None: errprint("Warning: unknown opcode {0:08x}", opcode) instruction_class = instructions.UNKNW # @UndefinedVariable #zzw.20180714 instruction = instruction_class() if instruction_class.opcode != opcode: instruction.opcode = opcode _set_instruction_operands(parser, codeword, instruction) return instruction def _set_instruction_operands(parser, codeword, instruction): if instruction.args_count == 3: A = (codeword >> 8) & 0xFF CD = (codeword >> 16) & 0xFF B = (codeword >> 24) & 0xFF else: A = (codeword >> 8) & 0xFF CD = (codeword >> 16) & 0xFFFF if instruction.A_type is not None: instruction.A = _process_operand(parser, instruction.A_type, A) if instruction.B_type is not None: instruction.B = _process_operand(parser, instruction.B_type, B) if instruction.CD_type is not None: instruction.CD = _process_operand(parser, instruction.CD_type, CD) def _process_operand(parser, operand_type, operand): if operand_type == instructions.T_STR \ or operand_type == instructions.T_TAB \ or operand_type == instructions.T_FUN \ or operand_type == instructions.T_CDT: return parser.complex_constants_count - operand - 1 elif operand_type == instructions.T_JMP: return operand - 0x8000 else: return operand def _init(): global _OPCODES, _MAP opcode = 0 for instruction in _OPCODES: _MAP[opcode] = instruction[1] opcode = opcode + 1 del globals()["_init"] del globals()["_OPCODES"] _init() ================================================ FILE: ljd/rawdump/constants.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # import sys import struct import ljd.bytecode.constants import gconfig BCDUMP_KGC_CHILD = 0 BCDUMP_KGC_TAB = 1 BCDUMP_KGC_I64 = 2 BCDUMP_KGC_U64 = 3 BCDUMP_KGC_COMPLEX = 4 BCDUMP_KGC_STR = 5 BCDUMP_KTAB_NIL = 0 BCDUMP_KTAB_FALSE = 1 BCDUMP_KTAB_TRUE = 2 BCDUMP_KTAB_INT = 3 BCDUMP_KTAB_NUM = 4 BCDUMP_KTAB_STR = 5 def read(parser, constants): r = True r = r and _read_upvalue_references(parser, constants.upvalue_references) r = r and _read_complex_constants(parser, constants.complex_constants) r = r and _read_numeric_constants(parser, constants.numeric_constants) return r def _read_upvalue_references(parser, references): i = 0 while i < parser.upvalues_count: i += 1 upvalue = parser.stream.read_uint(2) references.append(upvalue) return True def _read_complex_constants(parser, complex_constants): i = 0 while i < parser.complex_constants_count: constant_type = parser.stream.read_uleb128() if constant_type >= BCDUMP_KGC_STR: length = constant_type - BCDUMP_KGC_STR string = parser.stream.read_bytes(length) #print (string.decode("unicode-escape")) #print(str(complex_constants)) #zzw 20180714 support str encode complex_constants.append(string.decode(gconfig.gFlagDic['strEncode'])) elif constant_type == BCDUMP_KGC_TAB: table = ljd.bytecode.constants.Table() if not _read_table(parser, table): return False complex_constants.append(table) elif constant_type != BCDUMP_KGC_CHILD: number = _read_number(parser) if constant_type == BCDUMP_KGC_COMPLEX: imaginary = _read_number(parser) complex_constants.append((number, imaginary)) else: complex_constants.append(number) else: complex_constants.append(parser.prototypes.pop()) i += 1 return True def _read_numeric_constants(parser, numeric_constants): i = 0 while i < parser.numeric_constants_count: isnum, lo = parser.stream.read_uleb128_from33bit() if isnum: hi = parser.stream.read_uleb128() number = _assemble_number(lo, hi) else: number = _process_sign(lo) numeric_constants.append(number) i += 1 return True def _read_number(parser): lo = parser.stream.read_uleb128() hi = parser.stream.read_uleb128() return _assemble_number(lo, hi) def _read_signed_int(parser): number = parser.stream.read_uleb128() return _process_sign(number) def _assemble_number(lo, hi): if sys.byteorder == 'big': float_as_int = lo << 32 | hi else: float_as_int = hi << 32 | lo raw_bytes = struct.pack("=Q", float_as_int) return struct.unpack("=d", raw_bytes)[0] def _process_sign(number): if number & 0x80000000: return -0x100000000 + number else: return number def _read_table(parser, table): array_items_count = parser.stream.read_uleb128() hash_items_count = parser.stream.read_uleb128() while array_items_count > 0: constant = _read_table_item(parser) table.array.append(constant) array_items_count -= 1 while hash_items_count > 0: key = _read_table_item(parser) value = _read_table_item(parser) table.dictionary.append((key, value)) hash_items_count -= 1 return True def _read_table_item(parser): data_type = parser.stream.read_uleb128() if data_type >= BCDUMP_KTAB_STR: length = data_type - BCDUMP_KTAB_STR # zzw 20180714 support str encode return parser.stream.read_bytes(length).decode(gconfig.gFlagDic['strEncode']) elif data_type == BCDUMP_KTAB_INT: return _read_signed_int(parser) elif data_type == BCDUMP_KTAB_NUM: return _read_number(parser) elif data_type == BCDUMP_KTAB_TRUE: return True elif data_type == BCDUMP_KTAB_FALSE: return False else: assert data_type == BCDUMP_KTAB_NIL return None ================================================ FILE: ljd/rawdump/debuginfo.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # import sys import ljd.bytecode.debuginfo VARNAME_END = 0 VARNAME_FOR_IDX = 1 VARNAME_FOR_STOP = 2 VARNAME_FOR_STEP = 3 VARNAME_FOR_GEN = 4 VARNAME_FOR_STATE = 5 VARNAME_FOR_CTL = 6 VARNAME__MAX = 7 INTERNAL_VARNAMES = [ None, "", "", "", "", "", "" ] def read(parser, line_offset, debuginfo): r = True r = r and _read_lineinfo(parser, line_offset, debuginfo.addr_to_line_map) r = r and _read_upvalue_names(parser, debuginfo.upvalue_variable_names) r = r and _read_variable_infos(parser, debuginfo.variable_info) return r def _read_lineinfo(parser, line_offset, lineinfo): if parser.lines_count >= 65536: lineinfo_size = 4 elif parser.lines_count >= 256: lineinfo_size = 2 else: lineinfo_size = 1 lineinfo.append(0) while len(lineinfo) < parser.instructions_count + 1: line_number = parser.stream.read_uint(lineinfo_size) lineinfo.append(line_offset + line_number) return True def _read_upvalue_names(parser, names): while len(names) < parser.upvalues_count: string = parser.stream.read_zstring() names.append(string.decode("utf-8")) return True def _read_variable_infos(parser, infos): # pc - program counter last_addr = 0 while True: info = ljd.bytecode.debuginfo.VariableInfo() internal_vartype = parser.stream.read_byte() if internal_vartype >= VARNAME__MAX: prefix = internal_vartype.to_bytes(1, sys.byteorder) suffix = parser.stream.read_zstring() info.name = (prefix + suffix).decode("utf-8") info.type = info.T_VISIBILE elif internal_vartype == VARNAME_END: break else: index = internal_vartype info.name = INTERNAL_VARNAMES[index] info.type = info.T_INTERNAL start_addr = last_addr + parser.stream.read_uleb128() end_addr = start_addr + parser.stream.read_uleb128() info.start_addr = start_addr info.end_addr = end_addr last_addr = start_addr infos.append(info) return True ================================================ FILE: ljd/rawdump/header.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # from ljd.util.log import errprint _MAGIC = b'\x1bLJ' _MAX_VERSION = 0x80 _FLAG_IS_BIG_ENDIAN = 0b00000001 _FLAG_IS_STRIPPED = 0b00000010 _FLAG_HAS_FFI = 0b00000100 class Flags(): def __init__(self): self.is_big_endian = False self.is_stripped = False self.has_ffi = False class Header(): def __init__(self): self.version = 0 self.flags = Flags() self.origin = b'' self.name = b'' def read(state, header): r = True header.origin = state.stream.name r = r and _check_magic(state) r = r and _read_version(state, header) r = r and _read_flags(state, header) r = r and _read_name(state, header) #print("header good!") return r def _check_magic(state): if state.stream.read_bytes(3) != _MAGIC: errprint("Invalid magic, not a LuaJIT format") return False return True def _read_version(state, header): header.version = state.stream.read_byte() if header.version > _MAX_VERSION: errprint("Version {0}: propritary modifications", header.version) return False return True def _read_flags(parser, header): bits = parser.stream.read_uleb128() header.flags.is_big_endian = bits & _FLAG_IS_BIG_ENDIAN bits &= ~_FLAG_IS_BIG_ENDIAN header.flags.is_stripped = bits & _FLAG_IS_STRIPPED bits &= ~_FLAG_IS_STRIPPED header.flags.has_ffi = bits & _FLAG_HAS_FFI bits &= ~_FLAG_HAS_FFI # zzw.20180714 pitch: flag value is according to parser.flag when parse proto, not by header.flags parser.flags.is_big_endian = header.flags.is_big_endian parser.flags.is_stripped = header.flags.is_stripped parser.flags.is_big_endian = header.flags.has_ffi if bits != 0: errprint("Unknown flags set: {0:08b}", bits) return False return True def _read_name(state, header): if header.flags.is_stripped: header.name = state.stream.name else: length = state.stream.read_uleb128() header.name = state.stream.read_bytes(length).decode("utf8") return True ================================================ FILE: ljd/rawdump/parser.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # #!/usr/bin/python3 import ljd.util.binstream from ljd.util.log import errprint import ljd.bytecode.prototype import ljd.rawdump.header import ljd.rawdump.prototype class _State(): def __init__(self): self.stream = ljd.util.binstream.BinStream() self.flags = ljd.rawdump.header.Flags() self.prototypes = [] def parse(filename): parser = _State() parser.stream.open(filename) header = ljd.rawdump.header.Header() r = True try: r = r and _read_header(parser, header) #print("good1") r = r and _read_prototypes(parser, parser.prototypes) #print("good2") except IOError as e: errprint("I/O error while reading dump: {0}", str(e)) r = False if r and not parser.stream.eof(): errprint("Stopped before whole file was read, something wrong") r = False if r and len(parser.prototypes) != 1: errprint("Invalid prototypes stack order") r = False parser.stream.close() if r: return header, parser.prototypes[0] else: return None, None #zzw 20180716 解析字节码文件头部结构header ''' | magic(3 bytes)| version(1 byte) | flag(1 uleb128) [| name(1 uleb128) |] ''' def _read_header(parser, header): if not ljd.rawdump.header.read(parser, header): errprint("Failed to read raw-dump header") return False if header.flags.is_big_endian: parser.stream.data_byteorder = 'big' else: parser.stream.data_byteorder = 'little' return True def _read_prototypes(state, prototypes): while not state.stream.eof(): prototype = ljd.bytecode.prototype.Prototype() #print ("good rwsdump->parser->read_prototypes") if not ljd.rawdump.prototype.read(state, prototype): if state.stream.eof(): break else: errprint("Failed to read prototype") return False prototypes.append(prototype) return True ================================================ FILE: ljd/rawdump/prototype.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # from ljd.util.log import errprint import ljd.bytecode.instructions as ins import ljd.rawdump.constants import ljd.rawdump.debuginfo import ljd.rawdump.code FLAG_HAS_CHILD = 0b00000001 FLAG_IS_VARIADIC = 0b00000010 FLAG_HAS_FFI = 0b00000100 FLAG_JIT_DISABLED = 0b00001000 FLAG_HAS_ILOOP = 0b00010000 class _State(): def __init__(self, parser): for key, value in parser.__dict__.items(): setattr(self, key, value) self.upvalues_count = 0 self.complex_constants_count = 0 self.numeric_constants_count = 0 self.instructions_count = 0 self.debuginfo_size = 0 ''' zzw 20180716 | size(1 uleb128) | flag(1 byte) | arguments_count(1 byte) | framesize(1 byte) | upvalues_count(1 byte) | complex_constants_count(1 uleb128) | numeric_constants_count(1 uleb128) | instructions_count(1 uleb128) [| debuginfo_size(1 uleb128) | first_line_number(1 uleb128) | lines_count(1 uleb128) ] | instructions(protoheader define) | constants(protoheader define) | debginfo(protoheader flage define) | ''' def read(parser, prototype): parser = _State(parser) size = parser.stream.read_uleb128() if size == 0: return False if not parser.stream.check_data_available(size): errprint("File truncated") return False start = parser.stream.pos r = True r = r and _read_flags(parser, prototype) r = r and _read_counts_and_sizes(parser, prototype) r = r and _read_instructions(parser, prototype) r = r and _read_constants(parser, prototype) r = r and _read_debuginfo(parser, prototype) end = parser.stream.pos if r: assert end - start == size, \ "Incorrectly read: from {0} to {1} ({2}) instead of {3}"\ .format(start, end, end - start, size) return r def _read_flags(parser, prototype): bits = parser.stream.read_byte() prototype.flags.has_ffi = bool(bits & FLAG_HAS_FFI) bits &= ~FLAG_HAS_FFI prototype.flags.has_iloop = bool(bits & FLAG_HAS_ILOOP) bits &= ~FLAG_HAS_ILOOP prototype.flags.has_jit = not (bits & FLAG_JIT_DISABLED) bits &= ~FLAG_JIT_DISABLED prototype.flags.has_sub_prototypes = bool(bits & FLAG_HAS_CHILD) bits &= ~FLAG_HAS_CHILD prototype.flags.is_variadic = bool(bits & FLAG_IS_VARIADIC) bits &= ~FLAG_IS_VARIADIC if bits != 0: errprint("Unknown prototype flags: {0:08b}", bits) return False return True def _read_counts_and_sizes(parser, prototype): prototype.arguments_count = parser.stream.read_byte() prototype.framesize = parser.stream.read_byte() parser.upvalues_count = parser.stream.read_byte() parser.complex_constants_count = parser.stream.read_uleb128() parser.numeric_constants_count = parser.stream.read_uleb128() parser.instructions_count = parser.stream.read_uleb128() if parser.flags.is_stripped: parser.debuginfo_size = 0 else: parser.debuginfo_size = parser.stream.read_uleb128() if parser.debuginfo_size == 0: return True prototype.first_line_number = parser.stream.read_uleb128() prototype.lines_count = parser.stream.read_uleb128() parser.lines_count = prototype.lines_count return True def _read_instructions(parser, prototype): i = 0 if prototype.flags.is_variadic: header = ins.FUNCV() else: header = ins.FUNCF() header.A = prototype.framesize prototype.instructions.append(header) while i < parser.instructions_count: instruction = ljd.rawdump.code.read(parser) if not instruction: return False #print ("inst：%s" % instruction.name) #print ("opcode:%x" % instruction.opcode) #print ("A:%x" % instruction.A) #print ("B:%x" % instruction.B) #print ("CD:%x" % instruction.CD) prototype.instructions.append(instruction) i += 1 return True def _read_constants(parser, prototype): return ljd.rawdump.constants.read(parser, prototype.constants) def _read_debuginfo(stream, prototype): if stream.debuginfo_size == 0: return True return ljd.rawdump.debuginfo.read(stream, prototype.first_line_number, prototype.debuginfo) ================================================ FILE: ljd/util/__init__.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # ================================================ FILE: ljd/util/binstream.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # import io import os import sys class BinStream(): def __init__(self): self.fd = None self.size = 0 self.pos = 0 self.name = "" self.data_byteorder = sys.byteorder def open(self, filename): self.name = filename self.fd = io.open(filename, 'rb') self.size = os.stat(filename).st_size def close(self): self.fd.close() self.size = 0 self.pos = 0 def eof(self): return self.pos >= self.size def check_data_available(self, size=1): return self.pos + size <= self.size def read_bytes(self, size=1): if not self.check_data_available(size): raise IOError("Unexpected EOF while trying to read {0} bytes" .format(size)) data = self.fd.read(size) self.pos += len(data)#size return data def read_byte(self): if not self.check_data_available(1): raise IOError("Unexpected EOF while trying to read 1 byte") data = self.fd.read(1) self.pos += 1 return int.from_bytes(data, byteorder=sys.byteorder, signed=False) def read_zstring(self): string = b'' while not self.eof(): byte = self.read_bytes(1) if byte == b'\x00': return string else: string += byte return string def read_uleb128(self): value = self.read_byte() if value >= 0x80: bitshift = 0 value &= 0x7f while True: byte = self.read_byte() bitshift += 7 value |= (byte & 0x7f) << bitshift if byte < 0x80: break return value def read_uleb128_from33bit(self): first_byte = self.read_byte() is_number_bit = first_byte & 0x1 value = first_byte >> 1 if value >= 0x40: bitshift = -1 value &= 0x3f while True: byte = self.read_byte() bitshift += 7 value |= (byte & 0x7f) << bitshift if byte < 0x80: break return is_number_bit, value def read_uint(self, size=4): value = self.read_bytes(size) return int.from_bytes(value, byteorder=self.data_byteorder, signed=False) ================================================ FILE: ljd/util/indentedstream.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # _TAB_WIDTH = " " * 8 class IndentedStream(): def __init__(self, fd): self.fd = fd self.indent = 0 self.line_open = False def write_multiline(self, fmt, *args, **kargs): assert not self.line_open if len(args) + len(kargs) > 0: text = fmt.format(*args, **kargs) else: text = fmt lines = text.split("\n") if lines[0] == "": lines.pop(0) if lines[-1] == "": lines.pop(-1) spaces = "\t" * self.indent for line in lines: self.fd.write(spaces + line + "\n") def start_line(self): assert not self.line_open self.line_open = True self.fd.write("\t" * self.indent) def write(self, fmt="", *args, **kargs): assert self.line_open if len(args) + len(kargs) > 0: text = fmt.format(*args, **kargs) elif isinstance(fmt, str): text = fmt else: text = str(fmt) assert "\n" not in text self.fd.write(text) def end_line(self): assert self.line_open self.fd.write("\n") self.line_open = False def write_line(self, *args, **kargs): self.start_line() self.write(*args, **kargs) self.end_line() def open_block(self, *args, **kargs): if len(args) + len(kargs) > 0: self.write_line(*args, **kargs) self.indent += 1 def close_block(self, *args, **kargs): if len(args) + len(kargs) > 0: self.write_line(*args, **kargs) self.indent -= 1 ================================================ FILE: ljd/util/log.py ================================================ # # Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py # import sys def errprint(*args): fmt = None args = list(args) if isinstance(args[0], str): fmt = args.pop(0) if fmt: print(fmt.format(*args), file=sys.stderr) else: strs = [repr(x) for x in args] print(" ".join(strs), file=sys.stderr) ================================================ FILE: main.py ================================================ #!/usr/bin/python3 # # The MIT License (MIT) # # Copyright (c) 2013 Andrian Nord # # Permission is hereby granted, free of charge, to any person obtaining a copy # of this software and associated documentation files (the "Software"), to deal # in the Software without restriction, including without limitation the rights # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell # copies of the Software, and to permit persons to whom the Software is # furnished to do so, subject to the following conditions: # # The above copyright notice and this permission notice shall be included in # all copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. # import sys import ljd.rawdump.parser import ljd.pseudoasm.writer import ljd.ast.builder import ljd.ast.validator import ljd.ast.locals import ljd.ast.slotworks import ljd.ast.unwarper import ljd.ast.mutator import ljd.lua.writer #zzw 20180714 support str encode import gconfig def dump(name, obj, level=0): indent = level * '\t' if name is not None: prefix = indent + name + " = " else: prefix = indent if isinstance(obj, (int, float, str)): print(prefix + str(obj)) elif isinstance(obj, list): print (prefix + "[") for value in obj: dump(None, value, level + 1) print (indent + "]") elif isinstance(obj, dict): print (prefix + "{") for key, value in obj.items(): dump(key, value, level + 1) print (indent + "}") else: print (prefix + obj.__class__.__name__) for key in dir(obj): if key.startswith("__"): continue val = getattr(obj, key) dump(key, val, level + 1) def main(): file_in = sys.argv[1] header, prototype = ljd.rawdump.parser.parse(file_in) #print ("good") if not prototype: return 1 # TODO: args # ljd.pseudoasm.writer.write(sys.stdout, header, prototype) ast = ljd.ast.builder.build(prototype) assert ast is not None ljd.ast.validator.validate(ast, warped=True) ljd.ast.mutator.pre_pass(ast) # ljd.ast.validator.validate(ast, warped=True) ljd.ast.locals.mark_locals(ast) # ljd.ast.validator.validate(ast, warped=True) ljd.ast.slotworks.eliminate_temporary(ast) # ljd.ast.validator.validate(ast, warped=True) if True: ljd.ast.unwarper.unwarp(ast) # ljd.ast.validator.validate(ast, warped=False) if True: ljd.ast.locals.mark_local_definitions(ast) # ljd.ast.validator.validate(ast, warped=False) ljd.ast.mutator.primary_pass(ast) ljd.ast.validator.validate(ast, warped=False) ljd.lua.writer.write(sys.stdout, ast) return 0 if __name__ == "__main__": # zzw 20180714 support str encode gconfig.gFlagDic['strEncode'] = 'utf-8' retval = main() sys.exit(retval) # vim: ts=8 noexpandtab nosmarttab softtabstop=8 shiftwidth=8 ================================================ FILE: test/breaks.lua ================================================ --[[ --]] for i=1,2,3 do if x and y then print("Then") else break end end for i=1,2,3 do if x then print("Then") else break end print ("Something") if y then break end end for i=1,2,3 do if x then print("Then") else if y then break end end print ("Something") if y then break end end for i=1,2,3 do if x then print("Then") if y then break end else break end end for i=1,2,3 do if x then print("Then") if y then print ("Nested then") else break end else break end end for i=1,2,3 do if x then print("Then") else break end if y then print ("Y then") else break end end print ("Too bad") for i=1,2,3 do if x then print("Then") if y then print ("Y then") if z then print ("Z then") end end break end if y then print ("Y then") else break end end for i=1,2,3 do if x then if y then if z then if xi then print ("Xi then") end end end break else break end if y then print ("Y then") else break end end for i=1,2,3 do if x and y and z and xi then print ("Xi then") break else break end if y then print ("Y then") else break end end for i=1,2,3 do if x then print ("X then") if y then print ("Y then") if z then print ("Z then") if xi then print ("Xi then") end end end break else break end if y then print ("Y then") else break end end for i=1,2,3 do if x then if z then print("Z Then") else break end print ("Something") if y then print ("Y then") else break end else break end end for i=1,2,3 do if x then if z then print("Z Then") else break end print ("Something") else break end if y then print ("Y then") else break end end function updateAnimation() local currentTime = getElapsedTime() local startSequence = private.currentSequence + 1 local endSequence = #private.sequence for i=startSequence, endSequence, 1 do local entry = private.sequence[i] if entry.time <= currentTime then if entry.command == "start" then goToSlide(entry.element, 2) goToTime(entry.element, entry.animationTime) play(entry.element) else if entry.command == "step" then goToSlide(entry.element, 2) goToTime(entry.element, entry.animationTime) else pause(entry.element) end end if entry.name ~= nil then setAttribute(getElement("name", entry.element), "textstring", entry.name) end private.currentSequence = i break else break end end if #private.sequence == private.currentSequence then private.playing = false end return end --[[ --]] ================================================ FILE: test/expression.lua ================================================ --[[ --]] print ("true or true") b = true or true print ("false and false") b = false and false print ("false and or") b = false and (x or y) print ("false and ((and) or)") b = false and ((x and z) or y) print ("precalculated true expression") c = true or (x and y) or true print ("precalculated false expression") d = false and ((x and y) or true) print ("precalculated false expression with function") e = error() and false and ((x and y) or true) print ("precalculated true expression with function") e = error() and true and ((x and y) or true) print ("precalculated? false expression with variable") local z = false f = z and ((x and y) or true) print ("precalculated false expression with nil") f = nil and ((x and y) or true) print ("simple or expression") b = x or y print ("simple or not expression") b = not x or y print ("simple and expression") b = x and y print ("simple or expression with binary comparison") b = (x < 100) or y print ("simple and expression with binary comparison") b = (x < 100) and y print ("simple and expression with binary comparison and function call") b = (x < 100) and print(y) print ("simple and expression with double binary comparison") b = (x < 100) and (y > 100) print ("(and) or expression") b = (x and y) or z print ("(or) and expression") b = (x or y) and z print ("(and) and expression") b = (x and y) and z print ("(or) or expression") b = (x or y) or z print ("or (and) expression") b = x or (y and x) b = x < 100 or (y < 100 and x < 100) print ("and (or) expression") b = x and (y or x) print ("and (and) expression") b = x and (y and x) print ("or (or) expression") b = x or (y or x) print ("ond (or) and expression") b = x and (y or x) and z print ("or (and) or expression") b = x or (y and x) or z print ("and of two ors") b = (x or z) and (y or z) print ("or of two ands") b = (x and z) or (y and z) print ("x or string") local xi = "nothing" xi = x or "something" local xi print ("x and string") xi = x and "something" print ("and (or) and (or) expression with comparisons") b = x < 100 and (y < 100 or x < 100) and (z < 100 or x < 100) print ("and (or) and or or expression with comparisons") b = x < 100 and (y < 100 or x < 100) and z < 100 or x < 100 or y < 100 print ("and (or) and and and expression with comparisons") b = x < 100 and (y < 100 or x < 100) and z < 100 and x < 100 and y < 100 print ("or (and) or (and) expression with comparisons") b = x < 100 or (y < 100 and x < 100) or (z < 100 and x < 100) print ("and (and) and (and) expression with comparisons") b = x < 100 and (y < 100 and x < 100) and (z < 100 and x < 100) print ("or (or) or (or) expression with comparisons") b = x < 100 or (y < 100 or x < 100) or (z < 100 or x < 100) print ("4 and expression with comparisons") b = x < 100 and y < 100 and x < 100 and z < 100 and x < 100 print ("4 or expression with comparisons") b = x < 100 or y < 100 or x < 100 or z < 100 or x < 100 print ("and (or or) and (or or) expression with comparisons") b = x < 100 and (y < 100 or x < 100 or z < 100) and (y < 100 or x < 100 or z < 100) print ("and (or and or) and (or and or) expression with comparisons") b = x < 100 and (y < 100 or (x < 100 and x > 100) or z < 100) and (y < 100 or (x < 100 and x > 100) or z < 100) print ("or (and or and) or (and or and) expression with comparisons") b = x < 100 or (y < 100 and (x < 100 or x > 100) and z < 100) or (y < 100 and (x < 100 or x > 100) and z < 100) print ("(((or) and) or)") a = (((x < 100 or y < 100) and x < 100) or z < 100) print ("(((or or) and) or)") a = ((x < 100 or y < 100 or z < 100) and x < 100 or z < 100) print ("(((or and) and) or)") a = (x < 100 or y < 100 and z < 100) and x < 100 or z < 100 print ("(((or and) and) or) and error()") a = (((x < 100 or y < 100 and z < 100) and x < 100) or z < 100) and error() print ("(or (and (or)))") a = x < 100 or (y < 100 and (x < 100 or z < 100)) print ("(not or (and (or)))") a = (not (x < 100)) or (y < 100 and (x < 100 or z < 100)) local value = 1.0 value = scaleinfo.floorValue and math.floor(value) or math.ceil(value) local function foo(a) print(a) end local x = "" foo(x == "" and x or "test") local timeout = (menu.isOffer and (duration or -1)) or ((timeout and timeout ~= -1) and timeout or missiontime or -1) local a = x < 100 local exists = 0 exists = ffi.string(messageDetails.messageType) ~= "" local row, cells, rowdata, colspans, noscaling, bgColor if is_magic then row = foo(bgColor or Helper.defaultArrowRowBackgroundColor) else row = bar(bgColor) end local a = 0 a = z == 3 and (x < ((y == 0 and is_magic) and 3 or 2)) and "a" or "b" local a, is_magic, x, y, foo, bar a = is_magic and foo(x == "station" and y) or bar() local a = ( ((not commander and true or IsSameComponent(commander, menu.playership)) and "") or " [" .. ReadText(1001, 1001) .. "]" ) local a = isfirst and foo(bar((table[trade.ware] and "-") or "+", 1 < x)) or "" local a, x, y, is_magic a = (not x and ( (x < 50 and x < 100 and z) or (is_magic and 4 or 3) )) or x setElementPosition(iconelement, x, y, width%2 ~= 0, height%2 ~= 0) if xi then local unlocked_defence_status = x or ((y or z) < 100) bar(x and y or z) end local function foo() return { font = x or y, fontsize = x or y } end menu.logbook = foo(x or y) or {} local x, y, bar, do_not_collapse_this_if_please, a if x then if y then a = { x or y, bar(x or y) .. x or "" } end do_not_collapse_this_if_please() end --[[ --]] ================================================ FILE: test/ifs.lua ================================================ --[[ --]] if true then print("if true") end print ("something") if false then print("if false") end print ("something") if x then print ("something after") end if x and false then print("if and false") end if x then if false then print("if and false") end end if x then if false then print("if and false") else print("if and false with else!") end end if x then if false then print("if and false") else print("if and false with else!") end else print ("Else!") end if x and true then print("if and true") end if x or false then print("if or false") end if x or true then print("if or true") end if x < 100 then print ("then") else print ("else") end if x < 100 and y < 100 then print ("semi-nested then") else print ("else") end if x < 100 or z > 100 then if y < 100 or x > 100 then print ("Nested then") end end if x < 100 or z > 100 then if y < 100 or x > 100 then print ("Nested then") end print ("Enclosure") end if x < 100 or z > 100 then if y < 100 or x > 100 then print ("Nested then") else print ("Nested else") end print ("Enclosure") end if x < 100 or z > 100 then if y < 100 or x > 100 then print ("Nested then") end print ("Enclosure") else print ("Else") end if x < 100 or z > 100 then if y < 300 or x > 100 then print ("Nested then") else print ("Nested else") end end if x < 100 or z > 100 then if y < 100 or x > 100 then print ("Nested then") end else print ("Else") end if x < 100 or z > 100 then if y < 100 or x > 100 then print ("Nested then") else print ("Nested else") end else print ("Else") end if x < 100 or z > 100 then if y < 100 or x > 100 then print ("Nested then") else print ("Nested else") end else if y < 100 or x > 100 then print ("Nested else then") else print ("Nested else else") end end if x < 100 or y < 100 then print ("x or y with comparisons") end if x < 100 and y < 100 then print ("x and y with comparisons") end if x < 100 or y < 100 then print ("x or y with comparisons") else print ("ELSE x or y with comparisons") end if x < 100 and y < 100 then print ("x and y with comparisons") else print ("ELSE x and y with comparisons") end if (x < 100 and y < 100) or z < 100 then print ("(and) or with comparisons") else print ("ELSE (and) or with comparisons") end if x < 300 and y < 300 then print ("True terminator!") if x < 300 and z < 300 then print("Nested if") end else print ("False terminator!") if x < 300 and z < 300 then print("Enclosed nested if") end print ("Enclosure") end if x or y then print ("x or y") end if x and y then print ("x and y") end if x or y then print ("x or y") else print ("ELSE x or y") end if x and y then print ("x and y") else print ("ELSE x and y") end if (x < 300 and (y < 300 or z > 300)) or z < 300 and error() then print ("True terminator!") else print ("False terminator!") end if (x < 300 and (y < 300 or z > 300)) or z < 300 and error() then print ("True terminator!") if (x < 300 and (y < 300 or z > 300)) or z < 300 and error() then print ("True terminator!") if (x < 300 and (y < 300 or z > 300)) or z < 300 and error() then print ("True terminator!") else print ("False terminator!") end print ("Enclosed") else print ("False terminator!") end print ("Enclosed") else print ("False terminator!") end if (x < 300 and (y < 300 or z > 300)) or z < 300 and error() then print ("True terminator!") if (x < 300 and (y < 300 or z > 300)) or z < 300 and error() then print ("True terminator!") if (x < 300 and (y < 300 or z > 300)) or z < 300 and error() then print ("True terminator!") else print ("False terminator!") end print ("Enclosed") else print ("False terminator!") end else print ("False terminator!") end if (x < 300 and (y < 300 or z > 300)) or z < 300 and error() then print ("True terminator!") if (x < 300 and (y < 300 or z > 300)) or z < 300 and error() then print ("True terminator!") if (x < 300 and (y < 300 or z > 300)) or z < 300 and error() then print ("True terminator!") else print ("False terminator!") end else print ("False terminator!") end else print ("False terminator!") end if (y < 300 or z > 300) then print ("True terminator!") if (x < 300 and z < 300) then print ("True terminator!") if x < 300 or z < 300 then print ("True terminator!") else print ("False terminator!") end else print ("False terminator!") end print ("Enclosed") else print ("False terminator!") end if (x < 300 and (y < 300 or z > 300)) or z < 300 and error() then print ("True terminator!") if (x < 300 and (y < 300 or z > 300)) or z < 300 and error() then print ("True terminator!") if (x < 300 and (y < 300 or z > 300)) or z < 300 and error() then print ("True terminator!") else print ("False terminator!") end else print ("False terminator!") end else print ("False terminator!") if (x < 300 and (y < 300 or z > 300)) or z < 300 and error() then print ("True terminator!") if (x < 300 and (y < 300 or z > 300)) or z < 300 and error() then print ("True terminator!") else print ("False terminator!") end else print ("False terminator!") end end if (x < 300 and (y < 300 or z > 300)) or z < 300 and error() then if (x < 300 and (y < 300 or z > 300)) or z < 300 and error() then print ("True terminator!") if (x < 300 and (y < 300 or z > 300)) or z < 300 and error() then print ("True terminator!") else print ("False terminator!") end else print ("False terminator!") end else if (x < 300 and (y < 300 or z > 300)) or z < 300 and error() then print ("True terminator!") if (x < 300 and (y < 300 or z > 300)) or z < 300 and error() then print ("True terminator!") else print ("False terminator!") end else print ("False terminator!") end end local a = 0 if x % 2 == 0 then a = 1 end local a = x % 2 == 0 or a local a = "test" local b = "result" if a == "test" then b = "test" elseif a == "1234" then b = "1234" elseif a == "asd" then b = "asd" elseif a == "fadasd" then b = "fadasd" else b = "Otherwise" end print ("Same thing as expression") b = (a == "test" and "test") or (a == "1234" and "1234") or (a == "asd" and "asd") or (a == "fadasd" and "fadasd") or "Otherwise" print (a, b) local a = "test" if a == "test" then b = "test" if a == "1234" then b = "1234" if a == "asd" then a = "asd" if a == "fadasd" then b = "fadasd" end end end end local a = "test" local b = "result" if a == "test" then b = "test" if a == "test" then b = "test" elseif a == "1234" then b = "1234" elseif a == "asd" then b = "asd" elseif a == "fadasd" then b = "fadasd" else b = "Otherwise" end elseif a == "1234" then b = "1234" elseif a == "asd" then b = "asd" elseif a == "fadasd" then b = "fadasd" else b = "Otherwise" end print (a, b) if componentInfo.details ~= nil then layout = componentInfo.details.layout if layout == "table" then mainFrame = "tableFrame" else if layout == "header" then mainFrame = "headerFrame" else if layout == "long" then mainFrame = "fullFrame" else if layout == "short" then mainFrame = "topFrame" end end end end end print ("asd") local value = 1.3 if scaleinfo.floorValue then value = math.floor(value*scaleinfo.factor) else value = math.ceil(value*scaleinfo.factor) end function func(button, buttonElement) local stateEntry = buttonElement.buttonState local targetSlide = nil if buttonElement.active then if stateEntry.mouseClick or stateEntry.keyboardPress then targetSlide = "click" else if stateEntry.mouseOver or stateEntry.keyboard then targetSlide = "highlight" else targetSlide = "normal" end end else targetSlide = "unselect" end local _, curSide = getCurrentSlide(buttonElement.element) end local sliderelement = {} local centerValue = "" if sliderelement.scale[1].displayCenter then if sliderelement.scale[2] ~= nil and sliderelement.scale[2].displayCenter then centerValue = formatNumber( math.abs(getSliderCenterValue(sliderValue, sliderelement.scale[1])), sliderelement.scale[1].valueSuffix, math.abs(getSliderCenterValue(sliderValue, sliderelement.scale[2])), sliderelement.scale[2].valueSuffix) end else if sliderelement.scale[1].displayCenter then centerValue = formatNumber(math.abs(getSliderCenterValue(sliderValue, sliderelement.scale[1])), sliderelement.scale[1].valueSuffix) else if sliderelement.scale[2] and sliderelement.scale[2].displayCenter then centerValue = formatNumber(math.abs(getSliderCenterValue(sliderValue, sliderelement.scale[2])), sliderelement.scale[2].valueSuffix) end end end function setElementPosition(anarkElement, x, y, xUseHalfPixel, yUseHalfPixel) if config.verifyPixelExact then local testx = x if testx and ((xUseHalfPixel and private.offsetx % 1 == 0) or (not xUseHalfPixel and private.offsetx % 1 ~= 0)) then testx = testx + 0.5 end local testy = y if testy and ((yUseHalfPixel and private.offsety % 1 == 0) or (not yUseHalfPixel and private.offsety % 1 ~= 0)) then testy = testy + 0.5 end if (testx ~= nil and testx % 1 ~= 0) or (testy ~= nil and testy % 1 ~= 0) then DebugError("Widget system warning. Given position for element " .. tostring(anarkElement) .. " uses subpixels. This will lead to graphical issues. x/y: " .. tostring(x) .. " / " .. tostring(y) .. " - using halfpixels (x/y): " .. tostring(xUseHalfPixel) .. " / " .. tostring(yUseHalfPixel)) end end setElementPositionUnchecked(anarkElement, x, y) end local relationLEDValue, maxLED, minLED, boostActive if relationLEDValue < 0 then if boostActive then maxLED = maxLED - 1 end else minLED = relationLEDValue < 0 and boostLocal or minLED - 1 end function aaa() local a, b, c if a == 0 then if b == 2 then print("b") else print("bend") return end else function asd() print(a, b, c) end return end return end local nins, snapref, dumpreg, snapno, printsnap, tr, snap, tracesnap for ins=1, nins do if ins >= snapref then if dumpreg then out:write(format("", snapno)) else out:write(format("", snapno)) end printsnap(tr, snap) snapno = snapno + 1 snap = tracesnap(tr, snapno) snapref = snap and snap[0] or 65536 end local m, ot, op1, op2, ridsp = traceir(tr, ins) local oidx = shr(ot, 8)*6 local t = band(ot, 31) local op = sub(irnames, oidx + 1, oidx + 6) print ("Test") end if x == 0 then print ("then") else end local menu, x, y, test, xi print("asd") menu.onUpdate = function () if x and y then test = x or y print ("test") menu.attr = x or foo(y, "macro") end return end if shouldDisplayIcon(onScreen, targetElement.obstructed, (targetElement.outlined or targetElement.surfaceElement or targetElement.crate or targetElement.switchable) and targetElement.messageType ~= "missionobjective", targetElement.messageType == "missionobjective") then print("asd") end if test == 3 then print("Just a test") else end if x == 2 then print("This may crash the if else above!") end local xi, x, y, z if xi then z = x or y z = x or y z = x or y print ("asd") end if xi then local unlocked_defence_status = x or (y or z) < 100 bar((x and y) or z) end --[[ --]] ================================================ FILE: test/loop.lua ================================================ --[[ --]] for i=1, 100 do print ("Numeric loop") end for i=1, 100, 2 do print ("numeric for with step") end for i=1, 100 do for i=1, 100 do print ("Nested numeric loop") end end for i=1, 100 do print ("Numeric loop with break") break end for i=1, 100 do for i=1, 100 do print ("Nested numeric loop with outer break") end break end for i=1, 100 do for i=1, 100 do print ("Nested numeric loop with inner break") break end end for i=1, 100 do for i=1, 100 do break end end for i=0, 0 do print ("Zero loop") end for key, value in pairs(t) do print ("iterator for") print(key, value) end for key, value in ipairs(t) do print("iterator for with another iterator") print(key, value) end for key, value, x, y, z in iterate_over(t) do print("iterator for with crazy custom iterator") print(key, value, x, y, z) end a, b, c = pairs(t) for key, value in a, b, c do print("iterator for with dissected iterator") print(key, value) end x = 3 while x > 0 do print ("while") x = x - 1 end y = 0 x = y print ("while with emmidiate break") while x do break end while x and false do print ("while x and false") end while x or false do print ("while x or false") end while x and true do print ("while x and true") end while x or true do print ("while x or true") end print ("Something") while true do print ("while true") if x then print ("something") end end print ("Something") while true do if x then print ("something") end end print ("Something") while true do if x then print("something") break end end while true do if x then break end end while false do print ("while false") end while x do print ("while with copy check") x = y end while x > 100 do print ("while") while x > 100 do print ("Enclosed nested while") end print ("Enclosure") end while x > 100 do print ("Enclosure") while x > 100 do print ("Enclosed from ahead nested while") end end while x > 100 do while x > 100 do print ("Enclosed from below nested while") end print ("Enclosure") end while x > 100 do while x > 100 do print ("Nested while") end end while x < 100 or y < 100 do print ("while with expression") x = y end while x and y do print ("while with variables expression") x = y end while x < 100 or y < 100 do while x < 100 or y < 100 do print ("Nested while with expression") end end if x < 100 and y < 100 then while x < 100 or y < 100 do while x < 100 or y < 100 do print ("Nested while with expression within if") end end end while x < 100 or y < 100 do if x < 100 and y < 100 then while x < 100 or y < 100 do print ("Nested while with expression with if in middle") end end end while x < 100 or y < 100 do if x < 100 or y < 100 then while x < 100 or y < 100 do print ("Nested while with expression with if in middle") end end end while x < 100 or y < 100 do if x < 100 or y < 100 then while x < 100 or y < 100 do print ("Nested while with expression with break in middle") end break end end while x < 100 or y < 100 do if x < 100 or y < 100 then while x < 100 or y < 100 do print ("Nested while with expression with break in the end") break end end end while x < 100 or y < 100 do if (((x < 100 or y < 100 and z < 100)) or x > 300) and z == 3 then while x < 100 or y < 100 do print ("Nested while with really complex expression") end end end while (((x < 100 or y < 100 and z < 100)) or x > 300) and z == 3 do if (((x < 100 or y < 100 and z < 100)) or x > 300) and z == 3 then while x < 100 or y < 100 do print ("Nested while with really complex expression") end end end while x < 100 and y < 100 do if x < 100 or y < 100 then while (((x < 100 or y < 100 and z < 100)) or x > 300) and z == 3 do print ("Nested while with really complex expression") end end end while (((x < 100 or y < 100 and z < 100)) or x > 300) and z == 3 do if (((x < 100 or y < 100 and z < 100)) or x > 300) and z == 3 then while (((x < 100 or y < 100 and z < 100)) or x > 300) and z == 3 do if (((x < 100 or y < 100 and z < 100)) or x > 300) and z == 3 then print ("Nested while with really complex expression") end end end end while (((x < 100 or y < 100 and z < 100)) or x > 300) and z == 3 do if (((x < 100 or y < 100 and z < 100)) or x > 300) and z == 3 then while (((x < 100 or y < 100 and z < 100)) or x > 300) and z == 3 do if (((x < 100 or y < 100 and z < 100)) or x > 300) and z == 3 then print ("Nested while with really complex expression") break end end end end repeat print ("repeat until with copy check") x = y until not x repeat print ("repeat until with copy check") x = y until not x repeat print ("Repeat until with break") break until x < 3 repeat print ("Enclosed") repeat print ("Nested repeat until") until x < 3 print ("Enclosed") until x < 3 repeat repeat print ("Nested repeat until") until x < 3 print ("Enclosed") until x < 3 repeat repeat print ("Nested repeat until") until x < 3 until x < 3 repeat print ("repeat until") x = x + 1 until x > 5 or x < 3 repeat print ("Repeat until with expression") until x < 3 and y < 3 repeat print ("Repeat until with expression") repeat print ("Repeat until with expression") until x < 3 or y < 3 until x < 3 and y < 3 repeat if (((x < 100 or y < 100 and z < 100)) or x > 300) and z == 3 then print ("Repeat until with expression") repeat if (((x < 100 or y < 100 and z < 100)) or x > 300) and z == 3 then print ("Repeat until with expression") end until x < 3 or y < 3 end until x < 3 and y < 3 --[[ --]] if x then for i=1,2,3 do print ("something") if y then break end print ("something") end else print("else") end ================================================ FILE: test/primitive.lua ================================================ function assignment() function generator() return 1, 2, 3 end a = 1 b, c, d = nil e, f, g = 1, 2, 3 local i, k, l = 3, 2, g + 1 local m, n, o, p = 1, generator() print (a, b, c, d, e, f, g, i, k, l) end function vararg(...) a = ... c, d = ... t = {...} s = {1, 2, 3, ...} local f, g, h = ... t.x = ... assignment(...) if t.x == 3 then return a, t, s, ... else return ... end end function tables() function generator(x, y, z) return x, y, z end t = { [123.322] = 3, [3] = { "a", "b", "c" }, generator("a", "b", "c"), generator("d", "e", "f") } t.var = 1 t.str = "Nope" function t:foo(var) self.var = var; self.str = "!!!" .. var end t:foo(123) print(t) end function logical() x = 3 print ("if true") if true then print ("That's true") end print ("if false") if false then print ("That's false") end print ("Ordinary if") if x == 123 then print ("Good number") end print ("No then, only else") if x == 1 then else print ("Not one!") end print ("elseifs") if x == 3 then print("is three!") elseif x == 5 then print("is five!") elseif x == 8 then print("is eight!") return else print("is something else!") end print ("ordinary if") if x == 6 then print("Is six!") else print("Whatever else!") end print ("The same if as expression") local a = ((x == 6) and print("Is six!")) or print("Whatever else!") print ("Nested if") if x == 666 then print("Hellish X!") if x > 321 then if x > 333 then print ("X > 321 AND 333 - Isn't that obvious already?") end else print ("Seriously???") end else print("Not bad enough!") if x ~= 42 then print ("And it doesn't answer anything") end end end function logical_expressions() x = 0 y = 1 z = 2 print ("simple or expression") b = x or y print ("simple or not expression") b = not x or y print ("simple and expression") b = x and y print ("simple or expression with binary comparison") b = (x < 100) or y print ("simple and expression with binary comparison") b = (x < 100) and y print ("simple and expression with binary comparison and function call") b = (x < 100) and print(y) print ("simple and expression with double binary comparison") b = (x < 100) and (y > 100) print ("(and) or expression") b = (x and y) or z print ("(or) and expression") b = (x or y) and z print ("(and) and expression") b = (x and y) and z print ("(or) or expression") b = (x or y) or z print ("or (and) expression") b = x or (y and x) print ("and (or) expression") b = x and (y or x) print ("and (and) expression") b = x and (y and x) print ("or (or) expression") b = x or (y or x) print ("ond (or) and expression") b = x and (y or x) and z print ("or (and) or expression") b = x or (y and x) or z print ("and of two ors") b = (x or z) and (y or z) print ("or of two ands") b = (x and z) or (y and z) print ("and (or) and (or) expression with comparisons") b = x < 100 and (y < 100 or x < 100) and (z < 100 or x < 100) print ("and (or) and or expression with comparisons") b = x < 100 and (y < 100 or x < 100) and z < 100 or x < 100 print ("or (and) or (and) expression with comparisons") b = x < 100 or (y < 100 and x < 100) or (z < 100 and x < 100) print ("and (and) and (and) expression with comparisons") b = x < 100 and (y < 100 and x < 100) and (z < 100 and x < 100) print ("or (or) or (or) expression with comparisons") b = x < 100 or (y < 100 or x < 100) or (z < 100 or x < 100) print ("4 and expression with comparisons") b = x < 100 and y < 100 and x < 100 and z < 100 and x < 100 print ("4 or expression with comparisons") b = x < 100 or y < 100 or x < 100 or z < 100 or x < 100 print ("and (or or) and (or or) expression with comparisons") b = x < 100 and (y < 100 or x < 100 or z < 100) and (y < 100 or x < 100 or z < 100) print ("and (or and or) and (or and or) expression with comparisons") b = x < 100 and (y < 100 or (x < 100 and x > 100) or z < 100) and (y < 100 or (x < 100 and x > 100) or z < 100) print ("or (and or and) or (and or and) expression with comparisons") b = x < 100 or (y < 100 and (x < 100 or x > 100) and z < 100) or (y < 100 and (x < 100 or x > 100) and z < 100) print ("simple or and expression with binary comparison") b = x or (y and (x < 100)) print ("normal logical expression") b = (x and y) or ((y > 3) and (((x/2) < 1) or (y > 100)) and (x ~= 2)) print ("precalculated true expression") c = true or (x and y) or true print ("precalculated false expression") d = false and ((x and y) or true) print ("precalculated false expression with function") e = error() and false and ((x and y) or true) print ("precalculated true expression with function") e = error() and true and ((x and y) or true) print ("precalculated? false expression with variable") local z = false f = z and ((x and y) or true) print ("precalculated false expression with nil") f = nil and ((x and y) or true) print("if with expression") a = x or y a = x and y a = x < 100 or y < 100 a = x < 100 and y < 100 if x or y then print ("x or y") end if x and y then print ("x and y") end if x < 100 or y < 100 then print ("x or y with comparisons") end if x < 100 and y < 100 then print ("x and y with comparisons") end if x or y then print ("x or y") else print ("ELSE x or y") end if x and y then print ("x and y") else print ("ELSE x and y") end if x < 100 or y < 100 then print ("x or y with comparisons") else print ("ELSE x or y with comparisons") end if x < 100 and y < 100 then print ("x and y with comparisons") else print ("ELSE x and y with comparisons") end if (x < 100 and y < 100) or z < 100 then print ("(and) or with comparisons") else print ("ELSE (and) or with comparisons") end if (x < 300 and (y < 300 or z > 300)) or z < 300 and error() then print ("True terminator!") else print ("False terminator!") end if x < 300 and y < 300 then print ("True terminator!") if x < 300 and z < 300 then print("Nested if") end else print ("False terminator!") if x < 300 and z < 300 then print("Enclosed nested if") end print ("Enclosure") end while x > 300 and y < 300 do print ("In while") end repeat print ("In repeat until") until x < 300 and y > 300 print(x, y, b, c, d, e, f) end function functions() function func1(x, y) function sub(z) return z end return x, y, sub end x, y, z = func1(1, 2) print(z(4)) x = func1(1, 2) func1(1, 2) function func2(x) print (x) end function func3(x) return x*2 end func2(func3(3)) function func4(x, y, z) print (x, y, z) end func4(1, 2, func2(3)) end function locals(x, y, ...) local a, b, c = ... function generator() return 1, 2, 3 end local d, e, f = generator() local g, h, i, k = 4, generator() local l, m, n = f, e end function loops() function iterate_over(table) function iterator(table, index) key, value = next(table, index) return key, value, 1, 2, 3 end return iterator, table, nil end t = {1, 2, 3} print ("numeric for without step") for i=1, 100 do print(i) end print ("numeric for with step") for i=1, 100, 2 do print(i) end print ("iterator for") for key, value in pairs(t) do print(key, value) end print("iterator for with another iterator") for key, value in ipairs(t) do print(key, value) end print("iterator for with crazy custom iterator") for key, value, x, y, z in iterate_over(t) do print(key, value, x, y, z) end print("iterator for with dissected iterator") a, b, c = pairs(t) for key, value in a, b, c do print(key, value) end print ("while") x = 3 while x > 0 do x = x - 1 end print ("while with copy check") y = 0 x = y while x do x = y end print ("repeat until") repeat x = x + 1 until x > 5 print ("repeat until with copy check") repeat x = y until not x print ("While with break") while x > 5 do break end print ("Repeat until with break") repeat break until x < 3 print ("Numeric for with break") for i=0,1,2 do break end print ("Iterator for with break") for key,value in pairs(t) do break end print ("Loop with break and function inside") t = {} for i=0,100 do y = 3 t[i] = function () return i + y end if i == 5 then print ("then") break else print ("else") end end end function upvalues() test = 0 function sub(x) test = test + 1 test = 3 test = "asd" test = 4.0 return test + x end print(sub(3)) end function subblock() print ("Subblock with locals") x = 3 do local a = 2 print(a + x) end y = 4 end