Showing preview only (231K chars total). Download the full file or copy to clipboard to get everything.
Repository: zzwlpx/ljd
Branch: master
Commit: 16d485d0f822
Files: 46
Total size: 217.5 KB
Directory structure:
gitextract_2i5rc5bq/
├── .gitignore
├── LICENSE
├── README.md
├── README.md.txt
├── gconfig.py
├── ljd/
│ ├── __init__.py
│ ├── ast/
│ │ ├── __init__.py
│ │ ├── builder.py
│ │ ├── helpers.py
│ │ ├── locals.py
│ │ ├── mutator.py
│ │ ├── nodes.py
│ │ ├── slotworks.py
│ │ ├── traverse.py
│ │ ├── unwarper.py
│ │ └── validator.py
│ ├── bytecode/
│ │ ├── __init__.py
│ │ ├── constants.py
│ │ ├── debuginfo.py
│ │ ├── helpers.py
│ │ ├── instructions.py
│ │ └── prototype.py
│ ├── lua/
│ │ ├── __init__.py
│ │ └── writer.py
│ ├── pseudoasm/
│ │ ├── __init__.py
│ │ ├── constants.py
│ │ ├── instructions.py
│ │ ├── prototype.py
│ │ └── writer.py
│ ├── rawdump/
│ │ ├── __init__.py
│ │ ├── code.py
│ │ ├── constants.py
│ │ ├── debuginfo.py
│ │ ├── header.py
│ │ ├── parser.py
│ │ └── prototype.py
│ └── util/
│ ├── __init__.py
│ ├── binstream.py
│ ├── indentedstream.py
│ └── log.py
├── main.py
└── test/
├── breaks.lua
├── expression.lua
├── ifs.lua
├── loop.lua
└── primitive.lua
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
*.lua[co]
*.py[cod]
# C extensions
*.so
# Packages
*.egg
*.egg-info
dist
build
eggs
parts
bin
var
sdist
develop-eggs
.installed.cfg
lib
lib64
__pycache__
# Installer logs
pip-log.txt
# Unit test / coverage reports
.coverage
.tox
nosetests.xml
# Translations
*.mo
# Mr Developer
.mr.developer.cfg
.project
.pydevproject
.*.swp
================================================
FILE: LICENSE
================================================
The MIT License (MIT)
Copyright (c) 2013 Andrian Nord
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
================================================
FILE: README.md
================================================
代码在fork https://github.com/NightNord/ljd 的基础上修改的
主要修正了原版本没有将_read_header函数解析过的header中的flag值带进_read_protoypes。
解决了原代码只能解析带调试信息的luajit字节码，而解析不带调试信息字节码时报错的问题。
此版本可解析luajit2.1.0-beta2的字节码文件。如果要解析其它版本的luajit字节码文件，需要将
luajit源码中的opcode，写进instruction.py 和 code.py
使用说明：python main.py "path of luajit-bytecode"
解释文档：http://www.freebuf.com/column/177810.html
参考：https://github.com/NightNord/ljd
https://github.com/feicong/lua_re/blob/master/lua/lua_re3.md
https://bbs.pediy.com/thread-216800.htm
================================================
FILE: README.md.txt
================================================
LuaJIT raw-bytecode decompiler (LJD)
===
The original name was _ljwthgnd_ as in _LuaJIT 'What The Hell is Going On'
Decompiler_ named under the LuaJIT C sources variable naming convention.
__WARNING!__ This code is nor finished, nor tested yet! There is no even
slightest warranty that resulting code is even near to the original. Use it at
your own risk of the wasted time.
__SECOND WARNING!__ And, BTW, this all is a one huge prototype. Because the
"release" version should be written into lua itself. Because it's cool to
decompile the decompiler - a great test too!
Requirements
---
Python __3.0+__ from python.org
How to use it
---
There is no argument parsing right now, so comment out things in the ```main.py```
script and launch it as in ```main.py path/to/file.luac```
TODO
---
There is a lot of work to do, in the order of priority
0. Logical subexpressions in while statements:
```lua
while x < (xi and 2 or 3) do
print ("Hello crazy world!")
end
```
Logical subexpressions (the subexpressions used as operands in
ariphmetic or comparison operations inside other exrpressions) are
currently supported only for ifs. To support them for whiles and
repeat untils an expression unwarping logic should be moved at the
very beginning. But it won't work without all fixes being done in
a loop unwarping logic. So we need to split that and move the fixes
before expressions before loops before ifs. That's not that easy...
1. AST Mutations:
1. Use the line information (or common sense if there is no line
information) to squash similar expressions into a single expression.
2. Formatting improvements
1. Use the line information (or common sense) to preserve empty lines
and break long statements like in the original code.
This is mostly done, but only in the "common sense" part.
2. Use method-style calls and definitions for tables.
3. Features not supported:
1. GOTO statement (from lua 5.2). All the required functionality is
now in place, but that's rather a low-priority task right now.
2. Local sub-blocks:
```lua
do
...
end
```
These subblocks are not reflected anyhow directly in the bytecode.
The only way to guess them is to watch local variable scopes, which
is simple enough in case of non-stripped bytecode and a bit
harder otherwise.
================================================
FILE: gconfig.py
================================================
#zzw 20180714 support str encode
gFlagDic = {'strEncode' : 'ascii'}
================================================
FILE: ljd/__init__.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
================================================
FILE: ljd/ast/__init__.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
================================================
FILE: ljd/ast/builder.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
import ljd.bytecode.instructions as ins
from ljd.bytecode.helpers import get_jump_destination
from ljd.bytecode.constants import T_NIL, T_FALSE, T_TRUE
import ljd.ast.nodes as nodes
class _State():
def __init__(self):
self.constants = None
self.debuginfo = None
self.block = None
self.blocks = []
self.block_starts = {}
def _warp_in_block(self, addr):
#print (self.block_starts)
#print (addr)
block = self.block_starts[addr]
block.warpins_count += 1
return block
def build(prototype):
return _build_function_definition(prototype)
def _build_function_definition(prototype):
node = nodes.FunctionDefinition()
state = _State()
state.constants = prototype.constants
state.debuginfo = prototype.debuginfo
node._upvalues = prototype.constants.upvalue_references
node._debuginfo = prototype.debuginfo
node._instructions_count = len(prototype.instructions)
node.arguments.contents = _build_function_arguments(state, prototype)
if prototype.flags.is_variadic:
node.arguments.contents.append(nodes.Vararg())
instructions = prototype.instructions
#print (len(instructions))
node.statements.contents = _build_function_blocks(state, instructions)
return node
def _build_function_arguments(state, prototype):
arguments = []
count = prototype.arguments_count
slot = 0
while slot < count:
variable = _build_slot(state, 0, slot)
arguments.append(variable)
slot += 1
return arguments
def _build_function_blocks(state, instructions):
_blockenize(state, instructions)
_establish_warps(state, instructions)
state.blocks[0].warpins_count = 1
for block in state.blocks:
addr = block.first_address
state.block = block
while addr <= block._last_body_addr:
instruction = instructions[addr]
statement = _build_statement(state, addr, instruction)
if statement is not None:
line = state.debuginfo.lookup_line_number(addr)
setattr(statement, "_addr", addr)
setattr(statement, "_line", line)
block.contents.append(statement)
addr += 1
return state.blocks
_JUMP_WARP_INSTRUCTIONS = set((
ins.UCLO.opcode,
ins.ISNEXT.opcode,
ins.JMP.opcode,
ins.FORI.opcode,
ins.JFORI.opcode
))
_WARP_INSTRUCTIONS = _JUMP_WARP_INSTRUCTIONS | set((
ins.FORL.opcode, ins.IFORL.opcode, ins.JFORL.opcode,
ins.ITERL.opcode, ins.IITERL.opcode, ins.JITERL.opcode,
ins.LOOP.opcode
))
def _blockenize(state, instructions):
addr = 1
# Duplicates are possible and ok, but we need to sort them out
last_addresses = set()
while addr < len(instructions):
instruction = instructions[addr]
opcode = instruction.opcode
if opcode not in _WARP_INSTRUCTIONS:
addr += 1
continue
if opcode in _JUMP_WARP_INSTRUCTIONS:
destination = get_jump_destination(addr, instruction)
if opcode != ins.UCLO.opcode or destination != addr + 1:
last_addresses.add(destination - 1)
last_addresses.add(addr)
else:
last_addresses.add(addr)
addr += 1
last_addresses = sorted(list(last_addresses))
last_addresses.append(len(instructions) - 1)
# This could happen if something jumps to the first instruction
# We don't need "zero block" with function header, so simply ignore
# this
if last_addresses[0] == 0:
last_addresses.pop(0)
previous_last_address = 0
index = 0
for last_address in last_addresses:
block = nodes.Block()
block.index = index
block.first_address = previous_last_address + 1
block.last_address = last_address
state.blocks.append(block)
state.block_starts[block.first_address] = block
previous_last_address = last_address
index += 1
def _establish_warps(state, instructions):
state.blocks[0].warpins_count = 1
for block in state.blocks[:-1]:
state.block = block
end_addr = block.last_address + 1
start_addr = max(block.last_address - 1, block.first_address)
warp = instructions[start_addr:end_addr]
#print (block.first_address)
block.warp, shift = _build_warp(state, block.last_address, warp)
setattr(block, "_last_body_addr", block.last_address - shift)
setattr(block.warp, "_addr", block.last_address - shift + 1)
last_block = state.blocks[-1]
last_block.warp = nodes.EndWarp()
setattr(last_block, "_last_body_addr", last_block.last_address)
setattr(last_block.warp, "_addr", last_block.last_address)
def _build_warp(state, last_addr, instructions):
last = instructions[-1]
#print ("%x"%(last.opcode))
if last.opcode in (ins.JMP.opcode, ins.UCLO.opcode, ins.ISNEXT.opcode):
return _build_jump_warp(state, last_addr, instructions)
elif last.opcode >= ins.ITERL.opcode and last.opcode <= ins.JITERL.opcode:
assert len(instructions) == 2
return _build_iterator_warp(state, last_addr, instructions)
elif last.opcode >= ins.FORL.opcode and last.opcode <= ins.JFORL.opcode:
return _build_numeric_loop_warp(state, last_addr, last)
else:
return _build_flow_warp(state, last_addr, last)
def _build_jump_warp(state, last_addr, instructions):
last = instructions[-1]
opcode = 256 if len(instructions) == 1 else instructions[-2].opcode
if opcode <= ins.ISF.opcode:
assert last.opcode != ins.ISNEXT.opcode
return _build_conditional_warp(state, last_addr, instructions)
else:
return _build_unconditional_warp(state, last_addr, last)
def _build_conditional_warp(state, last_addr, instructions):
condition = instructions[-2]
condition_addr = last_addr - 1
warp = nodes.ConditionalWarp()
if condition.opcode in (ins.ISTC.opcode, ins.ISFC.opcode):
expression = _build_unary_expression(state,
condition_addr,
condition)
setattr(warp, "_slot", condition.A)
elif condition.opcode >= ins.IST.opcode:
expression = _build_unary_expression(state,
condition_addr,
condition)
setattr(warp, "_slot", condition.CD)
else:
expression = _build_comparison_expression(state,
condition_addr,
condition)
warp.condition = expression
jump = instructions[-1]
jump_addr = last_addr
destination = get_jump_destination(jump_addr, jump)
# A condition is inverted during the preparation phase above
warp.false_target = state._warp_in_block(destination)
warp.true_target = state._warp_in_block(jump_addr + 1)
return warp, 2
def _build_unconditional_warp(state, addr, instruction):
warp = nodes.UnconditionalWarp()
warp.type = nodes.UnconditionalWarp.T_JUMP
opcode = instruction.opcode
warp.is_uclo = opcode == ins.UCLO.opcode
if warp.is_uclo and instruction.CD == 0:
# Not a jump
return _build_flow_warp(state, addr, instruction)
else:
destination = get_jump_destination(addr, instruction)
warp.target = state._warp_in_block(destination)
return warp, 1
def _build_iterator_warp(state, last_addr, instructions):
iterator = instructions[-2]
iterator_addr = last_addr - 1
assert iterator.opcode in (ins.ITERC.opcode, ins.ITERN.opcode)
warp = nodes.IteratorWarp()
base = iterator.A
warp.controls.contents = [
_build_slot(state, iterator_addr, base - 3), # generator
_build_slot(state, iterator_addr, base - 2), # state
_build_slot(state, iterator_addr, base - 1) # control
]
last_slot = base + iterator.B - 2
slot = base
while slot <= last_slot:
variable = _build_slot(state, iterator_addr - 1, slot)
warp.variables.contents.append(variable)
slot += 1
jump = instructions[-1]
jump_addr = last_addr
destination = get_jump_destination(jump_addr, jump)
warp.way_out = state._warp_in_block(jump_addr + 1)
warp.body = state._warp_in_block(destination)
return warp, 2
def _build_numeric_loop_warp(state, addr, instruction):
warp = nodes.NumericLoopWarp()
base = instruction.A
warp.index = _build_slot(state, addr, base + 3)
warp.controls.contents = [
_build_slot(state, addr, base + 0), # start
_build_slot(state, addr, base + 1), # limit
_build_slot(state, addr, base + 2) # step
]
#print ("%x" % instruction.A)
#print ("%x" %instruction.CD)
destination = get_jump_destination(addr, instruction)
warp.body = state._warp_in_block(destination)
warp.way_out = state._warp_in_block(addr + 1)
return warp, 1
def _build_flow_warp(state, addr, instruction):
warp = nodes.UnconditionalWarp()
warp.type = nodes.UnconditionalWarp.T_FLOW
warp.target = state._warp_in_block(addr + 1)
opcode = instruction.opcode
shift = 1 if opcode in (ins.FORI.opcode, ins.UCLO.opcode) else 0
return warp, shift
def _build_statement(state, addr, instruction):
opcode = instruction.opcode
A_type = instruction.A_type
# Generic assignments - handle the ASSIGNMENT stuff below
if A_type == ins.T_DST or A_type == ins.T_UV:
return _build_var_assignment(state, addr, instruction)
# ASSIGNMENT starting from MOV and ending at KPRI
elif opcode == ins.KNIL.opcode:
return _build_knil(state, addr, instruction)
# ASSIGNMENT starting from UGET and ending at USETP
# SKIP UCL0 is handled below
# ASSIGNMENT starting from FNEW and ending at GGET
elif opcode == ins.GSET.opcode:
return _build_global_assignment(state, addr, instruction)
# ASSIGNMENT starting from TGETV and ending at TGETB
elif opcode >= ins.TSETV.opcode and opcode <= ins.TSETB.opcode:
return _build_table_assignment(state, addr, instruction)
elif opcode == ins.TSETM.opcode:
return _build_table_mass_assignment(state, addr, instruction)
elif opcode >= ins.CALLM.opcode and opcode <= ins.CALLT.opcode:
return _build_call(state, addr, instruction)
elif opcode == ins.VARG.opcode:
return _build_vararg(state, addr, instruction)
elif opcode >= ins.RETM.opcode and opcode <= ins.RET1.opcode:
return _build_return(state, addr, instruction)
else:
assert opcode == ins.UCLO.opcode or (
opcode >= ins.LOOP.opcode
and opcode <= ins.JLOOP.opcode)
# Noop
return None
def _build_var_assignment(state, addr, instruction):
opcode = instruction.opcode
assignment = nodes.Assignment()
# Unary assignment operators (A = op D)
if opcode == ins.MOV.opcode \
or opcode == ins.NOT.opcode \
or opcode == ins.UNM.opcode \
or opcode == ins.LEN.opcode:
expression = _build_unary_expression(state, addr, instruction)
# Binary assignment operators (A = B op C)
elif opcode <= ins.POW.opcode:
expression = _build_binary_expression(state, addr, instruction)
# Concat assignment type (A = B .. B + 1 .. ... .. C - 1 .. C)
elif opcode == ins.CAT.opcode:
expression = _build_concat_expression(state, addr, instruction)
# Constant assignment operators except KNIL, which is weird anyway
elif opcode <= ins.KPRI.opcode:
expression = _build_const_expression(state, addr, instruction)
elif opcode == ins.UGET.opcode:
expression = _build_upvalue(state, addr, instruction.CD)
elif opcode == ins.USETV.opcode:
expression = _build_slot(state, addr, instruction.CD)
elif opcode <= ins.USETP.opcode:
expression = _build_const_expression(state, addr, instruction)
elif opcode == ins.FNEW.opcode:
expression = _build_function(state, instruction.CD)
elif opcode == ins.TNEW.opcode:
expression = nodes.TableConstructor()
elif opcode == ins.TDUP.opcode:
expression = _build_table_copy(state, instruction.CD)
elif opcode == ins.GGET.opcode:
expression = _build_global_variable(state, addr, instruction.CD)
else:
assert opcode <= ins.TGETB.opcode
expression = _build_table_element(state, addr, instruction)
assignment.expressions.contents.append(expression)
if instruction.A_type == ins.T_DST:
destination = _build_slot(state, addr, instruction.A)
else:
assert instruction.A_type == ins.T_UV
destination = _build_upvalue(state, addr, instruction.A)
assignment.destinations.contents.append(destination)
return assignment
def _build_knil(state, addr, instruction):
node = _build_range_assignment(state, addr, instruction.A, instruction.CD)
node.expressions.contents = [_build_primitive(state, None)]
return node
def _build_global_assignment(state, addr, instruction):
assignment = nodes.Assignment()
variable = _build_global_variable(state, addr, instruction.CD)
expression = _build_slot(state, addr, instruction.A)
assignment.destinations.contents.append(variable)
assignment.expressions.contents.append(expression)
return assignment
def _build_table_assignment(state, addr, instruction):
assignment = nodes.Assignment()
destination = _build_table_element(state, addr, instruction)
expression = _build_slot(state, addr, instruction.A)
assignment.destinations.contents.append(destination)
assignment.expressions.contents.append(expression)
return assignment
def _build_table_mass_assignment(state, addr, instruction):
assignment = nodes.Assignment()
base = instruction.A
destination = nodes.TableElement()
destination.key = nodes.MULTRES()
destination.table = _build_slot(state, addr, base - 1)
assignment.destinations.contents = [destination]
assignment.expressions.contents = [nodes.MULTRES()]
return assignment
def _build_call(state, addr, instruction):
call = nodes.FunctionCall()
call.function = _build_slot(state, addr, instruction.A)
call.arguments.contents = _build_call_arguments(state, addr, instruction)
if instruction.opcode <= ins.CALL.opcode:
if instruction.B == 0:
node = nodes.Assignment()
node.destinations.contents.append(nodes.MULTRES())
node.expressions.contents.append(call)
elif instruction.B == 1:
node = call
else:
from_slot = instruction.A
to_slot = instruction.A + instruction.B - 2
node = _build_range_assignment(state, addr, from_slot,
to_slot)
node.expressions.contents.append(call)
else:
assert instruction.opcode <= ins.CALLT.opcode
node = nodes.Return()
node.returns.contents.append(call)
return node
def _build_vararg(state, addr, instruction):
base = instruction.A
last_slot = base + instruction.B - 2
if last_slot < base:
node = nodes.Assignment()
node.destinations.contents.append(nodes.MULTRES())
node.expressions.contents.append(nodes.Vararg())
else:
node = _build_range_assignment(state, addr, base, last_slot)
node.expressions.contents.append(nodes.Vararg())
return node
def _build_return(state, addr, instruction):
node = nodes.Return()
base = instruction.A
last_slot = base + instruction.CD - 1
if instruction.opcode != ins.RETM.opcode:
last_slot -= 1
slot = base
# Negative count for the RETM is OK
while slot <= last_slot:
variable = _build_slot(state, addr, slot)
node.returns.contents.append(variable)
slot += 1
if instruction.opcode == ins.RETM.opcode:
node.returns.contents.append(nodes.MULTRES())
return node
def _build_call_arguments(state, addr, instruction):
base = instruction.A
last_argument_slot = base + instruction.CD
is_variadic = (instruction.opcode == ins.CALLM.opcode \
or instruction.opcode == ins.CALLMT.opcode)
if not is_variadic:
last_argument_slot -= 1
arguments = []
slot = base + 1
while slot <= last_argument_slot:
argument = _build_slot(state, addr, slot)
arguments.append(argument)
slot += 1
if is_variadic:
arguments.append(nodes.MULTRES())
return arguments
def _build_range_assignment(state, addr, from_slot, to_slot):
assignment = nodes.Assignment()
slot = from_slot
assert from_slot <= to_slot
while slot <= to_slot:
destination = _build_slot(state, addr, slot)
assignment.destinations.contents.append(destination)
slot += 1
return assignment
_BINARY_OPERATOR_MAP = [None] * 255
_BINARY_OPERATOR_MAP[ins.ADDVN.opcode] = nodes.BinaryOperator.T_ADD
_BINARY_OPERATOR_MAP[ins.SUBVN.opcode] = nodes.BinaryOperator.T_SUBTRACT
_BINARY_OPERATOR_MAP[ins.MULVN.opcode] = nodes.BinaryOperator.T_MULTIPLY
_BINARY_OPERATOR_MAP[ins.DIVVN.opcode] = nodes.BinaryOperator.T_DIVISION
_BINARY_OPERATOR_MAP[ins.MODVN.opcode] = nodes.BinaryOperator.T_MOD
def _build_binary_expression(state, addr, instruction):
operator = nodes.BinaryOperator()
opcode = instruction.opcode
if opcode == ins.POW.opcode:
operator.type = nodes.BinaryOperator.T_POW
else:
map_index = opcode - ins.ADDVN.opcode
map_index %= 5
map_index += ins.ADDVN.opcode
operator.type = _BINARY_OPERATOR_MAP[map_index]
if instruction.B_type == ins.T_VAR and instruction.CD_type == ins.T_VAR:
operator.left = _build_slot(state, addr, instruction.B)
operator.right = _build_slot(state, addr, instruction.CD)
elif instruction.B_type == ins.T_VAR:
operator.left = _build_slot(state, addr, instruction.B)
operator.right = _build_numeric_constant(state, instruction.CD)
else:
assert instruction.CD_type == ins.T_VAR
operator.right = _build_slot(state, addr, instruction.B)
operator.left = _build_numeric_constant(state, instruction.CD)
return operator
def _build_concat_expression(state, addr, instruction):
operator = nodes.BinaryOperator()
operator.type = nodes.BinaryOperator.T_CONCAT
slot = instruction.B
operator.left = _build_slot(state, addr, slot)
operator.right = _build_slot(state, addr, slot + 1)
slot += 2
while slot <= instruction.CD:
upper_operator = nodes.BinaryOperator()
upper_operator.left = operator
upper_operator.right = _build_slot(state, addr, slot)
upper_operator.type = nodes.BinaryOperator.T_CONCAT
operator = upper_operator
slot += 1
return operator
def _build_const_expression(state, addr, instruction):
CD_type = instruction.CD_type
if CD_type == ins.T_STR:
return _build_string_constant(state, instruction.CD)
elif CD_type == ins.T_CDT:
return _build_cdata_constant(state, instruction.CD)
elif CD_type == ins.T_SLIT:
value = instruction.CD
if value & 0x8000:
value = -0x10000 + value
return _build_literal(state, value)
elif CD_type == ins.T_LIT:
return _build_literal(state, instruction.CD)
elif CD_type == ins.T_NUM:
return _build_numeric_constant(state, instruction.CD)
else:
assert CD_type == ins.T_PRI
return _build_primitive(state, instruction.CD)
def _build_table_element(state, addr, instruction):
node = nodes.TableElement()
node.table = _build_slot(state, addr, instruction.B)
if instruction.CD_type == ins.T_VAR:
node.key = _build_slot(state, addr, instruction.CD)
else:
node.key = _build_const_expression(state, addr, instruction)
return node
def _build_function(state, slot):
prototype = state.constants.complex_constants[slot]
return _build_function_definition(prototype)
def _build_table_copy(state, slot):
node = nodes.TableConstructor()
table = state.constants.complex_constants[slot]
i = 0
for value in table.array:
record = nodes.ArrayRecord()
record.value = _build_table_record_item(value)
node.array.contents.append(record)
i += 1
for key, value in table.dictionary:
record = nodes.TableRecord()
record.key = _build_table_record_item(key)
record.value = _build_table_record_item(value)
node.records.contents.append(record)
return node
def _build_table_record_item(value):
if value is None:
item = nodes.Primitive()
item.type = nodes.Primitive.T_NIL
elif value is True:
item = nodes.Primitive()
item.type = nodes.Primitive.T_TRUE
elif value is False:
item = nodes.Primitive()
item.type = nodes.Primitive.T_FALSE
elif isinstance(value, int):
item = nodes.Constant()
item.value = value
item.type = nodes.Constant.T_INTEGER
elif isinstance(value, float):
item = nodes.Constant()
item.value = value
item.type = nodes.Constant.T_FLOAT
elif isinstance(value, str):
item = nodes.Constant()
item.value = value
item.type = nodes.Constant.T_STRING
return item
_COMPARISON_MAP = [None] * 255
# Mind the inversion - comparison operators are affecting JMP to the next block
# So in the normal code a comparison will be inverted
_COMPARISON_MAP[ins.ISLT.opcode] = nodes.BinaryOperator.T_GREATER_OR_EQUAL
_COMPARISON_MAP[ins.ISGE.opcode] = nodes.BinaryOperator.T_LESS_THEN
_COMPARISON_MAP[ins.ISLE.opcode] = nodes.BinaryOperator.T_GREATER_THEN
_COMPARISON_MAP[ins.ISGT.opcode] = nodes.BinaryOperator.T_LESS_OR_EQUAL
_COMPARISON_MAP[ins.ISEQV.opcode] = nodes.BinaryOperator.T_NOT_EQUAL
_COMPARISON_MAP[ins.ISNEV.opcode] = nodes.BinaryOperator.T_EQUAL
_COMPARISON_MAP[ins.ISEQS.opcode] = nodes.BinaryOperator.T_NOT_EQUAL
_COMPARISON_MAP[ins.ISNES.opcode] = nodes.BinaryOperator.T_EQUAL
_COMPARISON_MAP[ins.ISEQN.opcode] = nodes.BinaryOperator.T_NOT_EQUAL
_COMPARISON_MAP[ins.ISNEN.opcode] = nodes.BinaryOperator.T_EQUAL
_COMPARISON_MAP[ins.ISEQP.opcode] = nodes.BinaryOperator.T_NOT_EQUAL
_COMPARISON_MAP[ins.ISNEP.opcode] = nodes.BinaryOperator.T_EQUAL
def _build_comparison_expression(state, addr, instruction):
operator = nodes.BinaryOperator()
operator.left = _build_slot(state, addr, instruction.A)
opcode = instruction.opcode
if opcode == ins.ISEQS.opcode or opcode == ins.ISNES.opcode:
operator.right = _build_string_constant(state, instruction.CD)
elif opcode == ins.ISEQN.opcode or opcode == ins.ISNEN.opcode:
operator.right = _build_numeric_constant(state, instruction.CD)
elif opcode == ins.ISEQP.opcode or opcode == ins.ISNEP.opcode:
operator.right = _build_primitive(state, instruction.CD)
else:
operator.right = _build_slot(state, addr, instruction.CD)
operator.type = _COMPARISON_MAP[instruction.opcode]
assert operator.type is not None
return operator
def _build_unary_expression(state, addr, instruction):
opcode = instruction.opcode
variable = _build_slot(state, addr, instruction.CD)
# Mind the inversion
if opcode == ins.ISFC.opcode \
or opcode == ins.ISF.opcode \
or opcode == ins.MOV.opcode:
return variable
operator = nodes.UnaryOperator()
operator.operand = variable
if opcode == ins.ISTC.opcode \
or opcode == ins.IST.opcode \
or opcode == ins.NOT.opcode:
operator.type = nodes.UnaryOperator.T_NOT
elif opcode == ins.UNM.opcode:
operator.type = nodes.UnaryOperator.T_MINUS
else:
assert opcode == ins.LEN.opcode
operator.type = nodes.UnaryOperator.T_LENGTH_OPERATOR
return operator
def _build_slot(state, addr, slot):
return _build_identifier(state, addr, slot, nodes.Identifier.T_LOCAL)
def _build_upvalue(state, addr, slot):
return _build_identifier(state, addr, slot, nodes.Identifier.T_UPVALUE)
def _build_identifier(state, addr, slot, want_type):
node = nodes.Identifier()
setattr(node, "_addr", addr)
node.slot = slot
node.type = nodes.Identifier.T_SLOT
if want_type == nodes.Identifier.T_UPVALUE:
name = state.debuginfo.lookup_upvalue_name(slot)
if name is not None:
node.name = name
node.type = want_type
return node
def _build_global_variable(state, addr, slot):
node = nodes.TableElement()
node.table = nodes.Identifier()
node.table.type = nodes.Identifier.T_BUILTIN
node.table.name = "_env"
node.key = _build_string_constant(state, slot)
return node
def _build_string_constant(state, index):
node = nodes.Constant()
node.type = nodes.Constant.T_STRING
node.value = state.constants.complex_constants[index]
return node
def _build_cdata_constant(state, index):
node = nodes.Constant()
node.type = nodes.Constant.T_CDATA
node.value = state.constants.complex_constants[index]
return node
def _build_numeric_constant(state, index):
number = state.constants.numeric_constants[index]
node = nodes.Constant()
node.value = number
if isinstance(number, int):
node.type = nodes.Constant.T_INTEGER
else:
node.type = nodes.Constant.T_FLOAT
return node
def _build_primitive(state, value):
node = nodes.Primitive()
if value is True or value == T_TRUE:
node.type = nodes.Primitive.T_TRUE
elif value is False or value == T_FALSE:
node.type = nodes.Primitive.T_FALSE
else:
assert value is None or value == T_NIL
node.type = nodes.Primitive.T_NIL
return node
def _build_literal(state, value):
node = nodes.Constant()
node.value = value
node.type = nodes.Constant.T_INTEGER
return node
================================================
FILE: ljd/ast/helpers.py
================================================
import ljd.ast.nodes as nodes
import ljd.ast.traverse as traverse
def insert_table_record(constructor, key, value):
array = constructor.array.contents
records = constructor.records.contents
if isinstance(key, nodes.MULTRES):
assert len(records) == 0 \
or isinstance(records[-1], nodes.TableRecord)
records.append(value)
return
while isinstance(key, nodes.Constant) \
and key.type == key.T_INTEGER \
and key.value >= 0:
index = key.value
if index == 1 and len(array) == 0:
record = nodes.ArrayRecord()
record.value = nodes.Primitive()
record.value.type = nodes.Primitive.T_NIL
array.append(record)
if (index > len(array)):
break
record = nodes.ArrayRecord()
record.value = value
if len(array) == 0 or index == len(array):
array.append(record)
else:
array[index] = record
return
record = nodes.TableRecord()
record.key = key
record.value = value
if len(records) == 0:
records.append(record)
return
last = records[-1]
if isinstance(last, (nodes.FunctionCall, nodes.Vararg)):
records.insert(-1, record)
else:
records.append(record)
def has_same_table(node, table):
class Checker(traverse.Visitor):
def __init__(self, table):
self.found = False
self.table = table
def visit_table_element(self, node):
if is_equal(self.table, node):
self.found = True
def _visit(self, node):
if not self.found:
traverse.Visitor._visit(self, node)
def _visit_list(self, nodes_list):
if not self.found:
traverse.Visitor._visit_list(self, nodes_list)
checker = Checker(table)
traverse.traverse(checker, node)
return checker.found
def is_equal(a, b):
if type(a) != type(b):
return False
if isinstance(a, nodes.Identifier):
return a.type == b.type and a.slot == b.slot
elif isinstance(a, nodes.TableElement):
return is_equal(a.table, b.table) \
and is_equal(a.key, b.key)
else:
assert isinstance(a, nodes.Constant)
return a.type == b.type and a.value == b.value
================================================
FILE: ljd/ast/locals.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
import ljd.ast.nodes as nodes
import ljd.ast.traverse as traverse
def mark_locals(ast):
traverse.traverse(_LocalsMarker(), ast)
def mark_local_definitions(ast):
traverse.traverse(_LocalDefinitionsMarker(), ast)
class _LocalsMarker(traverse.Visitor):
class _State():
def __init__(self):
self.pending_slots = {}
self.debuginfo = None
self.addr = -1
def __init__(self):
self._states = []
# ##
def _push_state(self):
self._states.append(_LocalsMarker._State())
def _pop_state(self):
self._states.pop()
def _state(self):
return self._states[-1]
def _process_slots(self, addr):
debuginfo = self._state().debuginfo
cleanup = []
for slot, nodes in self._state().pending_slots.items():
varinfo = debuginfo.lookup_local_name(addr, slot)
if varinfo is None:
continue
cleanup.append(slot)
if varinfo.type == varinfo.T_INTERNAL:
continue
for node in nodes:
node.name = varinfo.name
node.type = node.T_LOCAL
#zzw.20180712
setattr(node, "_varinfo", varinfo)
for slot in cleanup:
del self._state().pending_slots[slot]
def _reset_slot(self, slot):
self._state().pending_slots.pop(slot, None)
def _reset_all(self, slots):
for slot in slots:
if isinstance(slot, nodes.Identifier):
self._reset_slot(slot.slot)
# ##
def visit_function_definition(self, node):
self._push_state()
self._state().debuginfo = node._debuginfo
def leave_function_definition(self, node):
addr = node._instructions_count
self._process_slots(addr)
self._pop_state()
# ##
def visit_variables_list(self, node):
# Last chance for a local = local + 1 type assignments
self._process_slots(self._state().addr)
self._reset_all(node.contents)
def visit_identifiers_list(self, node):
self._reset_all(node.contents)
def visit_numeric_loop_warp(self, node):
self._reset_slot(node.index.slot)
def visit_identifier(self, node):
if node.type == nodes.Identifier.T_SLOT:
queue = self._state().pending_slots
slots = queue.setdefault(node.slot, [])
slots.append(node)
# ##
def _process_worthy_node(self, node):
addr = getattr(node, "_addr", None)
if not isinstance(node, nodes.Identifier) and addr is not None:
assert self._state().addr <= addr
self._state().addr = addr
self._process_slots(addr)
# We need to process slots twice as it could be the last
# statement in the function/block and it could be anassignment
# as well so we need to process slots before the reset
def _leave_node(self, handler, node):
traverse.Visitor._leave_node(self, handler, node)
self._process_worthy_node(node)
def _visit_node(self, handler, node):
self._process_worthy_node(node)
traverse.Visitor._visit_node(self, handler, node)
class _LocalDefinitionsMarker(traverse.Visitor):
class _State():
def __init__(self):
self.known_locals = [None] * 255
self.addr = 0
def __init__(self):
self._states = []
def _push_state(self):
self._states.append(_LocalDefinitionsMarker._State())
def _pop_state(self):
self._states.pop()
def _state(self):
return self._states[-1]
def _update_known_locals(self, local, addr):
varinfo = self._state().known_locals[local.slot]
#zzw 20180712
self._state().known_locals[local.slot] = getattr(local, "_varinfo", None)#local._varinfo
if varinfo is None:
return False
if varinfo.end_addr <= addr:
return False
return True
# ##
def visit_function_definition(self, node):
self._push_state()
for local in node.arguments.contents:
if not isinstance(local, nodes.Vararg):
self._update_known_locals(local, 1)
def leave_function_definition(self, node):
self._pop_state()
def visit_iterator_for(self, node):
addr = node._addr
for local in node.identifiers.contents:
if local.type == nodes.Identifier.T_LOCAL:
self._update_known_locals(local, addr)
def visit_numeric_for(self, node):
addr = node._addr
if node.variable.type == nodes.Identifier.T_LOCAL:
self._update_known_locals(node.variable, addr)
# ##
def visit_assignment(self, node):
dst = node.destinations.contents[0]
addr = self._state().addr
if not isinstance(dst, nodes.Identifier):
return
if dst.type != nodes.Identifier.T_LOCAL:
return
known_slot = self._update_known_locals(dst, addr)
for slot in node.destinations.contents[1:]:
if not isinstance(slot, nodes.Identifier):
return
if slot.type != nodes.Identifier.T_LOCAL:
return
also_known = self._update_known_locals(slot, addr)
assert known_slot == also_known
if not known_slot:
node.type = nodes.Assignment.T_LOCAL_DEFINITION
def _visit(self, node):
node_addr = getattr(node, "_addr", -1)
if node_addr >= 0:
self._state().addr = node_addr
traverse.Visitor._visit(self, node)
================================================
FILE: ljd/ast/mutator.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
import copy
from ljd.ast.helpers import *
class SimpleLoopWarpSwapper(traverse.Visitor):
def visit_statements_list(self, node):
blocks = node.contents
fixed = []
index_shift = 0
for i, block in enumerate(node.contents):
warp = block.warp
fixed.append(block)
block.index += index_shift
if isinstance(warp, nodes.IteratorWarp):
self._swap_iterator_warps(blocks, block)
continue
if isinstance(warp, nodes.NumericLoopWarp):
self._swap_numeric_loop_warps(blocks, block)
continue
if isinstance(warp, nodes.UnconditionalWarp) \
and warp.is_uclo:
assert block != node.contents[-1]
next_block = node.contents[i + 1]
self._fix_uclo_return(block, next_block)
if not isinstance(warp, nodes.ConditionalWarp):
continue
if warp.true_target != warp.false_target:
continue
slot = getattr(warp, "_slot", -1)
if slot < 0:
continue
next_index = block.index - index_shift + 1
assert block.warp.false_target.index == next_index
new_block = self._create_dummy_block(block, slot)
fixed.append(new_block)
index_shift += 1
node.contents = fixed
def _create_dummy_block(self, block, slot):
new_block = nodes.Block()
new_block.first_address = block.last_address
new_block.last_address = new_block.first_address
new_block.index = block.index + 1
new_block.warpins_count = 1
new_block.warp = nodes.UnconditionalWarp()
new_block.warp.type = nodes.UnconditionalWarp.T_FLOW
new_block.warp.target = block.warp.false_target
statement = nodes.Assignment()
identifier = nodes.Identifier()
identifier.type = nodes.Identifier.T_SLOT
identifier.slot = slot
statement.destinations.contents.append(identifier)
statement.expressions.contents.append(copy.copy(identifier))
new_block.contents.append(statement)
block.warp.true_target = new_block
return new_block
def _fix_uclo_return(self, block, next_block):
warp = block.warp
target = warp.target
if len(target.contents) != 1:
return
statement = target.contents[0]
if not isinstance(statement, nodes.Return):
return
block.contents.append(statement)
statement._addr = block.last_address
target.contents = []
warp.type = nodes.UnconditionalWarp.T_FLOW
warp.target = next_block
def _swap_iterator_warps(self, blocks, end):
warp = end.warp
index = blocks.index(warp.body)
assert index > 0
start = blocks[index - 1]
assert isinstance(start.warp, nodes.UnconditionalWarp)
assert start.warp.type == nodes.UnconditionalWarp.T_JUMP
assert start.warp.target == end
end_addr = end.warp._addr
start_addr = start.warp._addr
new_end_warp = start.warp
new_end_warp._addr = end_addr
new_start_warp = end.warp
new_start_warp._addr = start_addr
end.warp = new_end_warp
start.warp = new_start_warp
new_end_warp.target = start
def _swap_numeric_loop_warps(self, blocks, end):
warp = end.warp
index = blocks.index(warp.body)
assert index > 0
start = blocks[index - 1]
assert isinstance(start.warp, nodes.UnconditionalWarp)
assert start.warp.type == nodes.UnconditionalWarp.T_FLOW
assert start.warp.target == warp.body
end_addr = end.warp._addr
start_addr = start.warp._addr
new_end_warp = start.warp
new_end_warp._addr = end_addr
new_start_warp = end.warp
new_start_warp._addr = start_addr
end.warp = new_end_warp
start.warp = new_start_warp
new_end_warp.type = nodes.UnconditionalWarp.T_JUMP
new_end_warp.target = start
class MutatorVisitor(traverse.Visitor):
# ##
def leave_if(self, node):
if len(node.else_block.contents) != 1:
return
subif = node.else_block.contents[0]
if not isinstance(subif, nodes.If):
return
elseif = nodes.ElseIf()
elseif.expression = subif.expression
elseif.then_block = subif.then_block
node.elseifs.append(elseif)
node.elseifs += subif.elseifs
node.else_block = subif.else_block
def visit_statements_list(self, node):
patched = []
i = -1
while i < len(node.contents) - 1:
i += 1
statement = node.contents[i]
patched.append(statement)
if not isinstance(statement, nodes.Assignment):
continue
src = statement.expressions.contents[0]
if not isinstance(src, nodes.TableConstructor):
continue
assert len(statement.destinations.contents) == 1
dst = statement.destinations.contents[0]
i += self._fill_constructor(dst, src, node.contents[i + 1:])
node.contents = patched
def _fill_constructor(self, table, constructor, statements):
consumed = 0
for statement in statements:
if not isinstance(statement, nodes.Assignment):
break
if len(statement.destinations.contents) > 1:
break
dst = statement.destinations.contents[0]
if not isinstance(dst, nodes.TableElement):
break
if not is_equal(dst.table, table):
break
assert len(statement.expressions.contents) == 1
src = statement.expressions.contents[0]
if has_same_table(src, table):
break
insert_table_record(constructor, dst.key, src)
consumed += 1
return consumed
def pre_pass(ast):
traverse.traverse(SimpleLoopWarpSwapper(), ast)
return ast
def primary_pass(ast):
traverse.traverse(MutatorVisitor(), ast)
return ast
================================================
FILE: ljd/ast/nodes.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
# We should visit stuff in it's execution order. That's important
class FunctionDefinition():
def __init__(self):
self.arguments = IdentifiersList()
self.statements = StatementsList()
self._upvalues = None
self._debuginfo = None
self._instructions_count = 0
def _accept(self, visitor):
visitor._visit_node(visitor.visit_function_definition, self)
visitor._visit(self.arguments)
visitor._visit(self.statements)
visitor._leave_node(visitor.leave_function_definition, self)
class TableConstructor():
def __init__(self):
self.array = RecordsList()
self.records = RecordsList()
def _accept(self, visitor):
visitor._visit_node(visitor.visit_table_constructor, self)
visitor._visit(self.array)
visitor._visit(self.records)
visitor._leave_node(visitor.leave_table_constructor, self)
class ArrayRecord():
def __init__(self):
self.value = None
def _accept(self, visitor):
visitor._visit_node(visitor.visit_array_record, self)
visitor._visit(self.value)
visitor._leave_node(visitor.leave_array_record, self)
class TableRecord():
def __init__(self):
self.key = None
self.value = None
def _accept(self, visitor):
visitor._visit_node(visitor.visit_table_record, self)
visitor._visit(self.key)
visitor._visit(self.value)
visitor._leave_node(visitor.leave_table_record, self)
class Assignment():
T_LOCAL_DEFINITION = 0
T_NORMAL = 1
def __init__(self):
self.expressions = ExpressionsList()
self.destinations = VariablesList()
self.type = -1
def _accept(self, visitor):
visitor._visit_node(visitor.visit_assignment, self)
visitor._visit(self.expressions)
visitor._visit(self.destinations)
visitor._leave_node(visitor.leave_assignment, self)
class BinaryOperator():
T_LOGICAL_OR = 0 # left or right
T_LOGICAL_AND = 10 # left and right
T_LESS_THEN = 20 # left < right
T_GREATER_THEN = 21 # left > right
T_LESS_OR_EQUAL = 22 # left <= right
T_GREATER_OR_EQUAL = 23 # left >= right
T_NOT_EQUAL = 24 # left ~= right
T_EQUAL = 25 # left == right
T_CONCAT = 30 # left .. right
T_ADD = 40 # left + right
T_SUBTRACT = 41 # left - right
T_MULTIPLY = 50 # left * right
T_DIVISION = 51 # left / right
T_MOD = 52 # left % right
T_POW = 70 # left ^ right
def __init__(self):
self.type = -1
self.left = None
self.right = None
def _accept(self, visitor):
visitor._visit_node(visitor.visit_binary_operator, self)
visitor._visit(self.left)
visitor._visit(self.right)
visitor._leave_node(visitor.leave_binary_operator, self)
class UnaryOperator():
T_NOT = 60 # not operand
T_LENGTH_OPERATOR = 61 # #operand
T_MINUS = 62 # -operand
def __init__(self):
self.type = -1
self.operand = None
def _accept(self, visitor):
visitor._visit_node(visitor.visit_unary_operator, self)
visitor._visit(self.operand)
visitor._leave_node(visitor.leave_unary_operator, self)
class StatementsList():
def __init__(self):
self.contents = []
def _accept(self, visitor):
visitor._visit_node(visitor.visit_statements_list, self)
visitor._visit_list(self.contents)
visitor._leave_node(visitor.leave_statements_list, self)
class IdentifiersList():
def __init__(self):
self.contents = []
def _accept(self, visitor):
visitor._visit_node(visitor.visit_identifiers_list, self)
visitor._visit_list(self.contents)
visitor._leave_node(visitor.leave_identifiers_list, self)
class RecordsList():
def __init__(self):
self.contents = []
def _accept(self, visitor):
visitor._visit_node(visitor.visit_records_list, self)
visitor._visit_list(self.contents)
visitor._leave_node(visitor.leave_records_list, self)
class VariablesList():
def __init__(self):
self.contents = []
def _accept(self, visitor):
visitor._visit_node(visitor.visit_variables_list, self)
visitor._visit_list(self.contents)
visitor._leave_node(visitor.leave_variables_list, self)
class ExpressionsList():
def __init__(self):
self.contents = []
def _accept(self, visitor):
visitor._visit_node(visitor.visit_expressions_list, self)
visitor._visit_list(self.contents)
visitor._leave_node(visitor.leave_expressions_list, self)
# Called Name in the Lua 5.1 reference
class Identifier():
T_SLOT = 0
T_LOCAL = 1
T_UPVALUE = 2
T_BUILTIN = 3
def __init__(self):
self.name = None
self.type = -1
self.slot = -1
def _accept(self, visitor):
visitor._visit_node(visitor.visit_identifier, self)
visitor._leave_node(visitor.leave_identifier, self)
# helper vararg/varreturn
class MULTRES():
def _accept(self, visitor):
visitor._visit_node(visitor.visit_multres, self)
visitor._leave_node(visitor.leave_multres, self)
class TableElement():
def __init__(self):
self.table = None
self.key = None
def _accept(self, visitor):
visitor._visit_node(visitor.visit_table_element, self)
visitor._visit(self.key)
visitor._visit(self.table)
visitor._leave_node(visitor.leave_table_element, self)
class Vararg():
def _accept(self, visitor):
visitor._visit_node(visitor.visit_vararg, self)
visitor._leave_node(visitor.leave_vararg, self)
class FunctionCall():
def __init__(self):
self.function = None
self.arguments = ExpressionsList()
def _accept(self, visitor):
visitor._visit_node(visitor.visit_function_call, self)
visitor._visit(self.arguments)
visitor._visit(self.function)
visitor._leave_node(visitor.leave_function_call, self)
class If():
def __init__(self):
self.expression = None
self.then_block = StatementsList()
self.elseifs = []
self.else_block = StatementsList()
def _accept(self, visitor):
visitor._visit_node(visitor.visit_if, self)
visitor._visit(self.expression)
visitor._visit(self.then_block)
visitor._visit_list(self.elseifs)
visitor._visit(self.else_block)
visitor._leave_node(visitor.leave_if, self)
class ElseIf():
def __init__(self):
self.expression = None
self.then_block = StatementsList()
def _accept(self, visitor):
visitor._visit_node(visitor.visit_elseif, self)
visitor._visit(self.expression)
visitor._visit(self.then_block)
visitor._leave_node(visitor.leave_elseif, self)
# ##
class Block():
def __init__(self):
self.index = -1
self.warp = None
self.contents = []
self.first_address = 0
self.last_address = 0
self.warpins_count = 0
def _accept(self, visitor):
visitor._visit_node(visitor.visit_block, self)
visitor._visit_list(self.contents)
visitor._visit(self.warp)
visitor._leave_node(visitor.leave_block, self)
class UnconditionalWarp():
T_JUMP = 0
T_FLOW = 1
def __init__(self):
self.type = -1
self.target = None
self.is_uclo = False
def _accept(self, visitor):
visitor._visit_node(visitor.visit_unconditional_warp, self)
# DO NOT VISIT self.target - warps are not part of the tree
visitor._leave_node(visitor.leave_unconditional_warp, self)
class ConditionalWarp():
def __init__(self):
self.condition = None
self.true_target = None
self.false_target = None
def _accept(self, visitor):
visitor._visit_node(visitor.visit_conditional_warp, self)
visitor._visit(self.condition)
# DO NOT VISIT self.true_target - warps are not part of the tree
# DO NOT VISIT self.false_target - warps are not part of the tree
visitor._leave_node(visitor.leave_conditional_warp, self)
class IteratorWarp():
def __init__(self):
self.variables = VariablesList()
self.controls = ExpressionsList()
self.body = None
self.way_out = None
def _accept(self, visitor):
visitor._visit_node(visitor.visit_iterator_warp, self)
visitor._visit(self.variables)
visitor._visit(self.controls)
# DO NOT VISIT self.body - warps are not part of the tree
# DO NOT VISIT self.way_out - warps are not part of the tree
visitor._leave_node(visitor.leave_iterator_warp, self)
class NumericLoopWarp():
def __init__(self):
self.index = Identifier()
self.controls = ExpressionsList()
self.body = None
self.way_out = None
def _accept(self, visitor):
visitor._visit_node(visitor.visit_numeric_loop_warp, self)
visitor._visit(self.index)
visitor._visit(self.controls)
# DO NOT VISIT self.body - warps are not part of the tree
# DO NOT VISIT self.way_out - warps are not part of the tree
visitor._leave_node(visitor.leave_numeric_loop_warp, self)
class EndWarp():
def _accept(self, visitor):
visitor._visit_node(visitor.visit_end_warp, self)
visitor._leave_node(visitor.leave_end_warp, self)
# ##
class Return():
def __init__(self):
self.returns = ExpressionsList()
def _accept(self, visitor):
visitor._visit_node(visitor.visit_return, self)
visitor._visit(self.returns)
visitor._leave_node(visitor.leave_return, self)
class Break():
def _accept(self, visitor):
visitor._visit_node(visitor.visit_break, self)
visitor._leave_node(visitor.leave_break, self)
class While():
def __init__(self):
self.expression = None
self.statements = StatementsList()
def _accept(self, visitor):
visitor._visit_node(visitor.visit_while, self)
visitor._visit(self.expression)
visitor._visit(self.statements)
visitor._leave_node(visitor.leave_while, self)
class RepeatUntil():
def __init__(self):
self.expression = None
self.statements = StatementsList()
def _accept(self, visitor):
visitor._visit_node(visitor.visit_repeat_until, self)
visitor._visit(self.statements)
visitor._visit(self.expression)
visitor._leave_node(visitor.leave_repeat_until, self)
class NumericFor():
def __init__(self):
self.variable = None
self.expressions = ExpressionsList()
self.statements = StatementsList()
def _accept(self, visitor):
visitor._visit_node(visitor.visit_numeric_for, self)
visitor._visit(self.variable)
visitor._visit(self.expressions)
visitor._visit(self.statements)
visitor._leave_node(visitor.leave_numeric_for, self)
class IteratorFor():
def __init__(self):
self.expressions = ExpressionsList()
self.identifiers = VariablesList()
self.statements = StatementsList()
def _accept(self, visitor):
visitor._visit_node(visitor.visit_iterator_for, self)
visitor._visit(self.expressions)
visitor._visit(self.identifiers)
visitor._visit(self.statements)
visitor._leave_node(visitor.leave_iterator_for, self)
class Constant():
T_INTEGER = 0
T_FLOAT = 1
T_STRING = 2
T_CDATA = 3
def __init__(self):
self.type = -1
self.value = None
def _accept(self, visitor):
visitor._visit_node(visitor.visit_constant, self)
visitor._leave_node(visitor.leave_constant, self)
class Primitive():
T_NIL = 0
T_TRUE = 1
T_FALSE = 2
def __init__(self):
self.type = -1
def _accept(self, visitor):
visitor._visit_node(visitor.visit_primitive, self)
visitor._leave_node(visitor.leave_primitive, self)
================================================
FILE: ljd/ast/slotworks.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
import ljd.ast.nodes as nodes
import ljd.ast.traverse as traverse
from ljd.ast.helpers import insert_table_record
def eliminate_temporary(ast):
_eliminate_multres(ast)
slots, unused = _collect_slots(ast)
_eliminate_temporary(slots)
# _remove_unused(unused)
_cleanup_invalid_nodes(ast)
return ast
def _eliminate_temporary(slots):
simple = []
massive = []
tables = []
iterators = []
for info in slots:
assignment = info.assignment
if not isinstance(assignment, nodes.Assignment):
assert isinstance(assignment, (nodes.IteratorWarp,
nodes.NumericLoopWarp,
nodes.FunctionDefinition))
src = info.references[1].identifier
simple.append((info.references, src))
continue
assert len(assignment.expressions.contents) == 1
is_massive = len(assignment.destinations.contents) > 1
if is_massive:
_fill_massive_refs(info, simple, massive, iterators)
else:
_fill_simple_refs(info, simple, tables)
_eliminate_simple_cases(simple)
_eliminate_into_table_constructors(tables)
_eliminate_mass_assignments(massive)
_eliminate_iterators(iterators)
def _fill_massive_refs(info, simple, massive, iterators):
ref = info.references[1]
holder = _get_holder(ref.path)
src = info.assignment.expressions.contents[0]
assert isinstance(src, (nodes.FunctionCall,
nodes.Vararg,
nodes.Primitive))
if isinstance(holder, nodes.Assignment):
dst = holder.destinations.contents[0]
assert len(info.references) == 2
orig = info.references[0].identifier
assignment = ref.path[-3]
assert isinstance(assignment, nodes.Assignment)
massive.append((orig, info.assignment, assignment, dst))
elif isinstance(holder, nodes.IteratorWarp):
assert len(info.references) == 2
iterators.append((info.assignment, src, holder))
elif isinstance(src, nodes.Primitive) and src.type == src.T_NIL:
assert len(info.references) == 2
# Create a new primitive, so it won't mess with the
# writer's ignore list
src = nodes.Primitive()
src.type = nodes.Primitive.T_NIL
simple.append((info, ref, src))
def _fill_simple_refs(info, simple, tables):
src = info.assignment.expressions.contents[0]
if isinstance(src, nodes.FunctionCall) and len(info.references) > 3:
return
src_is_table = isinstance(src, nodes.TableConstructor)
for ref in info.references[1:]:
holder = _get_holder(ref.path)
is_element = isinstance(holder, nodes.TableElement)
path_index = ref.path.index(holder)
statement = _get_holder(ref.path[:path_index])
statement_is_assignment = isinstance(statement, nodes.Assignment)
if statement_is_assignment:
is_dst = statement.destinations.contents[0] == holder
else:
is_dst = False
# Could be more then one reference here
if src_is_table and is_element and is_dst:
assert holder.table == ref.identifier
tables.append((info, ref))
else:
simple.append((info, ref, None))
LIST_TYPES = (nodes.VariablesList,
nodes.IdentifiersList,
nodes.ExpressionsList,
nodes.StatementsList)
def _get_holder(path):
for node in reversed(path[:-1]):
if not isinstance(node, LIST_TYPES):
return node
return None
def _eliminate_simple_cases(simple):
for info, ref, src in simple:
holder = ref.path[-2]
dst = ref.identifier
if src is None:
src = info.assignment.expressions.contents[0]
_mark_invalidated(info.assignment)
if isinstance(holder, LIST_TYPES):
nodes = holder.contents
found = _replace_node_in_list(nodes, dst, src)
else:
found = _replace_node(holder, dst, src)
if found == None:
print("err: slotworks.py assert found")
def _eliminate_into_table_constructors(tables):
for info, ref in tables:
constructor = info.assignment.expressions.contents[0]
table_element = ref.path[-2]
assignment = ref.path[-4]
assert isinstance(assignment, nodes.Assignment)
assert len(assignment.expressions.contents) == 1
_mark_invalidated(assignment)
key = table_element.key
value = assignment.expressions.contents[0]
insert_table_record(constructor, key, value)
def _eliminate_mass_assignments(massive):
for identifier, assignment, base_assignment, globalvar in massive:
destinations = assignment.destinations.contents
found = _replace_node_in_list(destinations, identifier, globalvar)
_mark_invalidated(base_assignment)
assert found
def _replace_node(holder, original, replacement):
for key, value in holder.__dict__.items():
if value == original:
setattr(holder, key, replacement)
return True
return False
def _replace_node_in_list(nodes, original, replacement):
try:
index = nodes.index(original)
except ValueError:
return False
nodes[index] = replacement
return True
def _eliminate_iterators(iterators):
processed_warps = set()
for assignment, src, warp in iterators:
if warp in processed_warps:
continue
for i, slot in enumerate(assignment.destinations.contents):
if hasattr(warp.controls.contents[i], 'slot') and hasattr(slot, 'slot'):
#assert warp.controls.contents[i].slot == slot.slot
if warp.controls.contents[i].slot != slot.slot:
print ("err: slotworks.py assert warp.controls.contents[i].slot == slot.slot")
warp.controls.contents = [src]
processed_warps.add(warp)
_mark_invalidated(assignment)
def _mark_invalidated(node):
setattr(node, "_invalidated", True)
def _is_invalidated(node):
return getattr(node, "_invalidated", False)
def _remove_unused(unused):
pass
def _collect_slots(ast):
collector = _SlotsCollector()
traverse.traverse(collector, ast)
return collector.slots, collector.unused
def _eliminate_multres(ast):
traverse.traverse(_MultresEliminator(), ast)
_cleanup_invalid_nodes(ast)
class _MultresEliminator(traverse.Visitor):
def __init__(self):
self._last_multres_value = None
def leave_assignment(self, node):
src = node.expressions.contents[0]
dst = node.destinations.contents[0]
if isinstance(dst, nodes.MULTRES):
assert len(node.destinations.contents) == 1
assert len(node.expressions.contents) == 1
assert isinstance(src, (nodes.FunctionCall, nodes.Vararg))
assert self._last_multres_value is None
self._last_multres_value = src
_mark_invalidated(node)
else:
for i, src in enumerate(node.expressions.contents):
if isinstance(src, nodes.MULTRES):
break
else:
return
assert self._last_multres_value is not None
node.expressions.contents[i] = self._last_multres_value
self._last_multres_value = None
def _process_multres_in_list(self, nodes_list):
for i, node in enumerate(nodes_list):
if isinstance(node, nodes.MULTRES):
break
else:
return
assert self._last_multres_value is not None
nodes_list[i] = self._last_multres_value
self._last_multres_value = None
def visit_function_call(self, node):
self._process_multres_in_list(node.arguments.contents)
def visit_return(self, node):
self._process_multres_in_list(node.returns.contents)
class _SlotReference():
def __init__(self):
self.path = []
self.identifier = None
class _SlotInfo():
def __init__(self):
self.slot = 0
self.assignment = None
self.references = []
self.termination = None
self.function = None
class _SlotsCollector(traverse.Visitor):
class _State():
def __init__(self):
self.known_slots = {}
self.function = None
# ##
def __init__(self):
self._states = []
self._path = []
self._skip = None
self.slots = []
self.unused = []
self._push_state()
# ##
def _state(self):
return self._states[-1]
def _push_state(self):
self._states.append(_SlotsCollector._State())
def _pop_state(self):
self._states.pop()
def _commit_info(self, info):
assert len(info.references) > 0
if len(info.references) == 1:
self.unused.append(info)
else:
self.slots.append(info)
def _commit_slot(self, slot, node):
info = self._state().known_slots.get(slot)
if info is None:
return
info.termination = node
del self._state().known_slots[slot]
self._commit_info(info)
def _register_slot(self, slot, node):
self._commit_slot(slot, node)
info = _SlotInfo()
info.slot = slot
info.assignment = node
info.function = self._state().function
self._state().known_slots[slot] = info
def _register_all_slots(self, node, slots):
for slot in slots:
if not isinstance(slot, nodes.Identifier):
continue
if slot.type != nodes.Identifier.T_SLOT:
continue
self._register_slot(slot.slot, node)
def _commit_all_slots(self, slots, node):
for slot in slots:
if not isinstance(slot, nodes.Identifier):
continue
self._commit_slot(slot.slot, node)
def _register_slot_reference(self, slot, node):
info = self._state().known_slots.get(slot)
if info is None:
return
reference = _SlotReference()
reference.identifier = node
# Copy the list, but not contents
reference.path = self._path[:]
info.references.append(reference)
# ##
def visit_assignment(self, node):
self._visit(node.expressions)
self._skip = node.expressions
self._register_all_slots(node, node.destinations.contents)
def leave_assignment(self, node):
self._skip = None
def visit_identifier(self, node):
if node.type == nodes.Identifier.T_SLOT:
self._register_slot_reference(node.slot, node)
# ##
def visit_function_definition(self, node):
self._push_state()
self._state().function = node
def leave_function_definition(self, node):
self._pop_state()
def leave_block(self, node):
for info in self._state().known_slots.values():
self._commit_info(info)
self._state().known_slots = {}
def visit_iterator_warp(self, node):
self._commit_all_slots(node.variables.contents, node)
def visit_numeric_loop_warp(self, node):
self._commit_slot(node.index.slot, node)
# ##
def _visit_node(self, handler, node):
self._path.append(node)
traverse.Visitor._visit_node(self, handler, node)
def _leave_node(self, handler, node):
self._path.pop()
traverse.Visitor._leave_node(self, handler, node)
def _visit(self, node):
if self._skip == node:
return
traverse.Visitor._visit(self, node)
def _cleanup_invalid_nodes(ast):
traverse.traverse(_TreeCleanup(), ast)
class _TreeCleanup(traverse.Visitor):
def visit_block(self, node):
patched = []
for subnode in node.contents:
if not _is_invalidated(subnode):
patched.append(subnode)
node.contents = patched
================================================
FILE: ljd/ast/traverse.py
================================================
class Visitor():
def __init__(self):
pass
# ##
def visit_function_definition(self, node):
pass
def leave_function_definition(self, node):
pass
# ##
def visit_table_constructor(self, node):
pass
def leave_table_constructor(self, node):
pass
def visit_table_record(self, node):
pass
def leave_table_record(self, node):
pass
def visit_array_record(self, node):
pass
def leave_array_record(self, node):
pass
# ##
def visit_assignment(self, node):
pass
def leave_assignment(self, node):
pass
# ##
def visit_binary_operator(self, node):
pass
def leave_binary_operator(self, node):
pass
def visit_unary_operator(self, node):
pass
def leave_unary_operator(self, node):
pass
# ##
def visit_statements_list(self, node):
pass
def leave_statements_list(self, node):
pass
def visit_identifiers_list(self, node):
pass
def leave_identifiers_list(self, node):
pass
def visit_records_list(self, node):
pass
def leave_records_list(self, node):
pass
def visit_variables_list(self, node):
pass
def leave_variables_list(self, node):
pass
def visit_expressions_list(self, node):
pass
def leave_expressions_list(self, node):
pass
# ##
def visit_identifier(self, node):
pass
def leave_identifier(self, node):
pass
def visit_multres(self, node):
pass
def leave_multres(self, node):
pass
def visit_table_element(self, node):
pass
def leave_table_element(self, node):
pass
def visit_vararg(self, node):
pass
def leave_vararg(self, node):
pass
def visit_function_call(self, node):
pass
def leave_function_call(self, node):
pass
# ##
def visit_if(self, node):
pass
def leave_if(self, node):
pass
def visit_elseif(self, node):
pass
def leave_elseif(self, node):
pass
# ##
def visit_block(self, node):
pass
def leave_block(self, node):
pass
def visit_unconditional_warp(self, node):
pass
def leave_unconditional_warp(self, node):
pass
def visit_conditional_warp(self, node):
pass
def leave_conditional_warp(self, node):
pass
def visit_iterator_warp(self, node):
pass
def leave_iterator_warp(self, node):
pass
def visit_numeric_loop_warp(self, node):
pass
def leave_numeric_loop_warp(self, node):
pass
def visit_end_warp(self, node):
pass
def leave_end_warp(self, node):
pass
# ##
def visit_return(self, node):
pass
def leave_return(self, node):
pass
def visit_break(self, node):
pass
def leave_break(self, node):
pass
# ##
def visit_while(self, node):
pass
def leave_while(self, node):
pass
def visit_repeat_until(self, node):
pass
def leave_repeat_until(self, node):
pass
def visit_numeric_for(self, node):
pass
def leave_numeric_for(self, node):
pass
def visit_iterator_for(self, node):
pass
def leave_iterator_for(self, node):
pass
# ##
def visit_constant(self, node):
pass
def leave_constant(self, node):
pass
def visit_primitive(self, node):
pass
def leave_primitive(self, node):
pass
# ##
def _visit_node(self, handler, node):
handler(node)
def _leave_node(self, handler, node):
handler(node)
# ##
def _visit(self, node):
assert node is not None
node._accept(self)
def _visit_list(self, nodes_list):
assert isinstance(nodes_list, list)
for node in nodes_list:
self._visit(node)
def traverse(visitor, node):
if isinstance(node, list):
visitor._visit_list(node)
else:
visitor._visit(node)
================================================
FILE: ljd/ast/unwarper.py
================================================
import copy
import ljd.ast.nodes as nodes
import ljd.ast.traverse as traverse
import ljd.ast.slotworks as slotworks
binop = nodes.BinaryOperator
# ##
# ## REMEMBER
# ##
# ## Block indices are unreliable while you are mangling them!
# ##
# ## P.S. Probably they should not be named indices... But they ARE used as indices during other phases. Sometimes.
# ##
# ##
class _StatementsCollector(traverse.Visitor):
def __init__(self):
self.result = []
def visit_statements_list(self, node):
if len(node.contents) > 0:
self.result.append(node)
def unwarp(node):
# There could be many negative jumps within while conditions, so
# filter them first
_run_step(_unwarp_loops, node, repeat_until=False)
_run_step(_unwarp_loops, node, repeat_until=True)
_run_step(_unwarp_expressions, node)
_run_step(_unwarp_ifs, node)
_glue_flows(node)
def _run_step(step, node, **kargs):
for statements in _gather_statements_lists(node):
statements.contents = step(statements.contents, **kargs)
# Fix block indices in case anything was moved
for statements in _gather_statements_lists(node):
for i, block in enumerate(statements.contents):
block.index = i
def _gather_statements_lists(node):
collector = _StatementsCollector()
traverse.traverse(collector, node)
return collector.result
def _glue_flows(node):
for statements in _gather_statements_lists(node):
blocks = statements.contents
if hasattr(blocks[-1], 'warp'):
assert isinstance(blocks[-1].warp, nodes.EndWarp)
for i, block in enumerate(blocks[:-1]):
warp = block.warp
assert _is_flow(warp)
target = warp.target
assert target == blocks[i + 1]
target.contents = block.contents + target.contents
block.contents = []
statements.contents = blocks[-1].contents
# ##
# ## IFs AND EXPRESSIONs PROCESSING
# ##
def _unwarp_expressions(blocks):
pack = []
pack_set = set()
start_index = 0
while start_index < len(blocks) - 1:
start = blocks[start_index]
warp = start.warp
if isinstance(warp, nodes.UnconditionalWarp):
if warp.type == nodes.UnconditionalWarp.T_FLOW:
start_index += 1
continue
body, end, end_index = _extract_if_body(start_index,
blocks, None)
if body is None:
raise NotImplementedError("GOTO statements are not"
" supported")
expressions = _find_expressions(start, body, end)
assert pack_set.isdisjoint(expressions)
expressions_set = set(expressions)
assert len(expressions_set) == len(expressions)
if len(expressions) == 0:
start_index += 1
continue
pack += list(reversed(expressions))
pack_set |= expressions_set
endest_end = _find_endest_end(expressions)
if endest_end != end:
end_index = blocks.index(endest_end)
start_index = end_index
return _unwarp_expressions_pack(blocks, pack)
def _find_endest_end(expressions):
endest_end = expressions[0][1]
for _start, exp_end, _slot, _slot_type in expressions[1:]:
if exp_end.index > endest_end.index:
endest_end = exp_end
return endest_end
def _unwarp_ifs(blocks, top_end=None, topmost_end=None):
boundaries = []
start_index = 0
while start_index < len(blocks) - 1:
start = blocks[start_index]
warp = start.warp
if isinstance(warp, nodes.UnconditionalWarp):
if warp.type == nodes.UnconditionalWarp.T_FLOW:
start_index += 1
continue
body, end, end_index = _extract_if_body(start_index,
blocks, topmost_end)
if body is None:
raise NotImplementedError("GOTO statements are not"
" supported")
is_end = isinstance(body[-1].warp, nodes.EndWarp)
_unwarp_if_statement(start, body, end, end)
if is_end:
_set_end(start)
else:
_set_flow_to(start, end)
boundaries.append((start_index, end_index - 1))
start_index = end_index
return _remove_processed_blocks(blocks, boundaries)
def _extract_if_body(start_index, blocks, topmost_end):
end = _find_branching_end(blocks[start_index:], topmost_end)
try:
end_index = blocks.index(end)
except ValueError:
if end == topmost_end:
end_index = len(blocks)
else:
return None, None, None
body = blocks[start_index + 1:end_index]
return body, end, end_index
def _unwarp_expressions_pack(blocks, pack):
replacements = {}
for start, end, slot, slot_type in reversed(pack):
end = replacements.get(end, end)
start_index = blocks.index(start)
end_index = blocks.index(end)
body = blocks[start_index + 1:end_index]
_unwarp_logical_expression(start, end, body)
statements = start.contents + end.contents
if slot_type == nodes.Identifier.T_SLOT:
min_i = len(start.contents)
split_i = _split_by_slot_use(statements, min_i,
end.warp, slot)
else:
split_i = len(start.contents)
max_i = len(statements)
if split_i > max_i:
end.contents = start.contents + end.contents
start.contents = []
blocks = blocks[:start_index] + blocks[end_index:]
_replace_targets(blocks, start, end)
replacements[start] = end
slotworks.eliminate_temporary(end)
_set_flow_to(start, end)
else:
blocks = blocks[:start_index + 1] + blocks[end_index:]
start.contents = statements[:split_i]
end.contents = statements[split_i:]
# We need to kill the start's warp before slot
# elimination or it could result in a cycled AST.
_set_flow_to(start, end)
slotworks.eliminate_temporary(start)
return blocks
def _split_by_slot_use(statements, min_i, warp, slot):
known_slots = set([slot])
split_i = min_i
for i, statement in enumerate(statements):
if isinstance(statement, nodes.Assignment):
sets = _extract_destination_slots(statement)
if i < min_i:
known_slots |= sets
else:
known_slots -= sets
if len(known_slots) == 0:
break
split_i = i + 1
if split_i < len(statements):
return split_i
if isinstance(warp, nodes.ConditionalWarp):
known_slots -= _gather_slots(warp)
if len(known_slots) == 0:
split_i += 1
return split_i
def _extract_destination_slots(statement):
sets = set()
for node in statement.destinations.contents:
if not isinstance(node, nodes.Identifier):
continue
if node.type == nodes.Identifier.T_SLOT:
sets.add(node.slot)
return sets
def _gather_slots(node):
class Collector(traverse.Visitor):
def __init__(self):
self.slots = set()
def visit_identifier(self, node):
if node.type == nodes.Identifier.T_SLOT:
self.slots.add(node.slot)
collector = Collector()
traverse.traverse(collector, node)
return collector.slots
def _find_expressions(start, body, end):
# Explicitly allow the local a = x ~= "b" case
slot, slot_type = _get_simple_local_assignment_slot(start, body, end)
if slot >= 0:
return [(start, end, slot, slot_type)]
expressions = []
# We have something at the end, but not the true/false?
i = 0
extbody = [start] + body
is_local = False
sure_expression = False
while i < len(extbody):
block = extbody[i]
subs = _find_subexpressions(block, body[i:])
if len(subs) != 0:
endest_end = _find_endest_end(subs)
new_i = extbody.index(endest_end)
# Loop? No way!
if new_i <= i:
return expressions
# It should end with a conditional warp if that's
# really a subexpression-as-operand
end_warp = endest_end.warp
if not isinstance(end_warp, nodes.ConditionalWarp):
return expressions
expressions = subs + expressions
i = new_i
continue
if isinstance(block.warp, nodes.ConditionalWarp):
condition = block.warp.condition
is_end = block.warp.false_target == end
is_binop = isinstance(condition, nodes.BinaryOperator)
block_slot = getattr(block.warp, "_slot", slot)
if is_end:
if is_binop:
return expressions
elif slot < 0 and block_slot >= 0:
slot = block_slot
slot_type = nodes.Identifier.T_SLOT
sure_expression = True
elif slot != block_slot:
return expressions
else:
sure_expression = True
elif isinstance(block.warp, nodes.UnconditionalWarp):
if block == start and len(block.contents) == 0:
return []
if len(block.contents) == 0:
i += 1
continue
if block != start and len(block.contents) > 1:
return expressions
assignment = block.contents[-1]
if not isinstance(assignment, nodes.Assignment):
if block == start:
i += 1
continue
return expressions
destinations = assignment.destinations.contents
if len(destinations) != 1:
if block == start:
i += 1
continue
return expressions
dst = destinations[0]
if not isinstance(dst, nodes.Identifier):
if block == start:
i += 1
continue
return expressions
if isinstance(block.warp, nodes.ConditionalWarp):
if block == start:
i += 1
continue
return expressions
if slot < 0:
# If all encounters are locals, which means
# that the first encounter is a local
if dst.type == nodes.Identifier.T_LOCAL:
is_local = True
elif dst.type == nodes.Identifier.T_UPVALUE:
return []
slot = dst.slot
slot_type = dst.type
elif slot == dst.slot:
slot_type = dst.type
if dst.type == nodes.Identifier.T_UPVALUE:
return []
else:
assert block != start
return []
i += 1
if slot < 0:
return []
true, _false, body = _get_terminators(body)
if true is not None:
sure_expression = True
if len(expressions) > 0:
sure_expression = True
if not sure_expression and is_local:
return expressions
return expressions + [(start, end, slot, slot_type)]
def _find_subexpressions(start, body):
try:
body, end, _end_index = _extract_if_body(0, [start] + body, None)
except ValueError:
# a warp target is not in a list
return []
if body is None:
return []
return _find_expressions(start, body, end)
def _get_simple_local_assignment_slot(start, body, end):
if len(body) != 2:
return -1, None
true, _false, body = _get_terminators(body)
if true is None:
return -1, None
else:
slot = true.contents[0].destinations.contents[0]
return slot.slot, slot.type
def _find_expression_slot(body):
slot = None
for block in reversed(body):
if len(block.contents) != 1:
continue
slot = block.contents[0].destinations.contents[0]
break
return slot
def _unwarp_logical_expression(start, end, body):
slot = _find_expression_slot(body)
assert slot is not None
true, false, body = _get_terminators(body)
expression = _compile_expression([start] + body, end, true, false)
dst = copy.deepcopy(slot)
assignment = nodes.Assignment()
assignment.destinations.contents.append(dst)
assignment.expressions.contents.append(expression)
start.contents.append(assignment)
def _compile_expression(body, end, true, false):
parts = _unwarp_expression(body, end, true, false)
if len(parts) < 3:
assert len(parts) == 1
return parts[0]
explicit_parts = _make_explicit_subexpressions(parts)
return _assemble_expression(explicit_parts)
#
# The logical expressions:
#
# There are terminators: a true, a false and an end
#
# For an if case the true will be a "then" clause and the false - an "else" or
# "after-the-if" clause. The end is required for a single-variable (unary)
# components and is only used at during topmost phase.
#
# The last block in expression is always "double terminal" - both ends are
# pointing at terminators. It's rather useless so we just append it's condition
# (inverted if needed - it's easy to see if true end targets the false
# terminator) at the end of processing.
#
# Then we need to pack all other blocks into subexpressions. Subexpressions
# always end with a _terminal block_, i.e. the block which warp points to a
# terminator. Idea is that we can guess the operator only right of a
# terminal block, because we can check if the block's warp condition is
# inverted or not.
#
# If that's an "OR" clause then it will jump out of the current expression if
# the condition is true, so the condition is inverted and the false branch is
# pointing at the way out (at the TRUE terminator - because the result of the
# expression level will be true). (because in the bytecode there
# is only one JMP, so a ConditionalWarp's true branch is actually a fake and
# always points to the next block - in the bytecode a "positive" jump
# will be represented by a normal negative jump with inverted condition).
#
# Otherwise, if that's an "AND" clause then it will jump out of the current
# expression level if a condition is false, so the condition is not inverted
# and a false branch points to the false.
#
# This way we can understand which operators go just right of terminal blocks.
# Everything in-between these block is considered a subexpression. And just
# because we don't know where exactly the subexpression ends we are using
# greedy approach and trying to pack into a subexpression as much blocks as
# possible, including any terminal blocks pointing at the same terminator
# with the same inversion status (that is - we are always using the
# rightmost block if there are consequitive similar terminal blocks, ignoring
# all the blocks to the left).
#
# Then comes the trick: the subexpression is a component of this expression and
# we know the operator to the right of it. We can guess now what will
# be evaluated if the subexpression evaluates to "false" and what - if it's
# "true". If the operator is "AND" then the subexpression failure will cause
# the expression failure too, i.e. the "FALSE" target remains the same and the
# true terminator is set to the next block (after the "AND").
#
# If the operator is "OR" the the subexpression success will cause the success
# of the expression, so the "TRUE" target remains the same, but the false
# target is set to the next block (after the "OR").
#
# Now we have a subexpression and both TRUE and FALSE terminators for it.
# Recurse and repeat.
#
def _unwarp_expression(body, end, true, false):
parts = []
if true is not None:
terminator_index = min(true.index, false.index)
if end is not None:
terminator_index = min(end.index, terminator_index)
else:
assert end is not None
terminator_index = end.index
terminators = set((true, false, end))
subexpression_start = 0
i = 0
while i < len(body) - 1:
block = body[i]
warp = block.warp
target = _get_target(warp)
#
# A chance for
# (foo and (bar and y or z)) or x
# type expressions, because the first "foo and ... )) or" part
# will be broken by the "or z))" part in the code below.
#
# So we are going to intercept subexpressions by it's start
# instead of an end, but only if we are already at the
# subexpression start (so nothing formally changes, but creates
# a bit more correct execution order)
#
if target.index < terminator_index:
if i != subexpression_start:
i += 1
continue
target_index = body.index(target)
last_block = body[target_index - 1]
last_block_target = _get_target(last_block.warp)
if last_block_target.index < terminator_index:
i += 1
continue
assert last_block_target in terminators
subexpression = body[i:target_index]
else:
assert target in terminators
while i < len(body) - 2:
next_block = body[i + 1]
next_target = _get_target(next_block.warp)
if next_target != target:
break
next_inv = _is_inverted(next_block.warp, true, end)
this_inv = _is_inverted(warp, true, end)
# Special hack for unary expressions (x, not x)...
if next_inv != this_inv:
break
block = next_block
warp = next_block.warp
i += 1
subexpression = body[subexpression_start:i + 1]
last_block = subexpression[-1]
last_block_index = body.index(last_block)
next_block = body[last_block_index + 1]
operator = _get_operator(last_block, true, end)
subexpression = _compile_subexpression(subexpression, operator,
last_block, next_block,
true, end)
parts.append(subexpression)
parts.append(operator)
i = last_block_index + 1
subexpression_start = i
last = body[-1]
if isinstance(last.warp, nodes.ConditionalWarp):
if _is_inverted(last.warp, true, end):
last = _invert(last.warp.condition)
else:
last = last.warp.condition
parts.append(last)
else:
assert isinstance(last.warp, (nodes.EndWarp,
nodes.UnconditionalWarp))
src = _get_last_assignment_source(last)
if src is None:
src = nodes.Primitive()
if last.warp.target == true:
src.type = nodes.Primitive.T_TRUE
else:
src.type = nodes.Primitive.T_FALSE
parts.append(src)
return parts
def _get_target(warp, allow_end=False):
if isinstance(warp, nodes.ConditionalWarp):
return warp.false_target
else:
if allow_end and isinstance(warp, nodes.EndWarp):
return getattr(warp, "_target", None)
assert isinstance(warp, nodes.UnconditionalWarp)
return warp.target
def _set_target(warp, target):
if isinstance(warp, nodes.ConditionalWarp):
warp.false_target = target
else:
assert isinstance(warp, nodes.UnconditionalWarp)
warp.target = target
def _get_operator(block, true, end):
if isinstance(block.warp, nodes.UnconditionalWarp):
src = _get_last_assignment_source(block)
if isinstance(src, nodes.Constant):
is_true = src.value != 0
elif isinstance(src, nodes.BinaryOperator):
is_true = True
elif isinstance(src, nodes.Primitive):
is_true = src.type == nodes.Primitive.T_TRUE
elif isinstance(src, nodes.Identifier):
is_true = True
else:
#assert src is None
is_true = block.warp.target == true
if is_true:
return binop.T_LOGICAL_OR
else:
return binop.T_LOGICAL_AND
else:
is_inverted = _is_inverted(block.warp, true, end)
if is_inverted:
return binop.T_LOGICAL_OR
else:
return binop.T_LOGICAL_AND
def _get_last_assignment_source(block):
if len(block.contents) == 0:
return None
assignment = block.contents[-1]
assert isinstance(assignment, nodes.Assignment)
return assignment.expressions.contents[0]
def _get_and_remove_last_assignment_source(block):
assignment = block.contents.pop()
assert isinstance(assignment, nodes.Assignment)
return assignment.expressions.contents[0]
def _compile_subexpression(subexpression, operator,
block, next_block, true, end):
warp = block.warp
if len(subexpression) == 1:
if isinstance(warp, nodes.UnconditionalWarp):
return _get_and_remove_last_assignment_source(block)
elif _is_inverted(warp, true, end):
return _invert(warp.condition)
else:
return warp.condition
else:
if isinstance(warp, nodes.UnconditionalWarp):
if operator == binop.T_LOGICAL_OR:
subtrue = warp.target
subfalse = next_block
else:
subtrue = next_block
subfalse = warp.target
else:
if operator == binop.T_LOGICAL_OR:
subtrue = warp.false_target
subfalse = warp.true_target
else:
subtrue = warp.true_target
subfalse = warp.false_target
return _unwarp_expression(subexpression, None, subtrue, subfalse)
def _is_inverted(warp, true, end):
if isinstance(warp, nodes.UnconditionalWarp):
return warp.target == end
if warp.false_target == true:
return True
elif warp.false_target == end:
assert not isinstance(warp.condition, nodes.BinaryOperator)
if not isinstance(warp.condition, nodes.UnaryOperator):
return False
return warp.condition.type == nodes.UnaryOperator.T_NOT
return False
_NEGATION_MAP = [None] * 100
_NEGATION_MAP[binop.T_LESS_THEN] = binop.T_GREATER_OR_EQUAL
_NEGATION_MAP[binop.T_GREATER_THEN] = binop.T_LESS_OR_EQUAL
_NEGATION_MAP[binop.T_LESS_OR_EQUAL] = binop.T_GREATER_THEN
_NEGATION_MAP[binop.T_GREATER_OR_EQUAL] = binop.T_LESS_THEN
_NEGATION_MAP[binop.T_NOT_EQUAL] = binop.T_EQUAL
_NEGATION_MAP[binop.T_EQUAL] = binop.T_NOT_EQUAL
def _invert(expression):
if isinstance(expression, nodes.UnaryOperator):
return expression.operand
if not isinstance(expression, nodes.BinaryOperator):
node = nodes.UnaryOperator()
node.type = nodes.UnaryOperator.T_NOT
node.operand = expression
return node
# Just in case
expression = copy.deepcopy(expression)
new_type = _NEGATION_MAP[expression.type]
assert new_type is not None
expression.type = new_type
return expression
def _get_terminators(body):
if len(body) < 2:
return None, None, body
last = body[-1]
if len(last.contents) != 1:
return None, None, body
assignment = last.contents[0]
if not isinstance(assignment, nodes.Assignment):
return None, None, body
src = assignment.expressions.contents[0]
if not isinstance(src, nodes.Primitive) or src.type != src.T_TRUE:
return None, None, body
prev = body[-2]
if len(prev.contents) != 1:
return None, None, body
src = prev.contents[0].expressions.contents[0]
if not isinstance(src, nodes.Primitive) or src.type != src.T_FALSE:
return None, None, body
return last, prev, body[:-2]
def _assemble_expression(parts):
if not isinstance(parts, list):
return parts
node = nodes.BinaryOperator()
node.left = _assemble_expression(parts[-3])
node.type = parts[-2]
assert isinstance(node.type, int)
node.right = _assemble_expression(parts[-1])
i = len(parts) - 4
while i > 0:
operator = parts[i]
component = parts[i - 1]
upper_node = nodes.BinaryOperator()
upper_node.right = node
upper_node.left = _assemble_expression(component)
upper_node.type = operator
node = upper_node
i -= 2
return node
# Split the topmost expression into smaller subexpressions at each
# operator change to simplify the assembly phase
def _make_explicit_subexpressions(parts):
patched = []
i = 0
last_operator = parts[1]
subexpression_start = -1
while i < len(parts) - 1:
component = parts[i]
operator = parts[i + 1]
if operator < last_operator:
subexpression_start = i
last_operator = operator
elif subexpression_start > 0:
if operator > last_operator \
and (i - subexpression_start) % 2 != 0:
subexpression = parts[subexpression_start:i]
patched.append(subexpression)
subexpression_start = -1
else:
patched += [component, operator]
i += 2
if subexpression_start >= 0:
patched.append(parts[subexpression_start:])
else:
patched.append(parts[-1])
return patched
def _unwarp_if_statement(start, body, end, topmost_end):
expression, body, false = _extract_if_expression(start, body, end,
topmost_end)
node = nodes.If()
node.expression = expression
# has an else branch
if false != end and false != topmost_end:
else_start = false
else_start_index = body.index(else_start)
then_body = body[:else_start_index]
then_warp_out = then_body[-1].warp
assert _is_jump(then_warp_out)
assert then_warp_out.target in (end, topmost_end)
else_body = body[else_start_index:]
else_warp_out = else_body[-1].warp
if isinstance(else_warp_out, nodes.UnconditionalWarp):
if else_warp_out.type == nodes.UnconditionalWarp.T_JUMP:
assert else_warp_out.target in (end, topmost_end)
else:
assert else_warp_out.target == end
else:
print ("err: unwarper.py assert isinstance(else_warp_out, nodes.EndWarp), Block indices are unreliable while you are mangling them! P.S. Probably they should not be named indices... But they ARE used as indices during other phases. Sometimes.")
_set_end(then_body[-1])
then_blocks = _unwarp_ifs(then_body, then_body[-1], topmost_end)
node.then_block.contents = then_blocks
_set_end(else_body[-1])
else_blocks = _unwarp_ifs(else_body, else_body[-1], topmost_end)
node.else_block.contents = else_blocks
else:
warp_out = body[-1].warp
if not isinstance(warp_out, nodes.EndWarp):
assert isinstance(warp_out, nodes.UnconditionalWarp)
assert warp_out.target in (end, topmost_end)
_set_end(body[-1])
then_blocks = _unwarp_ifs(body, body[-1], topmost_end)
node.then_block.contents = then_blocks
start.contents.append(node)
def _extract_if_expression(start, body, end, topmost_end):
for i, block in enumerate(body):
if len(block.contents) != 0:
break
assert i < len(body)
expression = [start] + body[:i]
body = body[i:]
falses = set()
for i, block in enumerate(body[:-1]):
if not isinstance(block.warp, nodes.UnconditionalWarp):
continue
if block.warp.type != nodes.UnconditionalWarp.T_JUMP:
continue
if block.warp.target != end and block.warp.target != topmost_end:
continue
falses.add(body[i + 1])
falses.add(end)
if topmost_end is not None:
falses.add(topmost_end)
false, end_i = _search_expression_end(expression, falses)
assert end_i >= 0
body = expression[end_i:] + body
expression = expression[:end_i]
assert len(expression) > 0
true = body[0]
expression = _compile_expression(expression, None, true, false)
return expression, body, false
def _search_expression_end(expression, falses):
expression_end = -1
false = None
for i, block in enumerate(expression):
target = _get_target(block.warp)
if target not in falses:
continue
if false is None or target == false:
false = target
expression_end = i + 1
else:
break
assert false is not None
return false, expression_end
def _find_branching_end(blocks, topmost_end):
end = blocks[0]
for block in blocks:
warp = block.warp
target = _get_target(warp, allow_end=True)
if isinstance(warp, nodes.EndWarp) and target is None:
assert block == end
return block
if isinstance(warp, nodes.UnconditionalWarp) and target == end:
return end
if target.index > end.index:
end = target
return end
def _remove_processed_blocks(blocks, boundaries):
remains = []
last_end_index = -1
for start, end in boundaries:
if start == end:
up_to_index = start
else:
up_to_index = start + 1
remains += blocks[last_end_index + 1:up_to_index]
last_end_index = end
remains += blocks[last_end_index + 1:]
return remains
# ##
# ## LOOPS PROCESSING
# ##
def _unwarp_loops(blocks, repeat_until):
loops = _find_all_loops(blocks, repeat_until)
if len(loops) == 0:
return blocks
fixed = _cleanup_breaks_and_if_ends(loops, blocks)
for start, end in fixed:
start_index = blocks.index(start)
end_index = blocks.index(end)
if repeat_until:
body = blocks[start_index:end_index]
else:
body = blocks[start_index + 1:end_index]
loop = _unwarp_loop(start, end, body)
body = loop.statements.contents
block = nodes.Block()
block.first_address = body[0].first_address
block.last_address = body[-1].last_address
block.index = start.index + 1
block.contents.append(loop)
block.warp = nodes.UnconditionalWarp()
block.warp.type = nodes.UnconditionalWarp.T_FLOW
block.warp.target = end
_replace_targets(blocks, body[0], block)
_set_end(body[-1])
_unwarp_breaks(start, body, end)
blocks = blocks[:start_index + 1] + [block] + blocks[end_index:]
return blocks
def _cleanup_breaks_and_if_ends(loops, blocks):
outer_start_index = -1
outer_end = None
current_start_index = -1
current_end = None
fixed = []
for start, end in loops:
if start.index in (outer_start_index, current_start_index):
end_i = blocks.index(end)
last_in_body = blocks[end_i - 1]
warp = last_in_body.warp
assert isinstance(warp, nodes.UnconditionalWarp)
assert warp.target == start
# Break
if start.index == outer_start_index:
assert outer_end is not None
outer_end_i = blocks.index(outer_end)
warp.target = blocks[outer_end_i - 1]
assert blocks[outer_end_i - 2] != end
else:
assert current_end is not None
assert start.index == current_start_index
current_end_i = blocks.index(current_end)
last = blocks[current_end_i - 1]
if last == end:
last = _create_next_block(end)
last.warp = end.warp
_set_flow_to(end, last)
blocks.insert(current_end_i, last)
warp.target = last
else:
fixed.append((start, end))
if current_end is not None \
and current_start_index < start.index \
and current_end.index >= end.index:
outer_start_index = current_start_index
outer_end = current_end
else:
outer_start_index = -1
outer_end = None
current_start_index = start.index
current_end = end
return fixed
def _replace_targets(blocks, original, replacement):
for block in blocks:
warp = block.warp
if isinstance(warp, nodes.UnconditionalWarp):
if warp.target == original:
warp.target = replacement
elif isinstance(warp, nodes.ConditionalWarp):
if warp.true_target == original:
warp.true_target = replacement
if warp.false_target == original:
warp.false_target = replacement
elif isinstance(warp, nodes.EndWarp):
pass
else:
if warp.way_out == original:
warp.way_out = replacement
if warp.body == original:
warp.body = replacement
def _unwarp_loop(start, end, body):
if len(body) > 0:
last = body[-1]
else:
last = start
if isinstance(start.warp, nodes.IteratorWarp):
assert isinstance(last.warp, nodes.UnconditionalWarp)
assert last.warp.target == start
loop = nodes.IteratorFor()
loop.statements.contents = body
loop.identifiers = start.warp.variables
loop.expressions = start.warp.controls
loop._addr = body[0].first_address
_set_flow_to(start, body[0])
elif isinstance(start.warp, nodes.NumericLoopWarp):
assert isinstance(last.warp, nodes.UnconditionalWarp)
assert last.warp.target == start
loop = nodes.NumericFor()
loop.statements.contents = body
loop.variable = start.warp.index
loop.expressions = start.warp.controls
loop._addr = body[0].first_address
_set_flow_to(start, body[0])
# While (including "while true" and "repeat until false" which will be
# while true)
elif isinstance(last.warp, nodes.UnconditionalWarp):
assert last.warp.target == start
# while true
if _is_flow(start.warp):
loop = nodes.While()
loop.expression = nodes.Primitive()
loop.expression.type = nodes.Primitive.T_TRUE
loop.statements.contents = body
else:
# There shouldn't be many problems similar to ifs, as
# we are processing loops in the order from innermost
# to outermost
for i, block in enumerate(body):
assert len(block.contents) == 0
if _is_flow(block.warp):
break
assert i < len(body)
expression = [start] + body[:i]
body = body[i:]
# Sometimes expression may decide to jump to the
# outer loop start instead
_fix_expression(expression, start, end)
true = body[0]
false = end
expression = _compile_expression(expression, None,
true, false)
# If something jumps to the start (instead of the end)
# - that's a nested if
loop = nodes.While()
loop.expression = expression
loop.statements.contents = body
_fix_nested_ifs(body, start)
_set_flow_to(start, body[0])
# Repeat until
else:
assert isinstance(last.warp, nodes.ConditionalWarp)
assert last.warp.false_target == start
i = len(body) - 1
while i >= 0:
block = body[i]
warp = block.warp
if _is_flow(warp):
i += 1
break
if len(block.contents) != 0:
break
i -= 1
expression = body[i:]
body = body[:i + 1]
assert len(expression) > 0
first = expression[0]
if _is_jump(first.warp):
# Don't append to the body - it already has it
expression.pop(0)
body[-1].contents.append(nodes.Break())
false = body[0]
# Don't use end as it could be broken by a previous
# repeat until pass
true = expression[-1].warp.true_target
loop = nodes.RepeatUntil()
loop.expression = _compile_expression(expression, None,
true, false)
start_copy = copy.copy(start)
start.contents = []
if len(body) > 1:
_set_flow_to(start_copy, body[1])
else:
_set_end(start_copy)
_set_flow_to(start, start_copy)
body[0] = start_copy
loop.statements.contents = body
return loop
def _create_next_block(original):
block = nodes.Block()
block.first_address = original.last_address + 1
block.last_address = block.first_address
block.index = original.index + 1
block.warpins_count = original.warpins_count
return block
def _set_flow_to(block, target):
block.warp = nodes.UnconditionalWarp()
block.warp.type = nodes.UnconditionalWarp.T_FLOW
block.warp.target = target
def _set_end(block):
target = None
if block.warp is not None:
target = _get_target(block.warp, allow_end=True)
block.warp = nodes.EndWarp()
setattr(block.warp, "_target", target)
def _is_flow(warp):
return isinstance(warp, nodes.UnconditionalWarp) \
and warp.type == nodes.UnconditionalWarp.T_FLOW
def _is_jump(warp):
return isinstance(warp, nodes.UnconditionalWarp) \
and warp.type == nodes.UnconditionalWarp.T_JUMP
def _fix_nested_ifs(blocks, start):
# We can't point both targets of a conditional warp to the
# same block. We will have to create a new block
last = _create_next_block(blocks[-1])
if isinstance(blocks[-1].warp, nodes.ConditionalWarp):
blocks[-1].warp.false_target = last
else:
_set_flow_to(blocks[-1], last)
blocks.append(last)
_set_end(last)
for block in blocks[:-1]:
target = _get_target(block.warp)
if target == start:
_set_target(block.warp, last)
def _fix_expression(blocks, start, end):
for block in blocks:
if len(block.contents) != 0:
break
target = _get_target(block.warp)
if target.index < start.index:
_set_target(block.warp, end)
def _gather_possible_ends(block):
warp = block.warp
ends = set((block,))
while _is_jump(warp):
block = warp.target
warp = block.warp
ends.add(block)
return ends
BREAK_INFINITE = 0
BREAK_ONE_USE = 1
def _unwarp_breaks(start, blocks, next_block):
blocks_set = set([start] + blocks)
ends = _gather_possible_ends(next_block)
breaks = set()
patched = []
for i, block in enumerate(blocks):
warp = block.warp
if not isinstance(warp, nodes.UnconditionalWarp):
patched.append(block)
continue
target = _get_target(warp)
if target in blocks_set:
patched.append(block)
continue
assert target in ends, "GOTO statements are not supported"
if block.warpins_count != 0:
new_block = _create_next_block(block)
new_block.warpins_count = block.warpins_count
_set_flow_to(block, new_block)
patched.append(block)
patched.append(new_block)
block = new_block
else:
patched.append(block)
block.contents.append(nodes.Break())
if i + 1 == len(blocks):
_set_end(block)
else:
_set_flow_to(block, blocks[i + 1])
breaks.add(block)
blocks[:] = patched
if len(breaks) == 0:
return
breaks_stack = []
warpsout = []
pending_break = None
for i, block in enumerate(reversed(blocks)):
if block in breaks:
pending_break = None
if block.warpins_count == 0:
breaks_stack.append((BREAK_ONE_USE, block))
else:
breaks_stack.append((BREAK_INFINITE, block))
continue
warp = block.warp
if not isinstance(warp, nodes.ConditionalWarp):
if _is_flow(warp):
pending_break = None
continue
target = _get_target(warp)
if target in blocks_set:
continue
assert target in ends, "GOTO statements are not supported"
if pending_break is None:
assert len(breaks_stack) > 0
top_break = breaks_stack[-1]
_set_target(warp, top_break[1])
if top_break[0] == BREAK_ONE_USE:
pending_break = breaks_stack.pop()
warpsout = []
else:
warpsout.append(block)
else:
_set_target(warp, pending_break[1])
warpsout.append(block)
if len(block.contents) > 0:
pending_break = None
while len(breaks_stack) > 0 and breaks_stack[-1][0] == BREAK_INFINITE:
breaks_stack.pop()
# And pray for the best...
while len(warpsout) > 0 and len(breaks_stack) > 0:
_set_target(warpsout.pop().warp, breaks_stack.pop()[1])
#
# We don't need any complex checks here.
#
# Just search for any negative jump - that's a loop and what it's jumping to is
# a loop start.
#
def _find_all_loops(blocks, repeat_until):
# Duplicates are NOT possible
loops = []
i = 0
while i < len(blocks):
block = blocks[i]
warp = block.warp
if isinstance(warp, nodes.UnconditionalWarp):
if warp.type == nodes.UnconditionalWarp.T_FLOW:
i += 1
continue
if warp.target.index <= block.index:
assert not repeat_until
assert i < len(blocks) - 1
loops.append((warp.target, blocks[i + 1]))
elif repeat_until and isinstance(warp, nodes.ConditionalWarp):
if warp.false_target.index > block.index:
i += 1
continue
start = warp.false_target
first = block
end = block
last_i = i
# Find the end of the expression
while i < len(blocks):
block = blocks[i]
warp = block.warp
if block != first and len(block.contents) != 0:
break
if isinstance(warp, nodes.EndWarp):
break
# Go up to a first negative jump of an
# another loop
target = _get_target(warp)
if target.index < block.index:
if target == start:
start = target
end = block
last_i = i
else:
break
i += 1
# And then rollback to the last known negative jump
# of our loop
i = last_i
# There always should be at least one return block
end_index = blocks.index(end)
end = blocks[end_index + 1]
loops.append((start, end))
i += 1
# Reverse the order so inner "while" loops are processed before
# outer loops
return list(reversed(sorted(loops, key=lambda x: x[0].index)))
def _get_previous_block(block, blocks):
block_index = blocks.index(block)
assert block_index > 0
return blocks[block_index - 1]
================================================
FILE: ljd/ast/validator.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
import ljd.ast.nodes as nodes
import ljd.ast.traverse as traverse
class TypeRestriction():
def __init__(self, default, specific):
if isinstance(default, dict) and specific == {}:
specific = default
default = None
self.default = default
self.specific = specific
def check(self, node):
try:
typespec = self.specific[node]
except KeyError:
typespec = self.default
#assert typespec, "Unknown node: {0}".format(node)
#assert isinstance(node, typespec),"Invalid node type: {0} should be: {1}".format(type(node), typespec)
STATEMENT_TYPES = (
nodes.Assignment,
nodes.If,
nodes.IteratorFor,
nodes.NumericFor,
nodes.RepeatUntil,
nodes.Return,
nodes.Break,
nodes.FunctionCall,
nodes.While
)
EXPRESSION_TYPES = (
nodes.FunctionCall,
nodes.Primitive,
nodes.Constant,
nodes.Identifier,
nodes.FunctionDefinition,
nodes.TableConstructor,
nodes.Vararg,
nodes.BinaryOperator,
nodes.UnaryOperator,
nodes.MULTRES,
nodes.TableElement,
)
VARIABLE_TYPES = (
nodes.Identifier,
nodes.TableElement,
nodes.MULTRES # It's not valid here, but it is a hack anyway...
)
WARP_TYPES = (
nodes.UnconditionalWarp,
nodes.ConditionalWarp,
nodes.IteratorWarp,
nodes.NumericLoopWarp,
nodes.EndWarp
)
class Visitor(traverse.Visitor):
def __init__(self, warped=True):
# Restrictions for the upmost level
self.restrictions = [None]
self.warped = warped
def _set_restrictions(self, default, specific={}):
self.restrictions[-1] = TypeRestriction(default, specific)
# ##
def visit_function_definition(self, node):
self._set_restrictions(nodes.Block, {
node.arguments: nodes.IdentifiersList,
node.statements: nodes.StatementsList
})
# ##
def visit_table_constructor(self, node):
self._set_restrictions(nodes.RecordsList)
def visit_array_record(self, node):
self._set_restrictions(EXPRESSION_TYPES)
def visit_table_record(self, node):
self._set_restrictions(EXPRESSION_TYPES)
# ##
def visit_assignment(self, node):
self._set_restrictions({
node.destinations: nodes.VariablesList,
node.expressions: nodes.ExpressionsList
})
if not isinstance(node.destinations.contents[0], nodes.Identifier):
return
if node.destinations.contents[0].type != nodes.Identifier.T_LOCAL:
return
# Don't test type flag here
# ##
def visit_binary_operator(self, node):
self._set_restrictions(EXPRESSION_TYPES)
assert node.type == nodes.BinaryOperator.T_LOGICAL_OR \
or node.type == nodes.BinaryOperator.T_LOGICAL_AND \
\
or node.type == nodes.BinaryOperator.T_LESS_THEN \
or node.type == nodes.BinaryOperator.T_GREATER_THEN \
or node.type == nodes.BinaryOperator.T_LESS_OR_EQUAL \
or node.type == nodes.BinaryOperator.T_GREATER_OR_EQUAL \
\
or node.type == nodes.BinaryOperator.T_NOT_EQUAL \
or node.type == nodes.BinaryOperator.T_EQUAL \
\
or node.type == nodes.BinaryOperator.T_CONCAT \
\
or node.type == nodes.BinaryOperator.T_ADD \
or node.type == nodes.BinaryOperator.T_SUBTRACT \
\
or node.type == nodes.BinaryOperator.T_MULTIPLY \
or node.type == nodes.BinaryOperator.T_DIVISION \
or node.type == nodes.BinaryOperator.T_MOD \
\
or node.type == nodes.BinaryOperator.T_POW
def visit_unary_operator(self, node):
self._set_restrictions(EXPRESSION_TYPES)
assert node.type == nodes.UnaryOperator.T_NOT \
or node.type == nodes.UnaryOperator.T_LENGTH_OPERATOR \
or node.type == nodes.UnaryOperator.T_MINUS
# ##
def visit_statements_list(self, node):
if self.warped:
types = nodes.Block
else:
types = STATEMENT_TYPES
self._set_restrictions(types)
def visit_identifiers_list(self, node):
# HACK
self._set_restrictions((nodes.Identifier, nodes.Vararg))
def visit_records_list(self, node):
self._set_restrictions((nodes.TableRecord,
nodes.ArrayRecord,
nodes.FunctionCall,
nodes.Vararg))
if len(node.contents) == 0:
return
is_array = isinstance(node.contents[0], nodes.ArrayRecord)
for i, x in enumerate(node.contents):
if is_array:
assert isinstance(x, nodes.ArrayRecord)
elif not isinstance(x, nodes.TableRecord):
assert i == (len(node.contents) - 1)
def visit_variables_list(self, node):
self._set_restrictions(VARIABLE_TYPES)
def visit_expressions_list(self, node):
self._set_restrictions(EXPRESSION_TYPES)
# ##
def visit_identifier(self, node):
assert node.type == nodes.Identifier.T_SLOT \
or node.type == nodes.Identifier.T_BUILTIN \
or node.type == nodes.Identifier.T_UPVALUE \
or (node.name is not None \
and node.type == nodes.Identifier.T_LOCAL)
assert node.type == nodes.Identifier.T_BUILTIN or node.slot >= 0
def visit_table_element(self, node):
self._set_restrictions(EXPRESSION_TYPES)
def visit_function_call(self, node):
self._set_restrictions({
node.function: VARIABLE_TYPES,
node.arguments: nodes.ExpressionsList
})
# ##
def visit_if(self, node):
self._set_restrictions(nodes.ElseIf, {
node.expression: EXPRESSION_TYPES,
node.then_block: nodes.StatementsList,
node.else_block: nodes.StatementsList
})
def visit_elseif(self, node):
self._set_restrictions({
node.expression: EXPRESSION_TYPES,
node.then_block: nodes.StatementsList
})
# ##
def visit_block(self, node):
self._set_restrictions(STATEMENT_TYPES, {
node.warp: WARP_TYPES
})
assert node.index >= 0
assert node.first_address >= 0 \
and node.first_address <= node.last_address
# if false produce a statements without warps in
# assert node.warpins_count > 0
def visit_unconditional_warp(self, node):
assert node.target is not None
assert node.type == nodes.UnconditionalWarp.T_JUMP \
or node.type == nodes.UnconditionalWarp.T_FLOW
def visit_conditional_warp(self, node):
self._set_restrictions({
node.condition: EXPRESSION_TYPES
})
assert node.true_target is not None
assert node.false_target is not None
# It might happen in case of if blabla or true stuff
# or in case of a = a and foo(a) or a type expression
# assert node.true_target != node.false_target
# assert node.true_target.index != node.false_target.index
def visit_iterator_warp(self, node):
assert node.body is not None
assert node.way_out is not None
assert node.way_out.index > node.body.index
self._set_restrictions(nodes.Block, {
node.variables: nodes.VariablesList,
node.controls: nodes.ExpressionsList
})
def visit_numeric_loop_warp(self, node):
assert node.body is not None
assert node.way_out is not None
self._set_restrictions(nodes.Block, {
node.index: nodes.Identifier,
node.controls: nodes.ExpressionsList
})
# ##
def visit_return(self, node):
self._set_restrictions(nodes.ExpressionsList)
# ##
def visit_while(self, node):
self._set_restrictions({
node.expression: EXPRESSION_TYPES,
node.statements: nodes.StatementsList
})
def visit_repeat_until(self, node):
self._set_restrictions({
node.expression: EXPRESSION_TYPES,
node.statements: nodes.StatementsList
})
def visit_numeric_for(self, node):
self._set_restrictions({
node.expressions: nodes.ExpressionsList,
node.statements: nodes.StatementsList,
node.variable: VARIABLE_TYPES
})
def visit_iterator_for(self, node):
self._set_restrictions({
node.expressions: nodes.ExpressionsList,
node.identifiers: nodes.VariablesList,
node.statements: nodes.StatementsList
})
# ##
def visit_constant(self, node):
assert node.type == nodes.Constant.T_CDATA \
or node.type == nodes.Constant.T_FLOAT \
or node.type == nodes.Constant.T_INTEGER \
or node.type == nodes.Constant.T_STRING
def visit_primitive(self, node):
assert node.type == nodes.Primitive.T_NIL \
or node.type == nodes.Primitive.T_TRUE \
or node.type == nodes.Primitive.T_FALSE
# ##
def _visit(self, node):
restrictions = self.restrictions[-1]
if restrictions is not None:
restrictions.check(node)
# Add layer for the child node
self.restrictions.append(None)
traverse.Visitor._visit(self, node)
# And pop it back
self.restrictions.pop()
def validate(ast, warped=True):
visitor = Visitor(warped)
traverse.traverse(visitor, ast)
================================================
FILE: ljd/bytecode/__init__.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
================================================
FILE: ljd/bytecode/constants.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
T_NIL = 0
T_FALSE = 1
T_TRUE = 2
class Table():
def __init__(self):
self.array = []
# Use a list so we can keep the original items order in the
# table
self.dictionary = []
class Constants():
def __init__(self):
self.upvalue_references = []
self.numeric_constants = []
self.complex_constants = []
================================================
FILE: ljd/bytecode/debuginfo.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
class VariableInfo():
T_VISIBILE = 0
T_INTERNAL = 1
def __init__(self):
self.start_addr = 0
self.end_addr = 0
self.type = -1
self.name = ""
class DebugInformation():
def __init__(self):
self.addr_to_line_map = []
self.upvalue_variable_names = []
self.variable_info = []
def lookup_line_number(self, addr):
try:
return self.addr_to_line_map[addr]
except IndexError:
return 0
def lookup_local_name(self, addr, slot):
for info in self.variable_info:
if info.start_addr > addr:
break
if info.end_addr <= addr:
continue
elif slot == 0:
return info
else:
slot -= 1
return None
def lookup_upvalue_name(self, slot):
try:
return self.upvalue_variable_names[slot]
except IndexError:
return None
================================================
FILE: ljd/bytecode/helpers.py
================================================
def get_jump_destination(addr, instruction):
return addr + instruction.CD + 1
def set_jump_destination(addr, instruction, value):
instruction.CD = value - addr - 1
================================================
FILE: ljd/bytecode/instructions.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
#
# Almost direct wiki-to-code from
# http://wiki.luajit.org/Bytecode-2.0
#
# What the hell is an upvalue?
# It is value from the upper prototype, i.e. a variable stored in a closure.
# What the hell is the MULTRES thing?
# The magical VM's variable that counts the CALL* or VARG returns.
# I.e. it's used to speed-up things like foo(var1, var2, bar(var3, var4)) or
#
# function foo(...)
# bla-bla
# bar(...)
#
# Argument types
T_VAR = 0 # variable slot number
T_DST = 1 # variable slot number, used as a destination
T_BS = 2 # base slot number, read-write
T_RBS = 3 # base slot number, read-only
T_UV = 4 # upvalue number (slot number, but specific to upvalues)
T_LIT = 5 # literal
T_SLIT = 6 # signed literal
T_PRI = 7 # primitive type (0 = nil, 1 = false, 2 = true)
T_NUM = 8 # numeric constant, index into constant table
T_STR = 9 # string constant, negated index into constant table
T_TAB = 10 # template table, negated index into constant table
T_FUN = 11 # function prototype, negated index into constant table
T_CDT = 12 # cdata constant, negated index into constant table
T_JMP = 13 # branch target, relative to next instruction, biased with 0x8000
class _Instruction():
def __init__(self, definition):
for key, value in definition.__dict__.items():
setattr(self, key, value)
if self.A_type is not None:
self.A = 0
if self.B_type is not None:
self.B = 0
if self.CD_type is not None:
self.CD = 0
class _IDef():
_LAST_OPCODE = 0
def __init__(self, name, A_type, B_type, CD_type, description):
self.name = name
self.opcode = _IDef._LAST_OPCODE
self.A_type = A_type
self.B_type = B_type
self.CD_type = CD_type
self.description = description
self.args_count = (self.A_type is not None) \
+ (self.B_type is not None) \
+ (self.CD_type is not None)
_IDef._LAST_OPCODE += 1
def __call__(self):
return _Instruction(self)
# Names and order are in sync with luaJIT bytecode for ease of changing
# class = name A B C description
# Comparison ops
ISLT = _IDef("ISLT", T_VAR, None, T_VAR, "if {A} < {D}")
ISGE = _IDef("ISGE", T_VAR, None, T_VAR, "if {A} >= {D}")
ISLE = _IDef("ISLE", T_VAR, None, T_VAR, "if {A} <= {D}")
ISGT = _IDef("ISGT", T_VAR, None, T_VAR, "if {A} > {D}")
ISEQV = _IDef("ISEQV", T_VAR, None, T_VAR, "if {A} == {D}")
ISNEV = _IDef("ISNEV", T_VAR, None, T_VAR, "if {A} ~= {D}")
ISEQS = _IDef("ISEQS", T_VAR, None, T_STR, "if {A} == {D}")
ISNES = _IDef("ISNES", T_VAR, None, T_STR, "if {A} ~= {D}")
ISEQN = _IDef("ISEQN", T_VAR, None, T_NUM, "if {A} == {D}")
ISNEN = _IDef("ISNEN", T_VAR, None, T_NUM, "if {A} ~= {D}")
ISEQP = _IDef("ISEQP", T_VAR, None, T_PRI, "if {A} == {D}")
ISNEP = _IDef("ISNEP", T_VAR, None, T_PRI, "if {A} ~= {D}")
# Unary test and copy ops
ISTC = _IDef("ISTC", T_DST, None, T_VAR, "{A} = {D}; if {D}")
ISFC = _IDef("ISFC", T_DST, None, T_VAR, "{A} = {D}; if not {D}")
IST = _IDef("IST", None, None, T_VAR, "if {D}")
ISF = _IDef("ISF", None, None, T_VAR, "if not {D}")
ISTYPE = _IDef("ISTYPE", T_VAR, None, T_LIT, "ISTYPE unknow")
ISNUM = _IDef("ISNUM", T_VAR, None, T_LIT, "ISNUM unknow")
# Unary ops
MOV = _IDef("MOV", T_DST, None, T_VAR, "{A} = {D}")
NOT = _IDef("NOT", T_DST, None, T_VAR, "{A} = not {D}")
UNM = _IDef("UNM", T_DST, None, T_VAR, "{A} = -{D}")
LEN = _IDef("LEN", T_DST, None, T_VAR, "{A} = #{D}")
# Binary ops
ADDVN = _IDef("ADDVN", T_DST, T_VAR, T_NUM, "{A} = {B} + {C}")
SUBVN = _IDef("SUBVN", T_DST, T_VAR, T_NUM, "{A} = {B} - {C}")
MULVN = _IDef("MULVN", T_DST, T_VAR, T_NUM, "{A} = {B} * {C}")
DIVVN = _IDef("DIVVN", T_DST, T_VAR, T_NUM, "{A} = {B} / {C}")
MODVN = _IDef("MODVN", T_DST, T_VAR, T_NUM, "{A} = {B} % {C}")
ADDNV = _IDef("ADDNV", T_DST, T_VAR, T_NUM, "{A} = {C} + {B}")
SUBNV = _IDef("SUBNV", T_DST, T_VAR, T_NUM, "{A} = {C} - {B}")
MULNV = _IDef("MULNV", T_DST, T_VAR, T_NUM, "{A} = {C} * {B}")
DIVNV = _IDef("DIVNV", T_DST, T_VAR, T_NUM, "{A} = {C} / {B}")
MODNV = _IDef("MODNV", T_DST, T_VAR, T_NUM, "{A} = {C} % {B}")
ADDVV = _IDef("ADDVV", T_DST, T_VAR, T_VAR, "{A} = {B} + {C}")
SUBVV = _IDef("SUBVV", T_DST, T_VAR, T_VAR, "{A} = {B} - {C}")
MULVV = _IDef("MULVV", T_DST, T_VAR, T_VAR, "{A} = {B} * {C}")
DIVVV = _IDef("DIVVV", T_DST, T_VAR, T_VAR, "{A} = {B} / {C}")
MODVV = _IDef("MODVV", T_DST, T_VAR, T_VAR, "{A} = {B} % {C}")
POW = _IDef("POW", T_DST, T_VAR, T_VAR, "{A} = {B} ^ {C} (pow)")
CAT = _IDef("CAT", T_DST, T_RBS, T_RBS,
"{A} = {concat_from_B_to_C}")
# Constant ops.
KSTR = _IDef("KSTR", T_DST, None, T_STR, "{A} = {D}")
KCDATA = _IDef("KCDATA", T_DST, None, T_CDT, "{A} = {D}")
KSHORT = _IDef("KSHORT", T_DST, None, T_SLIT, "{A} = {D}")
KNUM = _IDef("KNUM", T_DST, None, T_NUM, "{A} = {D}")
KPRI = _IDef("KPRI", T_DST, None, T_PRI, "{A} = {D}")
KNIL = _IDef("KNIL", T_BS, None, T_BS, "{from_A_to_D} = nil")
# Upvalue and function ops.
UGET = _IDef("UGET", T_DST, None, T_UV, "{A} = {D}")
USETV = _IDef("USETV", T_UV, None, T_VAR, "{A} = {D}")
USETS = _IDef("USETS", T_UV, None, T_STR, "{A} = {D}")
USETN = _IDef("USETN", T_UV, None, T_NUM, "{A} = {D}")
USETP = _IDef("USETP", T_UV, None, T_PRI, "{A} = {D}")
UCLO = _IDef("UCLO", T_RBS, None, T_JMP,
"nil uvs >= {A}; goto {D}")
FNEW = _IDef("FNEW", T_DST, None, T_FUN, "{A} = function {D}")
# Table ops.
TNEW = _IDef("TNEW", T_DST, None, T_LIT, "{A} = new table("
" array: {D_array},"
" dict: {D_dict})")
TDUP = _IDef("TDUP", T_DST, None, T_TAB, "{A} = copy {D}")
GGET = _IDef("GGET", T_DST, None, T_STR, "{A} = _env[{D}]")
GSET = _IDef("GSET", T_VAR, None, T_STR, "_env[{D}] = {A}")
TGETV = _IDef("TGETV", T_DST, T_VAR, T_VAR, "{A} = {B}[{C}]")
TGETS = _IDef("TGETS", T_DST, T_VAR, T_STR, "{A} = {B}.{C}")
TGETB = _IDef("TGETB", T_DST, T_VAR, T_LIT, "{A} = {B}[{C}]")
TGETR = _IDef("TGETR", T_DST, T_VAR, T_VAR, "unkown TGETR")
TSETV = _IDef("TSETV", T_VAR, T_VAR, T_VAR, "{B}[{C}] = {A}")
TSETS = _IDef("TSETS", T_VAR, T_VAR, T_STR, "{B}.{C} = {A}")
TSETB = _IDef("TSETB", T_VAR, T_VAR, T_LIT, "{B}[{C}] = {A}")
TSETM = _IDef("TSETM", T_BS, None, T_NUM,
"for i = 0, MULTRES, 1 do"
" {A_minus_one}[{D_low} + i] = slot({A} + i)")
TSETR = _IDef("TSETR", T_VAR, T_VAR, T_VAR, "unkow TSETR")
# Calls and vararg handling. T = tail call.
CALLM = _IDef("CALLM", T_BS, T_LIT, T_LIT,
"{from_A_x_B_minus_two} = {A}({from_A_plus_one_x_C}, ...MULTRES)")
CALL = _IDef("CALL", T_BS, T_LIT, T_LIT,
"{from_A_x_B_minus_two} = {A}({from_A_plus_one_x_C_minus_one})")
CALLMT = _IDef("CALLMT", T_BS, None, T_LIT,
"return {A}({from_A_plus_one_x_D}, ...MULTRES)")
CALLT = _IDef("CALLT", T_BS, None, T_LIT,
"return {A}({from_A_plus_one_x_D_minus_one})")
ITERC = _IDef("ITERC", T_BS, T_LIT, T_LIT,
"{A}, {A_plus_one}, {A_plus_two} ="
" {A_minus_three}, {A_minus_two}, {A_minus_one};"
" {from_A_x_B_minus_two} ="
" {A_minus_three}({A_minus_two}, {A_minus_one})")
ITERN = _IDef("ITERN", T_BS, T_LIT, T_LIT,
"{A}, {A_plus_one}, {A_plus_two} ="
" {A_minus_three}, {A_minus_two}, {A_minus_one};"
" {from_A_x_B_minus_two} ="
" {A_minus_three}({A_minus_two}, {A_minus_one})")
VARG = _IDef("VARG", T_BS, T_LIT, T_LIT,
"{from_A_x_B_minus_two} = ...")
ISNEXT = _IDef("ISNEXT", T_BS, None, T_JMP,
"Verify ITERN at {D}; goto {D}")
# Returns.
RETM = _IDef("RETM", T_BS, None, T_LIT,
"return {from_A_x_D_minus_one}, ...MULTRES")
RET = _IDef("RET", T_RBS, None, T_LIT,
"return {from_A_x_D_minus_two}")
RET0 = _IDef("RET0", T_RBS, None, T_LIT, "return")
RET1 = _IDef("RET1", T_RBS, None, T_LIT, "return {A}")
# Loops and branches. I/J = interp/JIT, I/C/L = init/call/loop.
FORI = _IDef("FORI", T_BS, None, T_JMP,
"for {A_plus_three} = {A},{A_plus_one},{A_plus_two}"
" else goto {D}")
JFORI = _IDef("JFORI", T_BS, None, T_JMP,
"for {A_plus_three} = {A},{A_plus_one},{A_plus_two}"
" else goto {D}")
FORL = _IDef("FORL", T_BS, None, T_JMP,
"{A} = {A} + {A_plus_two};"
" if cmp({A}, sign {A_plus_two}, {A_plus_one}) goto {D}")
IFORL = _IDef("IFORL", T_BS, None, T_JMP,
"{A} = {A} + {A_plus_two};"
" if cmp({A}, sign {A_plus_two}, {A_plus_one}) goto {D}")
JFORL = _IDef("JFORL", T_BS, None, T_JMP,
"{A} = {A} + {A_plus_two};"
" if cmp({A}, sign {A_plus_two}, {A_plus_one}) goto {D}")
ITERL = _IDef("ITERL", T_BS, None, T_JMP,
"{A_minus_one} = {A}; if {A} != nil goto {D}")
IITERL = _IDef("IITERL", T_BS, None, T_JMP,
"{A_minus_one} = {A}; if {A} != nil goto {D}")
JITERL = _IDef("JITERL", T_BS, None, T_LIT,
"{A_minus_one} = {A}; if {A} != nil goto {D}")
LOOP = _IDef("LOOP", T_RBS, None, T_JMP, "Noop")
ILOOP = _IDef("ILOOP", T_RBS, None, T_JMP, "Noop")
JLOOP = _IDef("JLOOP", T_RBS, None, T_LIT, "Noop")
JMP = _IDef("JMP", T_RBS, None, T_JMP, " goto {D}")
# Function headers. I/J = interp/JIT, F/V/C = fixarg/vararg/C func.
# Shouldn't be ever seen - they are not stored in raw dump?
FUNCF = _IDef("FUNCF", T_RBS, None, None,
"Fixed-arg function with frame size {A}")
IFUNCF = _IDef("IFUNCF", T_RBS, None, None,
"Interpreted fixed-arg function with frame size {A}")
JFUNCF = _IDef("JFUNCF", T_RBS, None, T_LIT,
"JIT compiled fixed-arg function with frame size {A}")
FUNCV = _IDef("FUNCV", T_RBS, None, None,
"Var-arg function with frame size {A}")
IFUNCV = _IDef("IFUNCV", T_RBS, None, None,
"Interpreted var-arg function with frame size {A}")
JFUNCV = _IDef("JFUNCV", T_RBS, None, T_LIT,
"JIT compiled var-arg function with frame size {A}")
FUNCC = _IDef("FUNCC", T_RBS, None, None,
"C function with frame size {A}")
FUNCCW = _IDef("FUNCCW", T_RBS, None, None,
"Wrapped C function with frame size {A}")
UNKNW = _IDef("UNKNW", T_LIT, T_LIT, T_LIT, "Unknown instruction")
================================================
FILE: ljd/bytecode/prototype.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
import ljd.bytecode.constants as constants
import ljd.bytecode.debuginfo as debug
class Flags():
def __init(self):
self.has_sub_prototypes = False
self.is_variadic = False
self.has_ffi = False
self.has_jit = True
self.has_iloop = False
class Prototype():
def __init__(self):
self.flags = Flags()
self.arguments_count = 0
self.framesize = 0
self.first_line_number = 0
self.lines_count = 0
self.instructions = []
self.constants = constants.Constants()
self.debuginfo = debug.DebugInformation()
================================================
FILE: ljd/lua/__init__.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
================================================
FILE: ljd/lua/writer.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
import re
import ljd.ast.nodes as nodes
import ljd.ast.traverse as traverse
CMD_START_STATEMENT = 0
CMD_END_STATEMENT = 1
CMD_END_LINE = 3
CMD_START_BLOCK = 4
CMD_END_BLOCK = 5
CMD_WRITE = 6
OPERATOR_TYPES = (nodes.BinaryOperator, nodes.UnaryOperator)
STATEMENT_NONE = -1
STATEMENT_ASSIGNMENT = 0
STATEMENT_FUNCTION_CALL = 1
STATEMENT_RETURN = 2
STATEMENT_BREAK = 3
STATEMENT_IF = 4
STATEMENT_ITERATOR_FOR = 5
STATEMENT_NUMERIC_FOR = 6
STATEMENT_REPEAT_UNTIL = 7
STATEMENT_WHILE = 8
STATEMENT_FUNCTION = 9
VALID_IDENTIFIER = re.compile(r'^\w[\w\d]*$')
class _State():
def __init__(self):
self.current_statement = STATEMENT_NONE
self.function_name = None
self.function_local = False
class Visitor(traverse.Visitor):
def __init__(self):
traverse.Visitor.__init__(self)
self.print_queue = []
self._visited_nodes = [set()]
self._states = [_State()]
# ##
def _start_statement(self, statement):
assert self._state().current_statement == STATEMENT_NONE
self._state().current_statement = statement
self.print_queue.append((CMD_START_STATEMENT, statement))
def _end_statement(self, statement):
assert statement == self._state().current_statement
self._state().current_statement = STATEMENT_NONE
self.print_queue.append((CMD_END_STATEMENT, statement))
def _end_line(self):
self.print_queue.append((CMD_END_LINE,))
def _start_block(self):
self.print_queue.append((CMD_START_BLOCK,))
def _end_block(self):
self.print_queue.append((CMD_END_BLOCK,))
def _write(self, fmt, *args, **kargs):
self.print_queue.append((CMD_WRITE, fmt, args, kargs))
def _state(self):
return self._states[-1]
def _push_state(self):
return self._states.append(_State())
def _pop_state(self):
return self._states.pop()
# ##
def visit_function_definition(self, node):
is_statement = self._state().function_name is not None
if is_statement:
self._start_statement(STATEMENT_FUNCTION)
if self._state().function_local:
self._write("local ")
self._write("function ")
self._visit(self._state().function_name)
self._write("(")
self._state().function_name = None
else:
self._write("function (")
self._visit(node.arguments)
self._write(")")
self._end_line()
self._visit(node.statements)
self._write("end")
if is_statement:
self._end_statement(STATEMENT_FUNCTION)
# ##
def visit_table_constructor(self, node):
self._write("{")
if len(node.records.contents) + len(node.array.contents) > 0:
self._end_line()
self._start_block()
array = node.array.contents
records = node.records.contents
all_records = nodes.RecordsList()
all_records.contents = array + records
if len(array) > 0:
first = array[0].value
all_records.contents.pop(0)
if not isinstance(first, nodes.Primitive) \
or first.type != first.T_NIL:
record = nodes.TableRecord()
record.key = nodes.Constant()
record.key.type = nodes.Constant.T_INTEGER
record.key.value = 0
record.value = first
all_records.contents.insert(0, record)
self._visit(all_records)
self._skip(node.array)
self._skip(node.records)
self._end_block()
else:
self._skip(node.array)
self._skip(node.records)
self._write("}")
def visit_table_record(self, node):
if self._is_valid_name(node.key):
self._write(node.key.value)
self._skip(node.key)
self._write(" = ")
else:
self._write("[")
self._visit(node.key)
self._write("] = ")
self._visit(node.value)
# visit_array_record is a passthough
# ##
def visit_assignment(self, node):
is_local = node.type == node.T_LOCAL_DEFINITION
dsts = node.destinations.contents
srcs = node.expressions.contents
if len(dsts) == 1 and len(srcs) == 1:
dst = dsts[0]
src = srcs[0]
src_is_function = isinstance(src, nodes.FunctionDefinition)
dst_is_simple = self._is_variable(dst)
if src_is_function and dst_is_simple:
self._state().function_name = dst
self._state().function_local = is_local
self._visit(src)
self._skip(node.destinations)
self._skip(node.expressions)
return
if is_local:
self._write("local ")
self._start_statement(STATEMENT_ASSIGNMENT)
self._visit(node.destinations)
self._write(" = ")
self._visit(node.expressions)
self._end_statement(STATEMENT_ASSIGNMENT)
def _is_variable(self, node):
if isinstance(node, nodes.Identifier):
return True
return self._is_global(node)
def _is_global(self, node):
if isinstance(node, nodes.TableElement):
return self._is_builtin(node.table)
return False
def _is_builtin(self, node):
if not isinstance(node, nodes.Identifier):
return False
return node.type == nodes.Identifier.T_BUILTIN
def _is_valid_name(self, key):
if not isinstance(key, nodes.Constant) or key.type != key.T_STRING:
return False
return VALID_IDENTIFIER.match(key.value)
# ##
def visit_binary_operator(self, node):
is_left_op = isinstance(node.left, OPERATOR_TYPES)
is_right_op = isinstance(node.right, OPERATOR_TYPES)
# If the subexpressions are less in order then this expression,
# they should go with parentheses
left_parentheses = False
right_parentheses = False
binop = nodes.BinaryOperator
if is_left_op:
if node.type <= binop.T_LOGICAL_AND:
left_parentheses = (
node.left.type <= node.type
or (node.left.type <= binop.T_LOGICAL_AND
and node.type <= binop.T_LOGICAL_AND)
) and node.left.type != node.type
else:
left_parentheses = node.left.type < node.type
if is_right_op:
if node.type <= nodes.BinaryOperator.T_LOGICAL_AND:
right_parentheses = (
node.right.type <= node.type
or (node.right.type <= binop.T_LOGICAL_AND
and node.type <= binop.T_LOGICAL_AND)
) and node.right.type != node.type
else:
right_parentheses = node.right.type < node.type
if left_parentheses:
self._write("(")
self._visit(node.left)
if left_parentheses:
self._write(")")
if node.type == nodes.BinaryOperator.T_LOGICAL_OR:
self._write(" or ")
elif node.type == nodes.BinaryOperator.T_LOGICAL_AND:
self._write(" and ")
elif node.type == nodes.BinaryOperator.T_LESS_THEN:
self._write(" < ")
elif node.type == nodes.BinaryOperator.T_GREATER_THEN:
self._write(" > ")
elif node.type == nodes.BinaryOperator.T_LESS_OR_EQUAL:
self._write(" <= ")
elif node.type == nodes.BinaryOperator.T_GREATER_OR_EQUAL:
self._write(" >= ")
elif node.type == nodes.BinaryOperator.T_NOT_EQUAL:
self._write(" ~= ")
elif node.type == nodes.BinaryOperator.T_EQUAL:
self._write(" == ")
elif node.type == nodes.BinaryOperator.T_CONCAT:
self._write(" .. ")
elif node.type == nodes.BinaryOperator.T_ADD:
self._write(" + ")
elif node.type == nodes.BinaryOperator.T_SUBTRACT:
self._write(" - ")
elif node.type == nodes.BinaryOperator.T_DIVISION:
self._write("/")
elif node.type == nodes.BinaryOperator.T_MULTIPLY:
self._write("*")
elif node.type == nodes.BinaryOperator.T_MOD:
self._write("%")
else:
assert node.type == nodes.BinaryOperator.T_POW
self._write("^")
if right_parentheses:
self._write("(")
self._visit(node.right)
if right_parentheses:
self._write(")")
def visit_unary_operator(self, node):
if node.type == nodes.UnaryOperator.T_LENGTH_OPERATOR:
self._write("#")
elif node.type == nodes.UnaryOperator.T_MINUS:
self._write("-")
elif node.type == nodes.UnaryOperator.T_NOT:
self._write("not ")
has_subexp = isinstance(node.operand, OPERATOR_TYPES)
need_parentheses = has_subexp and node.operand.type < node.type
if need_parentheses:
self._write("(")
self._visit(node.operand)
if need_parentheses:
self._write(")")
# ##
def visit_statements_list(self, node):
if len(self._states) > 1:
self._start_block()
self._push_state()
def leave_statements_list(self, node):
self._pop_state()
if len(self._states) > 1:
self._end_block()
def _visit_comma_separated_list(self, node):
if node.contents == []:
return
for subnode in node.contents[:-1]:
self._visit(subnode)
self._write(", ")
self._visit(node.contents[-1])
visit_identifiers_list = _visit_comma_separated_list
def visit_records_list(self, node):
if node.contents == []:
return
for subnode in node.contents[:-1]:
self._visit(subnode)
self._write(",")
self._end_line()
self._visit(node.contents[-1])
self._end_line()
visit_variables_list = _visit_comma_separated_list
visit_expressions_list = _visit_comma_separated_list
# ##
def visit_identifier(self, node):
if node.type == nodes.Identifier.T_SLOT:
self._write("slot{0}", node.slot)
else:
self._write(node.name)
def visit_multres(self, node):
self._write("MULTRES")
def visit_table_element(self, node):
key = node.key
base = node.table
is_valid_name = self._is_valid_name(key)
if self._is_global(node):
assert is_valid_name
self._skip(base)
self._skip(key)
self._write(key.value)
return
base_is_constructor = isinstance(base, nodes.TableConstructor)
if not base_is_constructor and is_valid_name:
self._visit(base)
self._write(".")
self._write(key.value)
self._skip(key)
else:
if base_is_constructor:
self._write("(")
self._visit(base)
if base_is_constructor:
self._write(")")
self._write("[")
self._visit(key)
self._write("]")
def visit_vararg(self, node):
self._write("...")
def visit_function_call(self, node):
is_statement = self._state().current_statement == STATEMENT_NONE
if is_statement:
self._start_statement(STATEMENT_FUNCTION_CALL)
# We are going to modify this list so we can remove the first
# argument
args = node.arguments.contents
is_method = False
if len(args) > 0 and isinstance(node.function, nodes.TableElement):
table = node.function.table
first_arg = node.arguments.contents[0]
if self._is_valid_name(node.function.key):
is_method = table == first_arg
if is_method:
self._visit(node.function.table)
self._write(":")
self._write(node.function.key.value)
self._skip(node.function)
args.pop(0)
self._write("(")
self._visit(node.arguments)
self._write(")")
self._skip(node.arguments)
else:
self._visit(node.function)
self._write("(")
self._visit(node.arguments)
self._write(")")
if is_statement:
self._end_statement(STATEMENT_FUNCTION_CALL)
# ##
def visit_if(self, node):
self._start_statement(STATEMENT_IF)
self._write("if ")
self._visit(node.expression)
self._write(" then")
self._end_line()
self._visit(node.then_block)
self._visit_list(node.elseifs)
if len(node.else_block.contents) > 0:
self._write("else")
self._end_line()
self._visit(node.else_block)
else:
self._skip(node.else_block)
self._write("end")
self._end_statement(STATEMENT_IF)
def visit_elseif(self, node):
self._write("elseif ")
self._visit(node.expression)
self._write(" then")
self._end_line()
self._visit(node.then_block)
# ##
def visit_block(self, node):
self._write("--- BLOCK #{0} {1}-{2}, warpins: {3} ---",
node.index,
node.first_address, node.last_address,
node.warpins_count)
self._end_line()
self._visit_list(node.contents)
self._write("--- END OF BLOCK #{0} ---", node.index)
self._end_line()
self._end_line()
self._visit(node.warp)
self._end_line()
self._end_line()
def visit_unconditional_warp(self, node):
if node.type == nodes.UnconditionalWarp.T_FLOW:
self._write("FLOW")
elif node.type == nodes.UnconditionalWarp.T_JUMP:
self._write("UNCONDITIONAL JUMP")
self._write("; TARGET BLOCK #{0}", node.target.index)
self._end_line()
def visit_conditional_warp(self, node):
if hasattr(node, "_slot"):
self._write("slot" + str(node._slot))
self._write(" = ")
self._write("if ")
self._visit(node.condition)
self._write(" then")
self._end_line()
self._write("JUMP TO BLOCK #{0}", node.true_target.index)
self._end_line()
self._write("else")
self._end_line()
self._write("JUMP TO BLOCK #{0}", node.false_target.index)
self._end_line()
self._write("end")
self._end_line()
def visit_iterator_warp(self, node):
self._write("for ")
self._visit(node.variables)
self._write(" in ")
self._visit(node.controls)
self._end_line()
self._write("LOOP BLOCK #{0}", node.body.index)
self._end_line()
self._write("GO OUT TO BLOCK #{0}", node.way_out.index)
self._end_line()
def visit_numeric_loop_warp(self, node):
self._write("for ")
self._visit(node.index)
self._write("=")
self._visit(node.controls)
self._end_line()
self._write("LOOP BLOCK #{0}", node.body.index)
self._end_line()
self._write("GO OUT TO BLOCK #{0}", node.way_out.index)
# ##
def visit_return(self, node):
self._start_statement(STATEMENT_RETURN)
self._write("return ")
self._visit(node.returns)
self._end_statement(STATEMENT_RETURN)
def visit_break(self, node):
self._start_statement(STATEMENT_BREAK)
self._write("break")
self._end_statement(STATEMENT_BREAK)
# ##
def visit_while(self, node):
self._start_statement(STATEMENT_WHILE)
self._write("while ")
self._visit(node.expression)
self._write(" do")
self._end_line()
self._visit(node.statements)
self._write("end")
self._end_statement(STATEMENT_WHILE)
def visit_repeat_until(self, node):
self._start_statement(STATEMENT_REPEAT_UNTIL)
self._write("repeat")
self._end_line()
self._visit(node.statements)
self._write("until ")
self._visit(node.expression)
self._end_statement(STATEMENT_REPEAT_UNTIL)
def visit_numeric_for(self, node):
self._start_statement(STATEMENT_NUMERIC_FOR)
self._write("for ")
self._visit(node.variable)
self._write(" = ")
self._visit(node.expressions)
self._write(" do")
self._end_line()
self._visit(node.statements)
self._write("end")
self._end_statement(STATEMENT_NUMERIC_FOR)
def visit_iterator_for(self, node):
self._start_statement(STATEMENT_ITERATOR_FOR)
self._write("for ")
self._visit(node.identifiers)
self._write(" in ")
self._visit(node.expressions)
self._write(" do")
self._end_line()
self._visit(node.statements)
self._write("end")
self._end_statement(STATEMENT_ITERATOR_FOR)
# ##
def visit_constant(self, node):
if node.type != nodes.Constant.T_STRING:
self._write(node.value)
return
lines = node.value.count("\n")
if lines > 2:
self._write("[[")
self._write("\n")
self._write(node.value)
self._write("]]")
else:
text = node.value
text = text.replace("\\", "\\\\")
text = text.replace("\t", "\\t")
text = text.replace("\n", "\\n")
text = text.replace("\r", "\\r")
text = text.replace("\"", "\\\"")
self._write('"' + text + '"')
def visit_primitive(self, node):
if node.type == nodes.Primitive.T_FALSE:
self._write("false")
elif node.type == nodes.Primitive.T_TRUE:
self._write("true")
else:
self._write("nil")
def _skip(self, node):
self._visited_nodes[-1].add(node)
def _visit(self, node):
assert node is not None
if node in self._visited_nodes[-1]:
return
self._visited_nodes[-1].add(node)
# TODO: add check
# "It looks like you forgot about some node changes..."
self._visited_nodes.append(set())
traverse.Visitor._visit(self, node)
self._visited_nodes.pop()
def write(fd, ast):
assert isinstance(ast, nodes.FunctionDefinition)
visitor = Visitor()
traverse.traverse(visitor, ast.statements)
_process_queue(fd, visitor.print_queue)
def _get_next_significant(queue, i):
i += 1
while i < len(queue):
cmd = queue[i]
if cmd[0] not in (CMD_END_LINE, CMD_WRITE):
break
i += 1
if i < len(queue):
return queue[i]
else:
return (CMD_END_BLOCK,)
def _process_queue(fd, queue):
indent = 0
line_broken = True
for i, cmd in enumerate(queue):
assert isinstance(cmd, tuple)
if cmd[0] == CMD_START_STATEMENT:
# assert line_broken
pass
elif cmd[0] == CMD_END_STATEMENT:
fd.write("\n")
line_broken = True
next_cmd = _get_next_significant(queue, i)
if next_cmd[0] not in (CMD_END_BLOCK, CMD_START_BLOCK):
assert next_cmd[0] == CMD_START_STATEMENT
if next_cmd[1] != cmd[1] \
or cmd[1] >= STATEMENT_IF \
or next_cmd[1] >= STATEMENT_IF:
fd.write("\n")
elif cmd[0] == CMD_END_LINE:
fd.write("\n")
line_broken = True
elif cmd[0] == CMD_START_BLOCK:
indent += 1
elif cmd[0] == CMD_END_BLOCK:
indent -= 1
assert indent >= 0
else:
assert cmd[0] == CMD_WRITE
if line_broken:
fd.write(indent * '\t')
line_broken = False
_id, fmt, args, kargs = cmd
if len(args) + len(kargs) > 0:
text = fmt.format(*args, **kargs)
elif isinstance(fmt, str):
text = fmt
else:
text = str(fmt)
fd.write(text)
================================================
FILE: ljd/pseudoasm/__init__.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
================================================
FILE: ljd/pseudoasm/constants.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
import ljd.bytecode.constants
def write_tables(writer, prototype):
i = 0
for element in prototype.constants.complex_constants:
if isinstance(element, ljd.bytecode.constants.Table):
_write_table(writer, i, element)
i += 1
def _write_table(writer, index, table):
writer.stream.open_block("ktable#{0} = [", index)
i = 0
for element in table.array:
if i != 0 or element is not None:
text = _translate_element(element)
writer.stream.write_line("#{0}: {1},", i, text)
i += 1
for key, value in table.dictionary:
key = _translate_element(key)
value = _translate_element(value)
writer.stream.write_line("[{0}] = {1},", key, value)
writer.stream.close_block("]")
writer.stream.write_line()
writer.stream.write_line()
def _translate_element(element):
if element is True:
return "true"
elif element is False:
return "false"
elif element is None:
return "nil"
elif isinstance(element, bytes):
return '"' + element.decode("utf8") + '"'
else:
return str(element)
================================================
FILE: ljd/pseudoasm/instructions.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
import ljd.bytecode.instructions as ins
from ljd.bytecode.constants import T_NIL, T_FALSE, T_TRUE
import ljd.pseudoasm.prototype
_FORMAT = "{addr:3}\t[{line:3}]\t{name:<5}\t{a:3}\t{b}\t{c}\t; {description}"
_DESCRIPTION_HANDLERS = [None] * 255
class _State():
def __init__(self, writer, prototype, instructions):
for key, value in writer.__dict__.items():
setattr(self, key, value)
self.prototype = prototype
self.instructions = instructions
def write(writer, prototype):
global _MAP
# skip the first function header
addr = 1
instructions = prototype.instructions
writer = _State(writer, prototype, instructions)
while addr < len(instructions):
instruction = instructions[addr]
line = prototype.debuginfo.lookup_line_number(addr)
if instruction.opcode == ins.FNEW.opcode:
_write_function(writer, addr, line, instruction)
else:
_write_instruction(writer, addr, line, instruction)
addr += 1
def _write_instruction(writer, addr, line, instruction):
description = _translate_description(writer, addr, line, instruction)
writer.stream.write_multiline(_FORMAT,
addr=addr,
line=line,
name=instruction.name,
a=instruction.A if instruction.A_type is not None else "",
b=instruction.B if instruction.B_type is not None else "",
c=instruction.CD if instruction.CD_type is not None else "",
description=description
)
def _write_function(writer, addr, line, instruction):
prototype = writer.prototype.constants.complex_constants[instruction.CD]
description = ljd.pseudoasm.prototype.format_header(writer, prototype)
writer.stream.open_block(_FORMAT,
addr=addr,
line=line,
name="FNEW",
a=instruction.A,
b="",
c=instruction.CD,
description=description
)
writer.stream.write_line()
ljd.pseudoasm.prototype.write_body(writer, prototype)
writer.stream.close_block()
def _translate_description(writer, addr, line, instruction):
global _DESCRIPTION_HANDLERS
handler = _DESCRIPTION_HANDLERS[instruction.opcode]
description = instruction.description
return handler(writer, description, addr, line, instruction)
def _translate(writer, addr, value, attr_type):
prototype = writer.prototype
if attr_type == ins.T_DST or attr_type == ins.T_BS:
return "slot" + str(value)
if attr_type == ins.T_VAR:
name = _lookup_variable_name(writer, addr, value)
if name is not None:
return name
else:
return "slot" + str(value)
elif attr_type == ins.T_UV:
name = prototype.debuginfo.lookup_upvalue_name(value)
return "uv" + str(value) + '"' + name + '"'
elif attr_type == ins.T_PRI:
if value is None or value == T_NIL:
return "nil"
elif value is True or value == T_TRUE:
return "true"
else:
assert value is False or value == T_FALSE
return "false"
elif attr_type == ins.T_NUM:
return str(prototype.constants.numeric_constants[value])
elif attr_type == ins.T_STR:
binary = prototype.constants.complex_constants[value]
return '"' + binary + '"'
elif attr_type == ins.T_TAB:
return "table#k" + str(value)
elif attr_type == ins.T_CDT:
return str(prototype.constants.complex_constants[value])
elif attr_type == ins.T_JMP:
return str(1 + addr + value)
elif attr_type == ins.T_LIT or attr_type == ins.T_SLIT:
return str(value)
elif attr_type == ins.T_BS or attr_type == ins.T_RBS:
return "r" + str(value)
else:
return " " # "r" + str(value)
def _lookup_variable_name(writer, addr, slot):
while True:
result = _lookup_variable_name_step(writer, addr, slot)
if isinstance(result, tuple):
addr = result[0]
slot = result[1]
continue
return result
def _lookup_variable_name_step(writer, addr, slot):
info = writer.prototype.debuginfo.lookup_local_name(addr, slot)
if info is not None:
name = info.name
if name[0] == '<':
name = "slot" + str(slot) + name
return name
instructions = writer.instructions
knil_opcode = ins.KNIL.opcode
constants = writer.prototype.constants.complex_constants
while addr > 1:
addr -= 1
instruction = instructions[addr]
if instruction.A_type == ins.T_BS:
if slot >= instruction.A \
and (instruction.opcode == knil_opcode \
or slot <= instruction.CD):
return None
continue
if instruction.A_type != ins.T_DST or instruction.A != slot:
continue
if instruction.opcode == ins.MOV.opcode:
# Retry with new addr and slot
return addr, instruction.CD
if instruction.opcode == ins.GGET.opcode:
binary = constants[instruction.CD]
return binary
# field or method
if instruction.opcode == ins.TGETS.opcode:
table_slot = instruction.B
table = _lookup_variable_name_step(writer, addr, table_slot)
if table is None:
table = "<unknown table>"
binary = constants[instruction.CD]
return table + "." + binary
if instruction.opcode == ins.UGET.opcode:
uv = instruction.CD
name = writer.prototype.debuginfo.lookup_upvalue_name(uv)
return "uv" + str(uv) + '"' + name + '"'
return None
return None
def _translate_standard(writer, addr, line, instruction):
A = None
B = None
CD = None
if instruction.A_type is not None:
A = _translate(writer, addr, instruction.A, instruction.A_type)
if instruction.B_type is not None:
B = _translate(writer, addr, instruction.B, instruction.B_type)
if instruction.CD_type is not None:
CD = _translate(writer, addr, instruction.CD, instruction.CD_type)
return A, B, CD
def _translate_normal(writer, description, addr, line, instruction):
A, B, CD = _translate_standard(writer, addr, line, instruction)
return description.format(A=A, B=B, C=CD, D=CD)
def _translate_concat(writer, description, addr, line, instruction):
A = _translate(writer, addr, instruction.A, instruction.A_type)
args = []
start = instruction.B
end = instruction.CD + 1
while start != end:
var = _translate(writer, addr, start, ins.T_VAR)
args.append(var)
start += 1
return description.format(A=A, concat_from_B_to_C=" .. ".join(args))
def _translate_nil(writer, description, addr, line, instruction):
args = []
start = instruction.A
end = instruction.CD + 1
while start != end:
var = _translate(writer, addr, start, ins.T_VAR)
args.append(var)
start += 1
return description.format(from_A_to_D=", ".join(args))
def _translate_table_str_op(writer, description, addr, line, instruction):
A, B, CD = _translate_standard(writer, addr, line, instruction)
C = CD[1:-1]
return description.format(A=A, B=B, C=C)
def _translate_new_table(writer, description, addr, line, instruction):
A = _translate(writer, addr, instruction.A, instruction.A_type)
size = instruction.CD
array_size = size & 0b0000011111111111
dict_size = 2 ** (size >> 11)
return description.format(
A=A,
D_array=array_size,
D_dict=dict_size,
)
def _translate_mass_set(writer, description, addr, line, instruction):
base = instruction.A
table_var = _translate(writer, addr, base - 1, ins.T_VAR)
first = instruction.CD
return description.format(
A_minus_one=table_var,
A=base,
D_low=first,
)
def _translate_varg_call(writer, description, addr, line, instruction):
base = instruction.A
argn = instruction.CD
retn = instruction.B - 1
args = []
returns = []
i = 0
while i < argn:
args.append(_translate(writer, addr, base + i + 1, ins.T_VAR))
i += 1
i = 0
while i < retn:
returns.append(_translate(writer, addr, base + i, ins.T_DST))
i += 1
func_var = _translate(writer, addr, base, ins.T_VAR)
return description.format(
A=func_var,
from_A_x_B_minus_two=", ".join(returns) if retn >= 0 else "MULTRES",
from_A_plus_one_x_C=", ".join(args)
)
def _translate_call(writer, description, addr, line, instruction):
base = instruction.A
argn = instruction.CD - 1
retn = instruction.B - 1
args = []
returns = []
i = 0
while i < argn:
args.append(_translate(writer, addr, base + i + 1, ins.T_VAR))
i += 1
i = 0
while i < retn:
returns.append(_translate(writer, addr, base + i, ins.T_DST))
i += 1
func_var = _translate(writer, addr, base, ins.T_VAR)
return description.format(
A=func_var,
from_A_x_B_minus_two=", ".join(returns) if retn >= 0 else "MULTRES",
from_A_plus_one_x_C_minus_one=", ".join(args)
)
def _translate_varg_tailcall(writer, description, addr, line, instruction):
base = instruction.A
argn = instruction.CD - 1
args = []
i = 0
while i < argn:
args.append(_translate(writer, addr, base + i + 1, ins.T_VAR))
i += 1
func_var = _translate(writer, addr, base, ins.T_VAR)
return description.format(
A=func_var,
from_A_plus_one_x_D=", ".join(args)
)
def _translate_tailcall(writer, description, addr, line, instruction):
base = instruction.A
argn = instruction.CD - 1
args = []
i = 0
while i < argn:
args.append(_translate(writer, addr, base + i + 1, ins.T_VAR))
i += 1
func_var = _translate(writer, addr, base, ins.T_VAR)
return description.format(
A=func_var,
from_A_plus_one_x_D_minus_one=", ".join(args)
)
def _translate_iterator(writer, description, addr, line, instruction):
base = instruction.A
A = _translate(writer, addr, instruction.A, ins.T_DST)
A_plus_one = _translate(writer, addr, instruction.A + 1, ins.T_DST)
A_plus_two = _translate(writer, addr, instruction.A + 2, ins.T_DST)
A_minus_three = _translate(writer, addr, instruction.A - 3, ins.T_VAR)
A_minus_two = _translate(writer, addr, instruction.A - 2, ins.T_VAR)
A_minus_one = _translate(writer, addr, instruction.A - 1, ins.T_VAR)
retn = instruction.B - 1
returns = []
i = 0
while i < retn:
returns.append(_translate(writer, addr, base + i, ins.T_DST))
i += 1
return description.format(
A=A,
A_plus_one=A_plus_one,
A_plus_two=A_plus_two,
A_minus_three=A_minus_three,
A_minus_two=A_minus_two,
A_minus_one=A_minus_one,
from_A_x_B_minus_two=", ".join(returns)
)
def _translate_vararg(writer, description, addr, line, instruction):
returns = []
base = instruction.A
count = instruction.B - 2
if count < 0:
return description.format(from_A_x_B_minus_two="MULTRES")
i = 0
while i <= count:
returns.append(_translate(writer, addr, base + i, ins.T_DST))
i += 1
return description.format(
from_A_x_B_minus_two=", ".join(returns)
)
def _translate_return_mult(writer, description, addr, line, instruction):
returns = []
base = instruction.A
count = instruction.CD - 1
i = 0
while i < count:
returns.append(_translate(writer, addr, base + i, ins.T_VAR))
i += 1
return description.format(
from_A_x_D_minus_one=", ".join(returns)
)
def _translate_return_many(writer, description, addr, line, instruction):
returns = []
base = instruction.A
count = instruction.CD - 2
i = 0
while i < count:
returns.append(_translate(writer, addr, base + i, ins.T_VAR))
i += 1
return description.format(
from_A_x_D_minus_two=", ".join(returns)
)
def _translate_return_one(writer, description, addr, line, instruction):
A = _translate(writer, addr, instruction.A, ins.T_VAR)
return description.format(A=A)
def _translate_for_init(writer, description, addr, line, instruction):
idx = _translate(writer, addr, instruction.A, ins.T_BS)
stop = _translate(writer, addr, instruction.A + 1, ins.T_BS)
step = _translate(writer, addr, instruction.A + 2, ins.T_BS)
ext_idx = _translate(writer, addr, instruction.A + 3, ins.T_VAR)
return description.format(
A=idx,
A_plus_one=stop,
A_plus_two=step,
A_plus_three=ext_idx,
D=_translate(writer, addr, instruction.CD, ins.T_JMP)
)
def _translate_numeric_loop(writer, description, addr, line, instruction):
stop = _translate(writer, addr, instruction.A + 1, ins.T_VAR)
step = _translate(writer, addr, instruction.A + 2, ins.T_VAR)
ext_idx = _translate(writer, addr, instruction.A + 3, ins.T_VAR)
# ext_idx isn't correct var here, but for the visualisation sake we will
# omit all the stuff with the internal idx var
return description.format(
A=ext_idx,
A_plus_one=stop,
A_plus_two=step,
D=_translate(writer, addr, instruction.CD, ins.T_JMP)
)
def _translate_iter_loop(writer, description, addr, line, instruction):
A_minus_one = _translate(writer, addr, instruction.A - 1, ins.T_VAR)
A = _translate(writer, addr, instruction.A, ins.T_VAR)
return description.format(
A_minus_one=A_minus_one,
A=A,
D=_translate(writer, addr, instruction.CD, ins.T_JMP)
)
_HANDLERS_MAP = (
# Comparison ops
(ins.ISLT.opcode, _translate_normal),
(ins.ISGE.opcode, _translate_normal),
(ins.ISLE.opcode, _translate_normal),
(ins.ISGT.opcode, _translate_normal),
(ins.ISEQV.opcode, _translate_normal),
(ins.ISNEV.opcode, _translate_normal),
(ins.ISEQS.opcode, _translate_normal),
(ins.ISNES.opcode, _translate_normal),
(ins.ISEQN.opcode, _translate_normal),
(ins.ISNEN.opcode, _translate_normal),
(ins.ISEQP.opcode, _translate_normal),
(ins.ISNEP.opcode, _translate_normal),
# Unary test and copy ops
(ins.ISTC.opcode, _translate_normal),
(ins.ISFC.opcode, _translate_normal),
(ins.IST.opcode, _translate_normal),
(ins.ISF.opcode, _translate_normal),
# Unary ops
(ins.MOV.opcode, _translate_normal),
(ins.NOT.opcode, _translate_normal),
(ins.UNM.opcode, _translate_normal),
(ins.LEN.opcode, _translate_normal),
# Binary ops
(ins.ADDVN.opcode, _translate_normal),
(ins.SUBVN.opcode, _translate_normal),
(ins.MULVN.opcode, _translate_normal),
(ins.DIVVN.opcode, _translate_normal),
(ins.MODVN.opcode, _translate_normal),
(ins.ADDNV.opcode, _translate_normal),
(ins.SUBNV.opcode, _translate_normal),
(ins.MULNV.opcode, _translate_normal),
(ins.DIVNV.opcode, _translate_normal),
(ins.MODNV.opcode, _translate_normal),
(ins.ADDVV.opcode, _translate_normal),
(ins.SUBVV.opcode, _translate_normal),
(ins.MULVV.opcode, _translate_normal),
(ins.DIVVV.opcode, _translate_normal),
(ins.MODVV.opcode, _translate_normal),
(ins.POW.opcode, _translate_normal),
(ins.CAT.opcode, _translate_concat),
# Constant ops
(ins.KSTR.opcode, _translate_normal),
(ins.KCDATA.opcode, _translate_normal),
(ins.KSHORT.opcode, _translate_normal),
(ins.KNUM.opcode, _translate_normal),
(ins.KPRI.opcode, _translate_normal),
(ins.KNIL.opcode, _translate_nil),
# Upvalue and function ops
(ins.UGET.opcode, _translate_normal),
(ins.USETV.opcode, _translate_normal),
(ins.USETS.opcode, _translate_normal),
(ins.USETN.opcode, _translate_normal),
(ins.USETP.opcode, _translate_normal),
(ins.UCLO.opcode, _translate_normal),
(ins.FNEW.opcode, _translate_normal),
# Table ops
(ins.TNEW.opcode, _translate_new_table),
(ins.TDUP.opcode, _translate_normal),
(ins.GGET.opcode, _translate_normal),
(ins.GSET.opcode, _translate_normal),
(ins.TGETV.opcode, _translate_normal),
(ins.TGETS.opcode, _translate_table_str_op),
(ins.TGETB.opcode, _translate_normal),
(ins.TSETV.opcode, _translate_normal),
(ins.TSETS.opcode, _translate_table_str_op),
(ins.TSETB.opcode, _translate_normal),
(ins.TSETM.opcode, _translate_mass_set),
# Calls and vararg handling
(ins.CALLM.opcode, _translate_varg_call),
(ins.CALL.opcode, _translate_call),
(ins.CALLMT.opcode, _translate_varg_tailcall),
(ins.CALLT.opcode, _translate_tailcall),
(ins.ITERC.opcode, _translate_iterator),
(ins.ITERN.opcode, _translate_iterator),
(ins.VARG.opcode, _translate_vararg),
(ins.ISNEXT.opcode, _translate_normal),
# Returns
(ins.RETM.opcode, _translate_return_mult),
(ins.RET.opcode, _translate_return_many),
(ins.RET0.opcode, _translate_normal),
(ins.RET1.opcode, _translate_return_one),
# Loops and branches
(ins.FORI.opcode, _translate_for_init),
(ins.JFORI.opcode, _translate_for_init),
(ins.FORL.opcode, _translate_numeric_loop),
(ins.IFORL.opcode, _translate_numeric_loop),
(ins.JFORL.opcode, _translate_numeric_loop),
(ins.ITERL.opcode, _translate_iter_loop),
(ins.IITERL.opcode, _translate_iter_loop),
(ins.JITERL.opcode, _translate_iter_loop),
(ins.LOOP.opcode, _translate_normal),
(ins.ILOOP.opcode, _translate_normal),
(ins.JLOOP.opcode, _translate_normal),
(ins.JMP.opcode, _translate_normal),
# Function headers
(ins.FUNCF.opcode, _translate_normal),
(ins.IFUNCF.opcode, _translate_normal),
(ins.JFUNCF.opcode, _translate_normal),
(ins.FUNCV.opcode, _translate_normal),
(ins.IFUNCV.opcode, _translate_normal),
(ins.JFUNCV.opcode, _translate_normal),
(ins.FUNCC.opcode, _translate_normal),
(ins.FUNCCW.opcode, _translate_normal)
)
def _init():
global _HANDLERS_MAP, _DESCRIPTION_HANDLERS
for opcode, handler in _HANDLERS_MAP:
_DESCRIPTION_HANDLERS[opcode] = handler
del globals()["_init"]
del globals()["_HANDLERS_MAP"]
_init()
================================================
FILE: ljd/pseudoasm/prototype.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
import ljd.pseudoasm.constants
import ljd.pseudoasm.instructions
def write(writer, prototype):
_write_header(writer, prototype)
write_body(writer, prototype)
writer.stream.close_block("")
def _write_header(writer, prototype):
writer.stream.open_block("main {0}", format_header(writer, prototype))
def format_header(writer, prototype):
return "{s}:{start}-{end}: {argn}{varg} args," \
" {uvs} upvalues, {slots} slots".format(
s=writer.source,
start=prototype.first_line_number,
end=prototype.first_line_number + prototype.lines_count,
argn=prototype.arguments_count,
varg="+" if prototype.flags.is_variadic else "",
uvs=len(prototype.constants.upvalue_references),
slots=prototype.framesize
)
def write_body(writer, prototype):
writer.stream.write_line(";;;; constant tables ;;;;")
ljd.pseudoasm.constants.write_tables(writer, prototype)
writer.stream.write_line(";;;; instructions ;;;;")
ljd.pseudoasm.instructions.write(writer, prototype)
================================================
FILE: ljd/pseudoasm/writer.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
import ljd.util.indentedstream
import ljd.pseudoasm.prototype
class _State():
def __init__(self):
self.flags = None
self.stream = None
self.source = None
def write(fd, header, prototype):
writer = _State()
writer.stream = ljd.util.indentedstream.IndentedStream(fd)
writer.flags = header.flags
writer.source = "N/A" if header.flags.is_stripped else header.name
_write_header(writer, header)
ljd.pseudoasm.prototype.write(writer, prototype)
def _write_header(writer, header):
writer.stream.write_multiline("""
;
; Disassemble of {origin}
;
; Source file: {source}
;
; Flags:
; Stripped: {stripped}
; Endianness: {endianness}
; FFI: {ffi}
;
""", origin=header.origin,
source=writer.source,
stripped="Yes" if header.flags.is_stripped else "No",
endianness="Big" if header.flags.is_big_endian else "Little",
ffi="Present" if header.flags.has_ffi else "Not present")
================================================
FILE: ljd/rawdump/__init__.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
================================================
FILE: ljd/rawdump/code.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
from ljd.util.log import errprint
import ljd.bytecode.instructions as instructions
_OPCODES = (
# Comparison ops
(0x00, instructions.ISLT), # @UndefinedVariable
(0x01, instructions.ISGE), # @UndefinedVariable
(0x02, instructions.ISLE), # @UndefinedVariable
(0x03, instructions.ISGT), # @UndefinedVariable
(0x04, instructions.ISEQV), # @UndefinedVariable
(0x05, instructions.ISNEV), # @UndefinedVariable
(0x06, instructions.ISEQS), # @UndefinedVariable
(0x07, instructions.ISNES), # @UndefinedVariable
(0x08, instructions.ISEQN), # @UndefinedVariable
(0x09, instructions.ISNEN), # @UndefinedVariable
(0x0A, instructions.ISEQP), # @UndefinedVariable
(0x0B, instructions.ISNEP), # @UndefinedVariable
# Unary test and copy ops
(0x0C, instructions.ISTC), # @UndefinedVariable
(0x0D, instructions.ISFC), # @UndefinedVariable
(0x0E, instructions.IST), # @UndefinedVariable
(0x0F, instructions.ISF), # @UndefinedVariable
(0x0E, instructions.ISTYPE), # @UndefinedVariable
(0x0F, instructions.ISNUM), # @UndefinedVariable
# Unary ops
(0x10, instructions.MOV), # @UndefinedVariable
(0x11, instructions.NOT), # @UndefinedVariable
(0x12, instructions.UNM), # @UndefinedVariable
(0x13, instructions.LEN), # @UndefinedVariable
# Binary ops
(0x14, instructions.ADDVN), # @UndefinedVariable
(0x15, instructions.SUBVN), # @UndefinedVariable
(0x16, instructions.MULVN), # @UndefinedVariable
(0x17, instructions.DIVVN), # @UndefinedVariable
(0x18, instructions.MODVN), # @UndefinedVariable
(0x19, instructions.ADDNV), # @UndefinedVariable
(0x1A, instructions.SUBNV), # @UndefinedVariable
(0x1B, instructions.MULNV), # @UndefinedVariable
(0x1C, instructions.DIVNV), # @UndefinedVariable
(0x1D, instructions.MODNV), # @UndefinedVariable
(0x1E, instructions.ADDVV), # @UndefinedVariable
(0x1F, instructions.SUBVV), # @UndefinedVariable
(0x20, instructions.MULVV), # @UndefinedVariable
(0x21, instructions.DIVVV), # @UndefinedVariable
(0x22, instructions.MODVV), # @UndefinedVariable
(0x23, instructions.POW), # @UndefinedVariable
(0x24, instructions.CAT), # @UndefinedVariable
# Constant ops
(0x25, instructions.KSTR), # @UndefinedVariable
(0x26, instructions.KCDATA), # @UndefinedVariable
(0x27, instructions.KSHORT), # @UndefinedVariable
(0x28, instructions.KNUM), # @UndefinedVariable
(0x29, instructions.KPRI), # @UndefinedVariable
(0x2A, instructions.KNIL), # @UndefinedVariable
# Upvalue and function ops
(0x2B, instructions.UGET), # @UndefinedVariable
(0x2C, instructions.USETV), # @UndefinedVariable
(0x2D, instructions.USETS), # @UndefinedVariable
(0x2E, instructions.USETN), # @UndefinedVariable
(0x2F, instructions.USETP), # @UndefinedVariable
(0x30, instructions.UCLO), # @UndefinedVariable
(0x31, instructions.FNEW), # @UndefinedVariable
# Table ops
(0x32, instructions.TNEW), # @UndefinedVariable
(0x33, instructions.TDUP), # @UndefinedVariable
(0x34, instructions.GGET), # @UndefinedVariable
(0x35, instructions.GSET), # @UndefinedVariable
(0x36, instructions.TGETV), # @UndefinedVariable
(0x37, instructions.TGETS), # @UndefinedVariable
(0x38, instructions.TGETB), # @UndefinedVariable
(0x38, instructions.TGETR), # @UndefinedVariable
(0x39, instructions.TSETV), # @UndefinedVariable
(0x3A, instructions.TSETS), # @UndefinedVariable
(0x3B, instructions.TSETB), # @UndefinedVariable
(0x3C, instructions.TSETM), # @UndefinedVariable
(0x3B, instructions.TSETR), # @UndefinedVariable
# Calls and vararg handling
(0x3D, instructions.CALLM), # @UndefinedVariable
(0x3E, instructions.CALL), # @UndefinedVariable
(0x3F, instructions.CALLMT), # @UndefinedVariable
(0x40, instructions.CALLT), # @UndefinedVariable
(0x41, instructions.ITERC), # @UndefinedVariable
(0x42, instructions.ITERN), # @UndefinedVariable
(0x43, instructions.VARG), # @UndefinedVariable
(0x44, instructions.ISNEXT), # @UndefinedVariable
# Returns
(0x45, instructions.RETM), # @UndefinedVariable
(0x46, instructions.RET), # @UndefinedVariable
(0x47, instructions.RET0), # @UndefinedVariable
(0x48, instructions.RET1), # @UndefinedVariable
# Loops and branches
(0x49, instructions.FORI), # @UndefinedVariable
(0x4A, instructions.JFORI), # @UndefinedVariable
(0x4B, instructions.FORL), # @UndefinedVariable
(0x4C, instructions.IFORL), # @UndefinedVariable
(0x4D, instructions.JFORL), # @UndefinedVariable
(0x4E, instructions.ITERL), # @UndefinedVariable
(0x4F, instructions.IITERL), # @UndefinedVariable
(0x50, instructions.JITERL), # @UndefinedVariable
(0x51, instructions.LOOP), # @UndefinedVariable
(0x52, instructions.ILOOP), # @UndefinedVariable
(0x53, instructions.JLOOP), # @UndefinedVariable
(0x54, instructions.JMP), # @UndefinedVariable
# Function headers
(0x55, instructions.FUNCF), # @UndefinedVariable
(0x56, instructions.IFUNCF), # @UndefinedVariable
(0x57, instructions.JFUNCF), # @UndefinedVariable
(0x58, instructions.FUNCV), # @UndefinedVariable
(0x59, instructions.IFUNCV), # @UndefinedVariable
(0x5A, instructions.JFUNCV), # @UndefinedVariable
(0x5B, instructions.FUNCC), # @UndefinedVariable
(0x5C, instructions.FUNCCW) # @UndefinedVariable
)
_MAP = [None] * 256
def read(parser):
global _MAP
codeword = parser.stream.read_uint(4)
opcode = codeword & 0xFF
instruction_class = _MAP[opcode]
if instruction_class is None:
errprint("Warning: unknown opcode {0:08x}", opcode)
instruction_class = instructions.UNKNW # @UndefinedVariable #zzw.20180714
instruction = instruction_class()
if instruction_class.opcode != opcode:
instruction.opcode = opcode
_set_instruction_operands(parser, codeword, instruction)
return instruction
def _set_instruction_operands(parser, codeword, instruction):
if instruction.args_count == 3:
A = (codeword >> 8) & 0xFF
CD = (codeword >> 16) & 0xFF
B = (codeword >> 24) & 0xFF
else:
A = (codeword >> 8) & 0xFF
CD = (codeword >> 16) & 0xFFFF
if instruction.A_type is not None:
instruction.A = _process_operand(parser, instruction.A_type, A)
if instruction.B_type is not None:
instruction.B = _process_operand(parser, instruction.B_type, B)
if instruction.CD_type is not None:
instruction.CD = _process_operand(parser, instruction.CD_type, CD)
def _process_operand(parser, operand_type, operand):
if operand_type == instructions.T_STR \
or operand_type == instructions.T_TAB \
or operand_type == instructions.T_FUN \
or operand_type == instructions.T_CDT:
return parser.complex_constants_count - operand - 1
elif operand_type == instructions.T_JMP:
return operand - 0x8000
else:
return operand
def _init():
global _OPCODES, _MAP
opcode = 0
for instruction in _OPCODES:
_MAP[opcode] = instruction[1]
opcode = opcode + 1
del globals()["_init"]
del globals()["_OPCODES"]
_init()
================================================
FILE: ljd/rawdump/constants.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
import sys
import struct
import ljd.bytecode.constants
import gconfig
BCDUMP_KGC_CHILD = 0
BCDUMP_KGC_TAB = 1
BCDUMP_KGC_I64 = 2
BCDUMP_KGC_U64 = 3
BCDUMP_KGC_COMPLEX = 4
BCDUMP_KGC_STR = 5
BCDUMP_KTAB_NIL = 0
BCDUMP_KTAB_FALSE = 1
BCDUMP_KTAB_TRUE = 2
BCDUMP_KTAB_INT = 3
BCDUMP_KTAB_NUM = 4
BCDUMP_KTAB_STR = 5
def read(parser, constants):
r = True
r = r and _read_upvalue_references(parser, constants.upvalue_references)
r = r and _read_complex_constants(parser, constants.complex_constants)
r = r and _read_numeric_constants(parser, constants.numeric_constants)
return r
def _read_upvalue_references(parser, references):
i = 0
while i < parser.upvalues_count:
i += 1
upvalue = parser.stream.read_uint(2)
references.append(upvalue)
return True
def _read_complex_constants(parser, complex_constants):
i = 0
while i < parser.complex_constants_count:
constant_type = parser.stream.read_uleb128()
if constant_type >= BCDUMP_KGC_STR:
length = constant_type - BCDUMP_KGC_STR
string = parser.stream.read_bytes(length)
#print (string.decode("unicode-escape"))
#print(str(complex_constants))
#zzw 20180714 support str encode
complex_constants.append(string.decode(gconfig.gFlagDic['strEncode']))
elif constant_type == BCDUMP_KGC_TAB:
table = ljd.bytecode.constants.Table()
if not _read_table(parser, table):
return False
complex_constants.append(table)
elif constant_type != BCDUMP_KGC_CHILD:
number = _read_number(parser)
if constant_type == BCDUMP_KGC_COMPLEX:
imaginary = _read_number(parser)
complex_constants.append((number, imaginary))
else:
complex_constants.append(number)
else:
complex_constants.append(parser.prototypes.pop())
i += 1
return True
def _read_numeric_constants(parser, numeric_constants):
i = 0
while i < parser.numeric_constants_count:
isnum, lo = parser.stream.read_uleb128_from33bit()
if isnum:
hi = parser.stream.read_uleb128()
number = _assemble_number(lo, hi)
else:
number = _process_sign(lo)
numeric_constants.append(number)
i += 1
return True
def _read_number(parser):
lo = parser.stream.read_uleb128()
hi = parser.stream.read_uleb128()
return _assemble_number(lo, hi)
def _read_signed_int(parser):
number = parser.stream.read_uleb128()
return _process_sign(number)
def _assemble_number(lo, hi):
if sys.byteorder == 'big':
float_as_int = lo << 32 | hi
else:
float_as_int = hi << 32 | lo
raw_bytes = struct.pack("=Q", float_as_int)
return struct.unpack("=d", raw_bytes)[0]
def _process_sign(number):
if number & 0x80000000:
return -0x100000000 + number
else:
return number
def _read_table(parser, table):
array_items_count = parser.stream.read_uleb128()
hash_items_count = parser.stream.read_uleb128()
while array_items_count > 0:
constant = _read_table_item(parser)
table.array.append(constant)
array_items_count -= 1
while hash_items_count > 0:
key = _read_table_item(parser)
value = _read_table_item(parser)
table.dictionary.append((key, value))
hash_items_count -= 1
return True
def _read_table_item(parser):
data_type = parser.stream.read_uleb128()
if data_type >= BCDUMP_KTAB_STR:
length = data_type - BCDUMP_KTAB_STR
# zzw 20180714 support str encode
return parser.stream.read_bytes(length).decode(gconfig.gFlagDic['strEncode'])
elif data_type == BCDUMP_KTAB_INT:
return _read_signed_int(parser)
elif data_type == BCDUMP_KTAB_NUM:
return _read_number(parser)
elif data_type == BCDUMP_KTAB_TRUE:
return True
elif data_type == BCDUMP_KTAB_FALSE:
return False
else:
assert data_type == BCDUMP_KTAB_NIL
return None
================================================
FILE: ljd/rawdump/debuginfo.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
import sys
import ljd.bytecode.debuginfo
VARNAME_END = 0
VARNAME_FOR_IDX = 1
VARNAME_FOR_STOP = 2
VARNAME_FOR_STEP = 3
VARNAME_FOR_GEN = 4
VARNAME_FOR_STATE = 5
VARNAME_FOR_CTL = 6
VARNAME__MAX = 7
INTERNAL_VARNAMES = [
None,
"<index>",
"<limit>",
"<step>",
"<generator>",
"<state>",
"<control>"
]
def read(parser, line_offset, debuginfo):
r = True
r = r and _read_lineinfo(parser, line_offset, debuginfo.addr_to_line_map)
r = r and _read_upvalue_names(parser, debuginfo.upvalue_variable_names)
r = r and _read_variable_infos(parser, debuginfo.variable_info)
return r
def _read_lineinfo(parser, line_offset, lineinfo):
if parser.lines_count >= 65536:
lineinfo_size = 4
elif parser.lines_count >= 256:
lineinfo_size = 2
else:
lineinfo_size = 1
lineinfo.append(0)
while len(lineinfo) < parser.instructions_count + 1:
line_number = parser.stream.read_uint(lineinfo_size)
lineinfo.append(line_offset + line_number)
return True
def _read_upvalue_names(parser, names):
while len(names) < parser.upvalues_count:
string = parser.stream.read_zstring()
names.append(string.decode("utf-8"))
return True
def _read_variable_infos(parser, infos):
# pc - program counter
last_addr = 0
while True:
info = ljd.bytecode.debuginfo.VariableInfo()
internal_vartype = parser.stream.read_byte()
if internal_vartype >= VARNAME__MAX:
prefix = internal_vartype.to_bytes(1, sys.byteorder)
suffix = parser.stream.read_zstring()
info.name = (prefix + suffix).decode("utf-8")
info.type = info.T_VISIBILE
elif internal_vartype == VARNAME_END:
break
else:
index = internal_vartype
info.name = INTERNAL_VARNAMES[index]
info.type = info.T_INTERNAL
start_addr = last_addr + parser.stream.read_uleb128()
end_addr = start_addr + parser.stream.read_uleb128()
info.start_addr = start_addr
info.end_addr = end_addr
last_addr = start_addr
infos.append(info)
return True
================================================
FILE: ljd/rawdump/header.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
from ljd.util.log import errprint
_MAGIC = b'\x1bLJ'
_MAX_VERSION = 0x80
_FLAG_IS_BIG_ENDIAN = 0b00000001
_FLAG_IS_STRIPPED = 0b00000010
_FLAG_HAS_FFI = 0b00000100
class Flags():
def __init__(self):
self.is_big_endian = False
self.is_stripped = False
self.has_ffi = False
class Header():
def __init__(self):
self.version = 0
self.flags = Flags()
self.origin = b''
self.name = b''
def read(state, header):
r = True
header.origin = state.stream.name
r = r and _check_magic(state)
r = r and _read_version(state, header)
r = r and _read_flags(state, header)
r = r and _read_name(state, header)
#print("header good!")
return r
def _check_magic(state):
if state.stream.read_bytes(3) != _MAGIC:
errprint("Invalid magic, not a LuaJIT format")
return False
return True
def _read_version(state, header):
header.version = state.stream.read_byte()
if header.version > _MAX_VERSION:
errprint("Version {0}: propritary modifications",
header.version)
return False
return True
def _read_flags(parser, header):
bits = parser.stream.read_uleb128()
header.flags.is_big_endian = bits & _FLAG_IS_BIG_ENDIAN
bits &= ~_FLAG_IS_BIG_ENDIAN
header.flags.is_stripped = bits & _FLAG_IS_STRIPPED
bits &= ~_FLAG_IS_STRIPPED
header.flags.has_ffi = bits & _FLAG_HAS_FFI
bits &= ~_FLAG_HAS_FFI
# zzw.20180714 pitch: flag value is according to parser.flag when parse proto, not by header.flags
parser.flags.is_big_endian = header.flags.is_big_endian
parser.flags.is_stripped = header.flags.is_stripped
parser.flags.is_big_endian = header.flags.has_ffi
if bits != 0:
errprint("Unknown flags set: {0:08b}", bits)
return False
return True
def _read_name(state, header):
if header.flags.is_stripped:
header.name = state.stream.name
else:
length = state.stream.read_uleb128()
header.name = state.stream.read_bytes(length).decode("utf8")
return True
================================================
FILE: ljd/rawdump/parser.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
#!/usr/bin/python3
import ljd.util.binstream
from ljd.util.log import errprint
import ljd.bytecode.prototype
import ljd.rawdump.header
import ljd.rawdump.prototype
class _State():
def __init__(self):
self.stream = ljd.util.binstream.BinStream()
self.flags = ljd.rawdump.header.Flags()
self.prototypes = []
def parse(filename):
parser = _State()
parser.stream.open(filename)
header = ljd.rawdump.header.Header()
r = True
try:
r = r and _read_header(parser, header)
#print("good1")
r = r and _read_prototypes(parser, parser.prototypes)
#print("good2")
except IOError as e:
errprint("I/O error while reading dump: {0}", str(e))
r = False
if r and not parser.stream.eof():
errprint("Stopped before whole file was read, something wrong")
r = False
if r and len(parser.prototypes) != 1:
errprint("Invalid prototypes stack order")
r = False
parser.stream.close()
if r:
return header, parser.prototypes[0]
else:
return None, None
#zzw 20180716 解析字节码文件头部结构header
'''
| magic(3 bytes)| version(1 byte) | flag(1 uleb128) [| name(1 uleb128) |]
'''
def _read_header(parser, header):
if not ljd.rawdump.header.read(parser, header):
errprint("Failed to read raw-dump header")
return False
if header.flags.is_big_endian:
parser.stream.data_byteorder = 'big'
else:
parser.stream.data_byteorder = 'little'
return True
def _read_prototypes(state, prototypes):
while not state.stream.eof():
prototype = ljd.bytecode.prototype.Prototype()
#print ("good rwsdump->parser->read_prototypes")
if not ljd.rawdump.prototype.read(state, prototype):
if state.stream.eof():
break
else:
errprint("Failed to read prototype")
return False
prototypes.append(prototype)
return True
================================================
FILE: ljd/rawdump/prototype.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
from ljd.util.log import errprint
import ljd.bytecode.instructions as ins
import ljd.rawdump.constants
import ljd.rawdump.debuginfo
import ljd.rawdump.code
FLAG_HAS_CHILD = 0b00000001
FLAG_IS_VARIADIC = 0b00000010
FLAG_HAS_FFI = 0b00000100
FLAG_JIT_DISABLED = 0b00001000
FLAG_HAS_ILOOP = 0b00010000
class _State():
def __init__(self, parser):
for key, value in parser.__dict__.items():
setattr(self, key, value)
self.upvalues_count = 0
self.complex_constants_count = 0
self.numeric_constants_count = 0
self.instructions_count = 0
self.debuginfo_size = 0
'''
zzw 20180716
| size(1 uleb128) | flag(1 byte) | arguments_count(1 byte)
| framesize(1 byte) | upvalues_count(1 byte) | complex_constants_count(1 uleb128)
| numeric_constants_count(1 uleb128) | instructions_count(1 uleb128) [| debuginfo_size(1 uleb128)
| first_line_number(1 uleb128) | lines_count(1 uleb128) ] | instructions(protoheader define)
| constants(protoheader define) | debginfo(protoheader flage define) |
'''
def read(parser, prototype):
parser = _State(parser)
size = parser.stream.read_uleb128()
if size == 0:
return False
if not parser.stream.check_data_available(size):
errprint("File truncated")
return False
start = parser.stream.pos
r = True
r = r and _read_flags(parser, prototype)
r = r and _read_counts_and_sizes(parser, prototype)
r = r and _read_instructions(parser, prototype)
r = r and _read_constants(parser, prototype)
r = r and _read_debuginfo(parser, prototype)
end = parser.stream.pos
if r:
assert end - start == size, \
"Incorrectly read: from {0} to {1} ({2}) instead of {3}"\
.format(start, end, end - start, size)
return r
def _read_flags(parser, prototype):
bits = parser.stream.read_byte()
prototype.flags.has_ffi = bool(bits & FLAG_HAS_FFI)
bits &= ~FLAG_HAS_FFI
prototype.flags.has_iloop = bool(bits & FLAG_HAS_ILOOP)
bits &= ~FLAG_HAS_ILOOP
prototype.flags.has_jit = not (bits & FLAG_JIT_DISABLED)
bits &= ~FLAG_JIT_DISABLED
prototype.flags.has_sub_prototypes = bool(bits & FLAG_HAS_CHILD)
bits &= ~FLAG_HAS_CHILD
prototype.flags.is_variadic = bool(bits & FLAG_IS_VARIADIC)
bits &= ~FLAG_IS_VARIADIC
if bits != 0:
errprint("Unknown prototype flags: {0:08b}", bits)
return False
return True
def _read_counts_and_sizes(parser, prototype):
prototype.arguments_count = parser.stream.read_byte()
prototype.framesize = parser.stream.read_byte()
parser.upvalues_count = parser.stream.read_byte()
parser.complex_constants_count = parser.stream.read_uleb128()
parser.numeric_constants_count = parser.stream.read_uleb128()
parser.instructions_count = parser.stream.read_uleb128()
if parser.flags.is_stripped:
parser.debuginfo_size = 0
else:
parser.debuginfo_size = parser.stream.read_uleb128()
if parser.debuginfo_size == 0:
return True
prototype.first_line_number = parser.stream.read_uleb128()
prototype.lines_count = parser.stream.read_uleb128()
parser.lines_count = prototype.lines_count
return True
def _read_instructions(parser, prototype):
i = 0
if prototype.flags.is_variadic:
header = ins.FUNCV()
else:
header = ins.FUNCF()
header.A = prototype.framesize
prototype.instructions.append(header)
while i < parser.instructions_count:
instruction = ljd.rawdump.code.read(parser)
if not instruction:
return False
#print ("inst：%s" % instruction.name)
#print ("opcode:%x" % instruction.opcode)
#print ("A:%x" % instruction.A)
#print ("B:%x" % instruction.B)
#print ("CD:%x" % instruction.CD)
prototype.instructions.append(instruction)
i += 1
return True
def _read_constants(parser, prototype):
return ljd.rawdump.constants.read(parser, prototype.constants)
def _read_debuginfo(stream, prototype):
if stream.debuginfo_size == 0:
return True
return ljd.rawdump.debuginfo.read(stream,
prototype.first_line_number,
prototype.debuginfo)
================================================
FILE: ljd/util/__init__.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
================================================
FILE: ljd/util/binstream.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
import io
import os
import sys
class BinStream():
def __init__(self):
self.fd = None
self.size = 0
self.pos = 0
self.name = ""
self.data_byteorder = sys.byteorder
def open(self, filename):
self.name = filename
self.fd = io.open(filename, 'rb')
self.size = os.stat(filename).st_size
def close(self):
self.fd.close()
self.size = 0
self.pos = 0
def eof(self):
return self.pos >= self.size
def check_data_available(self, size=1):
return self.pos + size <= self.size
def read_bytes(self, size=1):
if not self.check_data_available(size):
raise IOError("Unexpected EOF while trying to read {0} bytes"
.format(size))
data = self.fd.read(size)
self.pos += len(data)#size
return data
def read_byte(self):
if not self.check_data_available(1):
raise IOError("Unexpected EOF while trying to read 1 byte")
data = self.fd.read(1)
self.pos += 1
return int.from_bytes(data,
byteorder=sys.byteorder,
signed=False)
def read_zstring(self):
string = b''
while not self.eof():
byte = self.read_bytes(1)
if byte == b'\x00':
return string
else:
string += byte
return string
def read_uleb128(self):
value = self.read_byte()
if value >= 0x80:
bitshift = 0
value &= 0x7f
while True:
byte = self.read_byte()
bitshift += 7
value |= (byte & 0x7f) << bitshift
if byte < 0x80:
break
return value
def read_uleb128_from33bit(self):
first_byte = self.read_byte()
is_number_bit = first_byte & 0x1
value = first_byte >> 1
if value >= 0x40:
bitshift = -1
value &= 0x3f
while True:
byte = self.read_byte()
bitshift += 7
value |= (byte & 0x7f) << bitshift
if byte < 0x80:
break
return is_number_bit, value
def read_uint(self, size=4):
value = self.read_bytes(size)
return int.from_bytes(value, byteorder=self.data_byteorder,
signed=False)
================================================
FILE: ljd/util/indentedstream.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
_TAB_WIDTH = " " * 8
class IndentedStream():
def __init__(self, fd):
self.fd = fd
self.indent = 0
self.line_open = False
def write_multiline(self, fmt, *args, **kargs):
assert not self.line_open
if len(args) + len(kargs) > 0:
text = fmt.format(*args, **kargs)
else:
text = fmt
lines = text.split("\n")
if lines[0] == "":
lines.pop(0)
if lines[-1] == "":
lines.pop(-1)
spaces = "\t" * self.indent
for line in lines:
self.fd.write(spaces + line + "\n")
def start_line(self):
assert not self.line_open
self.line_open = True
self.fd.write("\t" * self.indent)
def write(self, fmt="", *args, **kargs):
assert self.line_open
if len(args) + len(kargs) > 0:
text = fmt.format(*args, **kargs)
elif isinstance(fmt, str):
text = fmt
else:
text = str(fmt)
assert "\n" not in text
self.fd.write(text)
def end_line(self):
assert self.line_open
self.fd.write("\n")
self.line_open = False
def write_line(self, *args, **kargs):
self.start_line()
self.write(*args, **kargs)
self.end_line()
def open_block(self, *args, **kargs):
if len(args) + len(kargs) > 0:
self.write_line(*args, **kargs)
self.indent += 1
def close_block(self, *args, **kargs):
if len(args) + len(kargs) > 0:
self.write_line(*args, **kargs)
self.indent -= 1
================================================
FILE: ljd/util/log.py
================================================
#
# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py
#
import sys
def errprint(*args):
fmt = None
args = list(args)
if isinstance(args[0], str):
fmt = args.pop(0)
if fmt:
print(fmt.format(*args), file=sys.stderr)
else:
strs = [repr(x) for x in args]
print(" ".join(strs), file=sys.stderr)
================================================
FILE: main.py
================================================
#!/usr/bin/python3
#
# The MIT License (MIT)
#
# Copyright (c) 2013 Andrian Nord
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
#
import sys
import ljd.rawdump.parser
import ljd.pseudoasm.writer
import ljd.ast.builder
import ljd.ast.validator
import ljd.ast.locals
import ljd.ast.slotworks
import ljd.ast.unwarper
import ljd.ast.mutator
import ljd.lua.writer
#zzw 20180714 support str encode
import gconfig
def dump(name, obj, level=0):
indent = level * '\t'
if name is not None:
prefix = indent + name + " = "
else:
prefix = indent
if isinstance(obj, (int, float, str)):
print(prefix + str(obj))
elif isinstance(obj, list):
print (prefix + "[")
for value in obj:
dump(None, value, level + 1)
print (indent + "]")
elif isinstance(obj, dict):
print (prefix + "{")
for key, value in obj.items():
dump(key, value, level + 1)
print (indent + "}")
else:
print (prefix + obj.__class__.__name__)
for key in dir(obj):
if key.startswith("__"):
continue
val = getattr(obj, key)
dump(key, val, level + 1)
def main():
file_in = sys.argv[1]
header, prototype = ljd.rawdump.parser.parse(file_in)
#print ("good")
if not prototype:
return 1
# TODO: args
# ljd.pseudoasm.writer.write(sys.stdout, header, prototype)
ast = ljd.ast.builder.build(prototype)
assert ast is not None
ljd.ast.validator.validate(ast, warped=True)
ljd.ast.mutator.pre_pass(ast)
# ljd.ast.validator.validate(ast, warped=True)
ljd.ast.locals.mark_locals(ast)
# ljd.ast.validator.validate(ast, warped=True)
ljd.ast.slotworks.eliminate_temporary(ast)
# ljd.ast.validator.validate(ast, warped=True)
if True:
ljd.ast.unwarper.unwarp(ast)
# ljd.ast.validator.validate(ast, warped=False)
if True:
ljd.ast.locals.mark_local_definitions(ast)
# ljd.ast.validator.validate(ast, warped=False)
ljd.ast.mutator.primary_pass(ast)
ljd.ast.validator.validate(ast, warped=False)
ljd.lua.writer.write(sys.stdout, ast)
return 0
if __name__ == "__main__":
# zzw 20180714 support str encode
gconfig.gFlagDic['strEncode'] = 'utf-8'
retval = main()
sys.exit(retval)
# vim: ts=8 noexpandtab nosmarttab softtabstop=8 shiftwidth=8
================================================
FILE: test/breaks.lua
================================================
--[[
--]]
for i=1,2,3 do
if x and y then
print("Then")
else
break
end
end
for i=1,2,3 do
if x then
print("Then")
else
break
end
print ("Something")
if y then
break
end
end
for i=1,2,3 do
if x then
print("Then")
else
if y then
break
end
end
print ("Something")
if y then
break
end
end
for i=1,2,3 do
if x then
print("Then")
if y then
break
end
else
break
end
end
for i=1,2,3 do
if x then
print("Then")
if y then
print ("Nested then")
else
break
end
else
break
end
end
for i=1,2,3 do
if x then
print("Then")
else
break
end
if y then
print ("Y then")
else
break
end
end
print ("Too bad")
for i=1,2,3 do
if x then
print("Then")
if y then
print ("Y then")
if z then
print ("Z then")
end
end
break
end
if y then
print ("Y then")
else
break
end
end
for i=1,2,3 do
if x then
if y then
if z then
if xi then
print ("Xi then")
end
end
end
break
else
break
end
if y then
print ("Y then")
else
break
end
end
for i=1,2,3 do
if x and y and z and xi then
print ("Xi then")
break
else
break
end
if y then
print ("Y then")
else
break
end
end
for i=1,2,3 do
if x then
print ("X then")
if y then
print ("Y then")
if z then
print ("Z then")
if xi then
print ("Xi then")
end
end
end
break
else
break
end
if y then
print ("Y then")
else
break
end
end
for i=1,2,3 do
if x then
if z then
print("Z Then")
else
break
end
print ("Something")
if y then
print ("Y then")
else
break
end
else
break
end
end
for i=1,2,3 do
if x then
if z then
print("Z Then")
else
break
end
print ("Something")
else
break
end
if y then
print ("Y then")
else
break
end
end
function updateAnimation()
local currentTime = getElapsedTime()
local startSequence = private.currentSequence + 1
local endSequence = #private.sequence
for i=startSequence, endSequence, 1 do
local entry = private.sequence[i]
if entry.time <= currentTime then
if entry.command == "start" then
goToSlide(entry.element, 2)
goToTime(entry.element, entry.animationTime)
play(entry.element)
else
if entry.command == "step" then
goToSlide(entry.element, 2)
goToTime(entry.element, entry.animationTime)
else
pause(entry.element)
end
end
if entry.name ~= nil then
setAttribute(getElement("name", entry.element), "textstring", entry.name)
end
private.currentSequence = i
break
else
break
end
end
if #private.sequence == private.currentSequence then
private.playing = false
end
return
end
--[[
--]]
================================================
FILE: test/expression.lua
================================================
--[[
--]]
print ("true or true")
b = true or true
print ("false and false")
b = false and false
print ("false and or")
b = false and (x or y)
print ("false and ((and) or)")
b = false and ((x and z) or y)
print ("precalculated true expression")
c = true or (x and y) or true
print ("precalculated false expression")
d = false and ((x and y) or true)
print ("precalculated false expression with function")
e = error() and false and ((x and y) or true)
print ("precalculated true expression with function")
e = error() and true and ((x and y) or true)
print ("precalculated? false expression with variable")
local z = false
f = z and ((x and y) or true)
print ("precalculated false expression with nil")
f = nil and ((x and y) or true)
print ("simple or expression")
b = x or y
print ("simple or not expression")
b = not x or y
print ("simple and expression")
b = x and y
print ("simple or expression with binary comparison")
b = (x < 100) or y
print ("simple and expression with binary comparison")
b = (x < 100) and y
print ("simple and expression with binary comparison and function call")
b = (x < 100) and print(y)
print ("simple and expression with double binary comparison")
b = (x < 100) and (y > 100)
print ("(and) or expression")
b = (x and y) or z
print ("(or) and expression")
b = (x or y) and z
print ("(and) and expression")
b = (x and y) and z
print ("(or) or expression")
b = (x or y) or z
print ("or (and) expression")
b = x or (y and x)
b = x < 100 or (y < 100 and x < 100)
print ("and (or) expression")
b = x and (y or x)
print ("and (and) expression")
b = x and (y and x)
print ("or (or) expression")
b = x or (y or x)
print ("ond (or) and expression")
b = x and (y or x) and z
print ("or (and) or expression")
b = x or (y and x) or z
print ("and of two ors")
b = (x or z) and (y or z)
print ("or of two ands")
b = (x and z) or (y and z)
print ("x or string")
local xi = "nothing"
xi = x or "something"
local xi
print ("x and string")
xi = x and "something"
print ("and (or) and (or) expression with comparisons")
b = x < 100 and (y < 100 or x < 100) and (z < 100 or x < 100)
print ("and (or) and or or expression with comparisons")
b = x < 100 and (y < 100 or x < 100) and z < 100 or x < 100 or y < 100
print ("and (or) and and and expression with comparisons")
b = x < 100 and (y < 100 or x < 100) and z < 100 and x < 100 and y < 100
print ("or (and) or (and) expression with comparisons")
b = x < 100 or (y < 100 and x < 100) or (z < 100 and x < 100)
print ("and (and) and (and) expression with comparisons")
b = x < 100 and (y < 100 and x < 100) and (z < 100 and x < 100)
print ("or (or) or (or) expression with comparisons")
b = x < 100 or (y < 100 or x < 100) or (z < 100 or x < 100)
print ("4 and expression with comparisons")
b = x < 100 and y < 100 and x < 100 and z < 100 and x < 100
print ("4 or expression with comparisons")
b = x < 100 or y < 100 or x < 100 or z < 100 or x < 100
print ("and (or or) and (or or) expression with comparisons")
b = x < 100 and (y < 100 or x < 100 or z < 100)
and (y < 100 or x < 100 or z < 100)
print ("and (or and or) and (or and or) expression with comparisons")
b = x < 100 and (y < 100 or (x < 100 and x > 100) or z < 100)
and (y < 100 or (x <
gitextract_2i5rc5bq/
├── .gitignore
├── LICENSE
├── README.md
├── README.md.txt
├── gconfig.py
├── ljd/
│ ├── __init__.py
│ ├── ast/
│ │ ├── __init__.py
│ │ ├── builder.py
│ │ ├── helpers.py
│ │ ├── locals.py
│ │ ├── mutator.py
│ │ ├── nodes.py
│ │ ├── slotworks.py
│ │ ├── traverse.py
│ │ ├── unwarper.py
│ │ └── validator.py
│ ├── bytecode/
│ │ ├── __init__.py
│ │ ├── constants.py
│ │ ├── debuginfo.py
│ │ ├── helpers.py
│ │ ├── instructions.py
│ │ └── prototype.py
│ ├── lua/
│ │ ├── __init__.py
│ │ └── writer.py
│ ├── pseudoasm/
│ │ ├── __init__.py
│ │ ├── constants.py
│ │ ├── instructions.py
│ │ ├── prototype.py
│ │ └── writer.py
│ ├── rawdump/
│ │ ├── __init__.py
│ │ ├── code.py
│ │ ├── constants.py
│ │ ├── debuginfo.py
│ │ ├── header.py
│ │ ├── parser.py
│ │ └── prototype.py
│ └── util/
│ ├── __init__.py
│ ├── binstream.py
│ ├── indentedstream.py
│ └── log.py
├── main.py
└── test/
├── breaks.lua
├── expression.lua
├── ifs.lua
├── loop.lua
└── primitive.lua
SYMBOL INDEX (584 symbols across 29 files)
FILE: ljd/ast/builder.py
class _State (line 13) | class _State():
method __init__ (line 14) | def __init__(self):
method _warp_in_block (line 21) | def _warp_in_block(self, addr):
function build (line 29) | def build(prototype):
function _build_function_definition (line 33) | def _build_function_definition(prototype):
function _build_function_arguments (line 57) | def _build_function_arguments(state, prototype):
function _build_function_blocks (line 72) | def _build_function_blocks(state, instructions):
function _blockenize (line 116) | def _blockenize(state, instructions):
function _establish_warps (line 167) | def _establish_warps(state, instructions):
function _build_warp (line 190) | def _build_warp(state, last_addr, instructions):
function _build_jump_warp (line 207) | def _build_jump_warp(state, last_addr, instructions):
function _build_conditional_warp (line 218) | def _build_conditional_warp(state, last_addr, instructions):
function _build_unconditional_warp (line 255) | def _build_unconditional_warp(state, addr, instruction):
function _build_iterator_warp (line 273) | def _build_iterator_warp(state, last_addr, instructions):
function _build_numeric_loop_warp (line 308) | def _build_numeric_loop_warp(state, addr, instruction):
function _build_flow_warp (line 328) | def _build_flow_warp(state, addr, instruction):
function _build_statement (line 339) | def _build_statement(state, addr, instruction):
function _build_var_assignment (line 386) | def _build_var_assignment(state, addr, instruction):
function _build_knil (line 449) | def _build_knil(state, addr, instruction):
function _build_global_assignment (line 457) | def _build_global_assignment(state, addr, instruction):
function _build_table_assignment (line 469) | def _build_table_assignment(state, addr, instruction):
function _build_table_mass_assignment (line 481) | def _build_table_mass_assignment(state, addr, instruction):
function _build_call (line 496) | def _build_call(state, addr, instruction):
function _build_vararg (line 522) | def _build_vararg(state, addr, instruction):
function _build_return (line 537) | def _build_return(state, addr, instruction):
function _build_call_arguments (line 560) | def _build_call_arguments(state, addr, instruction):
function _build_range_assignment (line 585) | def _build_range_assignment(state, addr, from_slot, to_slot):
function _build_binary_expression (line 611) | def _build_binary_expression(state, addr, instruction):
function _build_concat_expression (line 639) | def _build_concat_expression(state, addr, instruction):
function _build_const_expression (line 663) | def _build_const_expression(state, addr, instruction):
function _build_table_element (line 686) | def _build_table_element(state, addr, instruction):
function _build_function (line 698) | def _build_function(state, slot):
function _build_table_copy (line 704) | def _build_table_copy(state, slot):
function _build_table_record_item (line 729) | def _build_table_record_item(value):
function _build_comparison_expression (line 777) | def _build_comparison_expression(state, addr, instruction):
function _build_unary_expression (line 799) | def _build_unary_expression(state, addr, instruction):
function _build_slot (line 826) | def _build_slot(state, addr, slot):
function _build_upvalue (line 830) | def _build_upvalue(state, addr, slot):
function _build_identifier (line 834) | def _build_identifier(state, addr, slot, want_type):
function _build_global_variable (line 851) | def _build_global_variable(state, addr, slot):
function _build_string_constant (line 862) | def _build_string_constant(state, index):
function _build_cdata_constant (line 870) | def _build_cdata_constant(state, index):
function _build_numeric_constant (line 878) | def _build_numeric_constant(state, index):
function _build_primitive (line 892) | def _build_primitive(state, value):
function _build_literal (line 907) | def _build_literal(state, value):
FILE: ljd/ast/helpers.py
function insert_table_record (line 5) | def insert_table_record(constructor, key, value):
function has_same_table (line 57) | def has_same_table(node, table):
function is_equal (line 81) | def is_equal(a, b):
FILE: ljd/ast/locals.py
function mark_locals (line 10) | def mark_locals(ast):
function mark_local_definitions (line 14) | def mark_local_definitions(ast):
class _LocalsMarker (line 18) | class _LocalsMarker(traverse.Visitor):
class _State (line 19) | class _State():
method __init__ (line 20) | def __init__(self):
method __init__ (line 25) | def __init__(self):
method _push_state (line 30) | def _push_state(self):
method _pop_state (line 33) | def _pop_state(self):
method _state (line 36) | def _state(self):
method _process_slots (line 39) | def _process_slots(self, addr):
method _reset_slot (line 65) | def _reset_slot(self, slot):
method _reset_all (line 68) | def _reset_all(self, slots):
method visit_function_definition (line 75) | def visit_function_definition(self, node):
method leave_function_definition (line 79) | def leave_function_definition(self, node):
method visit_variables_list (line 87) | def visit_variables_list(self, node):
method visit_identifiers_list (line 92) | def visit_identifiers_list(self, node):
method visit_numeric_loop_warp (line 95) | def visit_numeric_loop_warp(self, node):
method visit_identifier (line 98) | def visit_identifier(self, node):
method _process_worthy_node (line 107) | def _process_worthy_node(self, node):
method _leave_node (line 119) | def _leave_node(self, handler, node):
method _visit_node (line 124) | def _visit_node(self, handler, node):
class _LocalDefinitionsMarker (line 130) | class _LocalDefinitionsMarker(traverse.Visitor):
class _State (line 131) | class _State():
method __init__ (line 132) | def __init__(self):
method __init__ (line 136) | def __init__(self):
method _push_state (line 139) | def _push_state(self):
method _pop_state (line 142) | def _pop_state(self):
method _state (line 145) | def _state(self):
method _update_known_locals (line 148) | def _update_known_locals(self, local, addr):
method visit_function_definition (line 163) | def visit_function_definition(self, node):
method leave_function_definition (line 170) | def leave_function_definition(self, node):
method visit_iterator_for (line 173) | def visit_iterator_for(self, node):
method visit_numeric_for (line 180) | def visit_numeric_for(self, node):
method visit_assignment (line 188) | def visit_assignment(self, node):
method _visit (line 214) | def _visit(self, node):
FILE: ljd/ast/mutator.py
class SimpleLoopWarpSwapper (line 10) | class SimpleLoopWarpSwapper(traverse.Visitor):
method visit_statements_list (line 11) | def visit_statements_list(self, node):
method _create_dummy_block (line 59) | def _create_dummy_block(self, block, slot):
method _fix_uclo_return (line 85) | def _fix_uclo_return(self, block, next_block):
method _swap_iterator_warps (line 104) | def _swap_iterator_warps(self, blocks, end):
method _swap_numeric_loop_warps (line 130) | def _swap_numeric_loop_warps(self, blocks, end):
class MutatorVisitor (line 158) | class MutatorVisitor(traverse.Visitor):
method leave_if (line 161) | def leave_if(self, node):
method visit_statements_list (line 178) | def visit_statements_list(self, node):
method _fill_constructor (line 205) | def _fill_constructor(self, table, constructor, statements):
function pre_pass (line 236) | def pre_pass(ast):
function primary_pass (line 242) | def primary_pass(ast):
FILE: ljd/ast/nodes.py
class FunctionDefinition (line 9) | class FunctionDefinition():
method __init__ (line 10) | def __init__(self):
method _accept (line 18) | def _accept(self, visitor):
class TableConstructor (line 27) | class TableConstructor():
method __init__ (line 28) | def __init__(self):
method _accept (line 32) | def _accept(self, visitor):
class ArrayRecord (line 41) | class ArrayRecord():
method __init__ (line 42) | def __init__(self):
method _accept (line 45) | def _accept(self, visitor):
class TableRecord (line 53) | class TableRecord():
method __init__ (line 54) | def __init__(self):
method _accept (line 58) | def _accept(self, visitor):
class Assignment (line 67) | class Assignment():
method __init__ (line 71) | def __init__(self):
method _accept (line 76) | def _accept(self, visitor):
class BinaryOperator (line 85) | class BinaryOperator():
method __init__ (line 108) | def __init__(self):
method _accept (line 113) | def _accept(self, visitor):
class UnaryOperator (line 122) | class UnaryOperator():
method __init__ (line 127) | def __init__(self):
method _accept (line 131) | def _accept(self, visitor):
class StatementsList (line 139) | class StatementsList():
method __init__ (line 140) | def __init__(self):
method _accept (line 143) | def _accept(self, visitor):
class IdentifiersList (line 151) | class IdentifiersList():
method __init__ (line 152) | def __init__(self):
method _accept (line 155) | def _accept(self, visitor):
class RecordsList (line 163) | class RecordsList():
method __init__ (line 164) | def __init__(self):
method _accept (line 167) | def _accept(self, visitor):
class VariablesList (line 175) | class VariablesList():
method __init__ (line 176) | def __init__(self):
method _accept (line 179) | def _accept(self, visitor):
class ExpressionsList (line 187) | class ExpressionsList():
method __init__ (line 188) | def __init__(self):
method _accept (line 191) | def _accept(self, visitor):
class Identifier (line 200) | class Identifier():
method __init__ (line 206) | def __init__(self):
method _accept (line 211) | def _accept(self, visitor):
class MULTRES (line 218) | class MULTRES():
method _accept (line 219) | def _accept(self, visitor):
class TableElement (line 224) | class TableElement():
method __init__ (line 225) | def __init__(self):
method _accept (line 229) | def _accept(self, visitor):
class Vararg (line 238) | class Vararg():
method _accept (line 239) | def _accept(self, visitor):
class FunctionCall (line 244) | class FunctionCall():
method __init__ (line 245) | def __init__(self):
method _accept (line 249) | def _accept(self, visitor):
class If (line 258) | class If():
method __init__ (line 259) | def __init__(self):
method _accept (line 265) | def _accept(self, visitor):
class ElseIf (line 278) | class ElseIf():
method __init__ (line 279) | def __init__(self):
method _accept (line 283) | def _accept(self, visitor):
class Block (line 294) | class Block():
method __init__ (line 295) | def __init__(self):
method _accept (line 303) | def _accept(self, visitor):
class UnconditionalWarp (line 312) | class UnconditionalWarp():
method __init__ (line 316) | def __init__(self):
method _accept (line 321) | def _accept(self, visitor):
class ConditionalWarp (line 329) | class ConditionalWarp():
method __init__ (line 330) | def __init__(self):
method _accept (line 335) | def _accept(self, visitor):
class IteratorWarp (line 345) | class IteratorWarp():
method __init__ (line 346) | def __init__(self):
method _accept (line 352) | def _accept(self, visitor):
class NumericLoopWarp (line 363) | class NumericLoopWarp():
method __init__ (line 364) | def __init__(self):
method _accept (line 370) | def _accept(self, visitor):
class EndWarp (line 381) | class EndWarp():
method _accept (line 382) | def _accept(self, visitor):
class Return (line 390) | class Return():
method __init__ (line 391) | def __init__(self):
method _accept (line 394) | def _accept(self, visitor):
class Break (line 402) | class Break():
method _accept (line 403) | def _accept(self, visitor):
class While (line 408) | class While():
method __init__ (line 409) | def __init__(self):
method _accept (line 413) | def _accept(self, visitor):
class RepeatUntil (line 422) | class RepeatUntil():
method __init__ (line 423) | def __init__(self):
method _accept (line 427) | def _accept(self, visitor):
class NumericFor (line 436) | class NumericFor():
method __init__ (line 437) | def __init__(self):
method _accept (line 442) | def _accept(self, visitor):
class IteratorFor (line 452) | class IteratorFor():
method __init__ (line 453) | def __init__(self):
method _accept (line 458) | def _accept(self, visitor):
class Constant (line 468) | class Constant():
method __init__ (line 474) | def __init__(self):
method _accept (line 478) | def _accept(self, visitor):
class Primitive (line 483) | class Primitive():
method __init__ (line 488) | def __init__(self):
method _accept (line 491) | def _accept(self, visitor):
FILE: ljd/ast/slotworks.py
function eliminate_temporary (line 10) | def eliminate_temporary(ast):
function _eliminate_temporary (line 23) | def _eliminate_temporary(slots):
function _fill_massive_refs (line 56) | def _fill_massive_refs(info, simple, massive, iterators):
function _fill_simple_refs (line 90) | def _fill_simple_refs(info, simple, tables):
function _get_holder (line 128) | def _get_holder(path):
function _eliminate_simple_cases (line 136) | def _eliminate_simple_cases(simple):
function _eliminate_into_table_constructors (line 156) | def _eliminate_into_table_constructors(tables):
function _eliminate_mass_assignments (line 174) | def _eliminate_mass_assignments(massive):
function _replace_node (line 184) | def _replace_node(holder, original, replacement):
function _replace_node_in_list (line 193) | def _replace_node_in_list(nodes, original, replacement):
function _eliminate_iterators (line 203) | def _eliminate_iterators(iterators):
function _mark_invalidated (line 222) | def _mark_invalidated(node):
function _is_invalidated (line 226) | def _is_invalidated(node):
function _remove_unused (line 230) | def _remove_unused(unused):
function _collect_slots (line 234) | def _collect_slots(ast):
function _eliminate_multres (line 241) | def _eliminate_multres(ast):
class _MultresEliminator (line 246) | class _MultresEliminator(traverse.Visitor):
method __init__ (line 247) | def __init__(self):
method leave_assignment (line 250) | def leave_assignment(self, node):
method _process_multres_in_list (line 277) | def _process_multres_in_list(self, nodes_list):
method visit_function_call (line 289) | def visit_function_call(self, node):
method visit_return (line 292) | def visit_return(self, node):
class _SlotReference (line 296) | class _SlotReference():
method __init__ (line 297) | def __init__(self):
class _SlotInfo (line 302) | class _SlotInfo():
method __init__ (line 303) | def __init__(self):
class _SlotsCollector (line 313) | class _SlotsCollector(traverse.Visitor):
class _State (line 314) | class _State():
method __init__ (line 315) | def __init__(self):
method __init__ (line 321) | def __init__(self):
method _state (line 333) | def _state(self):
method _push_state (line 336) | def _push_state(self):
method _pop_state (line 339) | def _pop_state(self):
method _commit_info (line 342) | def _commit_info(self, info):
method _commit_slot (line 350) | def _commit_slot(self, slot, node):
method _register_slot (line 362) | def _register_slot(self, slot, node):
method _register_all_slots (line 372) | def _register_all_slots(self, node, slots):
method _commit_all_slots (line 382) | def _commit_all_slots(self, slots, node):
method _register_slot_reference (line 389) | def _register_slot_reference(self, slot, node):
method visit_assignment (line 405) | def visit_assignment(self, node):
method leave_assignment (line 411) | def leave_assignment(self, node):
method visit_identifier (line 414) | def visit_identifier(self, node):
method visit_function_definition (line 420) | def visit_function_definition(self, node):
method leave_function_definition (line 424) | def leave_function_definition(self, node):
method leave_block (line 427) | def leave_block(self, node):
method visit_iterator_warp (line 433) | def visit_iterator_warp(self, node):
method visit_numeric_loop_warp (line 436) | def visit_numeric_loop_warp(self, node):
method _visit_node (line 441) | def _visit_node(self, handler, node):
method _leave_node (line 446) | def _leave_node(self, handler, node):
method _visit (line 451) | def _visit(self, node):
function _cleanup_invalid_nodes (line 458) | def _cleanup_invalid_nodes(ast):
class _TreeCleanup (line 462) | class _TreeCleanup(traverse.Visitor):
method visit_block (line 463) | def visit_block(self, node):
FILE: ljd/ast/traverse.py
class Visitor (line 1) | class Visitor():
method __init__ (line 2) | def __init__(self):
method visit_function_definition (line 7) | def visit_function_definition(self, node):
method leave_function_definition (line 10) | def leave_function_definition(self, node):
method visit_table_constructor (line 15) | def visit_table_constructor(self, node):
method leave_table_constructor (line 18) | def leave_table_constructor(self, node):
method visit_table_record (line 21) | def visit_table_record(self, node):
method leave_table_record (line 24) | def leave_table_record(self, node):
method visit_array_record (line 27) | def visit_array_record(self, node):
method leave_array_record (line 30) | def leave_array_record(self, node):
method visit_assignment (line 35) | def visit_assignment(self, node):
method leave_assignment (line 38) | def leave_assignment(self, node):
method visit_binary_operator (line 43) | def visit_binary_operator(self, node):
method leave_binary_operator (line 46) | def leave_binary_operator(self, node):
method visit_unary_operator (line 49) | def visit_unary_operator(self, node):
method leave_unary_operator (line 52) | def leave_unary_operator(self, node):
method visit_statements_list (line 57) | def visit_statements_list(self, node):
method leave_statements_list (line 60) | def leave_statements_list(self, node):
method visit_identifiers_list (line 63) | def visit_identifiers_list(self, node):
method leave_identifiers_list (line 66) | def leave_identifiers_list(self, node):
method visit_records_list (line 69) | def visit_records_list(self, node):
method leave_records_list (line 72) | def leave_records_list(self, node):
method visit_variables_list (line 75) | def visit_variables_list(self, node):
method leave_variables_list (line 78) | def leave_variables_list(self, node):
method visit_expressions_list (line 81) | def visit_expressions_list(self, node):
method leave_expressions_list (line 84) | def leave_expressions_list(self, node):
method visit_identifier (line 89) | def visit_identifier(self, node):
method leave_identifier (line 92) | def leave_identifier(self, node):
method visit_multres (line 95) | def visit_multres(self, node):
method leave_multres (line 98) | def leave_multres(self, node):
method visit_table_element (line 101) | def visit_table_element(self, node):
method leave_table_element (line 104) | def leave_table_element(self, node):
method visit_vararg (line 107) | def visit_vararg(self, node):
method leave_vararg (line 110) | def leave_vararg(self, node):
method visit_function_call (line 113) | def visit_function_call(self, node):
method leave_function_call (line 116) | def leave_function_call(self, node):
method visit_if (line 121) | def visit_if(self, node):
method leave_if (line 124) | def leave_if(self, node):
method visit_elseif (line 127) | def visit_elseif(self, node):
method leave_elseif (line 130) | def leave_elseif(self, node):
method visit_block (line 135) | def visit_block(self, node):
method leave_block (line 138) | def leave_block(self, node):
method visit_unconditional_warp (line 141) | def visit_unconditional_warp(self, node):
method leave_unconditional_warp (line 144) | def leave_unconditional_warp(self, node):
method visit_conditional_warp (line 147) | def visit_conditional_warp(self, node):
method leave_conditional_warp (line 150) | def leave_conditional_warp(self, node):
method visit_iterator_warp (line 153) | def visit_iterator_warp(self, node):
method leave_iterator_warp (line 156) | def leave_iterator_warp(self, node):
method visit_numeric_loop_warp (line 159) | def visit_numeric_loop_warp(self, node):
method leave_numeric_loop_warp (line 162) | def leave_numeric_loop_warp(self, node):
method visit_end_warp (line 165) | def visit_end_warp(self, node):
method leave_end_warp (line 168) | def leave_end_warp(self, node):
method visit_return (line 173) | def visit_return(self, node):
method leave_return (line 176) | def leave_return(self, node):
method visit_break (line 179) | def visit_break(self, node):
method leave_break (line 182) | def leave_break(self, node):
method visit_while (line 187) | def visit_while(self, node):
method leave_while (line 190) | def leave_while(self, node):
method visit_repeat_until (line 193) | def visit_repeat_until(self, node):
method leave_repeat_until (line 196) | def leave_repeat_until(self, node):
method visit_numeric_for (line 199) | def visit_numeric_for(self, node):
method leave_numeric_for (line 202) | def leave_numeric_for(self, node):
method visit_iterator_for (line 205) | def visit_iterator_for(self, node):
method leave_iterator_for (line 208) | def leave_iterator_for(self, node):
method visit_constant (line 213) | def visit_constant(self, node):
method leave_constant (line 216) | def leave_constant(self, node):
method visit_primitive (line 219) | def visit_primitive(self, node):
method leave_primitive (line 222) | def leave_primitive(self, node):
method _visit_node (line 227) | def _visit_node(self, handler, node):
method _leave_node (line 230) | def _leave_node(self, handler, node):
method _visit (line 235) | def _visit(self, node):
method _visit_list (line 240) | def _visit_list(self, nodes_list):
function traverse (line 247) | def traverse(visitor, node):
FILE: ljd/ast/unwarper.py
class _StatementsCollector (line 20) | class _StatementsCollector(traverse.Visitor):
method __init__ (line 21) | def __init__(self):
method visit_statements_list (line 24) | def visit_statements_list(self, node):
function unwarp (line 29) | def unwarp(node):
function _run_step (line 41) | def _run_step(step, node, **kargs):
function _gather_statements_lists (line 51) | def _gather_statements_lists(node):
function _glue_flows (line 57) | def _glue_flows(node):
function _unwarp_expressions (line 83) | def _unwarp_expressions(blocks):
function _find_endest_end (line 129) | def _find_endest_end(expressions):
function _unwarp_ifs (line 139) | def _unwarp_ifs(blocks, top_end=None, topmost_end=None):
function _extract_if_body (line 176) | def _extract_if_body(start_index, blocks, topmost_end):
function _unwarp_expressions_pack (line 192) | def _unwarp_expressions_pack(blocks, pack):
function _split_by_slot_use (line 244) | def _split_by_slot_use(statements, min_i, warp, slot):
function _extract_destination_slots (line 275) | def _extract_destination_slots(statement):
function _gather_slots (line 288) | def _gather_slots(node):
function _find_expressions (line 304) | def _find_expressions(start, body, end):
function _find_subexpressions (line 447) | def _find_subexpressions(start, body):
function _get_simple_local_assignment_slot (line 460) | def _get_simple_local_assignment_slot(start, body, end):
function _find_expression_slot (line 473) | def _find_expression_slot(body):
function _unwarp_logical_expression (line 486) | def _unwarp_logical_expression(start, end, body):
function _compile_expression (line 504) | def _compile_expression(body, end, true, false):
function _unwarp_expression (line 570) | def _unwarp_expression(body, end, true, false):
function _get_target (line 691) | def _get_target(warp, allow_end=False):
function _set_target (line 702) | def _set_target(warp, target):
function _get_operator (line 710) | def _get_operator(block, true, end):
function _get_last_assignment_source (line 740) | def _get_last_assignment_source(block):
function _get_and_remove_last_assignment_source (line 749) | def _get_and_remove_last_assignment_source(block):
function _compile_subexpression (line 755) | def _compile_subexpression(subexpression, operator,
function _is_inverted (line 785) | def _is_inverted(warp, true, end):
function _invert (line 813) | def _invert(expression):
function _get_terminators (line 836) | def _get_terminators(body):
function _assemble_expression (line 868) | def _assemble_expression(parts):
function _make_explicit_subexpressions (line 901) | def _make_explicit_subexpressions(parts):
function _unwarp_if_statement (line 936) | def _unwarp_if_statement(start, body, end, topmost_end):
function _extract_if_expression (line 989) | def _extract_if_expression(start, body, end, topmost_end):
function _search_expression_end (line 1034) | def _search_expression_end(expression, falses):
function _find_branching_end (line 1055) | def _find_branching_end(blocks, topmost_end):
function _remove_processed_blocks (line 1076) | def _remove_processed_blocks(blocks, boundaries):
function _unwarp_loops (line 1099) | def _unwarp_loops(blocks, repeat_until):
function _cleanup_breaks_and_if_ends (line 1139) | def _cleanup_breaks_and_if_ends(loops, blocks):
function _replace_targets (line 1200) | def _replace_targets(blocks, original, replacement):
function _unwarp_loop (line 1223) | def _unwarp_loop(start, end, body):
function _create_next_block (line 1356) | def _create_next_block(original):
function _set_flow_to (line 1366) | def _set_flow_to(block, target):
function _set_end (line 1372) | def _set_end(block):
function _is_flow (line 1383) | def _is_flow(warp):
function _is_jump (line 1388) | def _is_jump(warp):
function _fix_nested_ifs (line 1393) | def _fix_nested_ifs(blocks, start):
function _fix_expression (line 1413) | def _fix_expression(blocks, start, end):
function _gather_possible_ends (line 1424) | def _gather_possible_ends(block):
function _unwarp_breaks (line 1442) | def _unwarp_breaks(start, blocks, next_block):
function _find_all_loops (line 1556) | def _find_all_loops(blocks, repeat_until):
function _get_previous_block (line 1628) | def _get_previous_block(block, blocks):
FILE: ljd/ast/validator.py
class TypeRestriction (line 9) | class TypeRestriction():
method __init__ (line 10) | def __init__(self, default, specific):
method check (line 18) | def check(self, node):
class Visitor (line 70) | class Visitor(traverse.Visitor):
method __init__ (line 71) | def __init__(self, warped=True):
method _set_restrictions (line 76) | def _set_restrictions(self, default, specific={}):
method visit_function_definition (line 81) | def visit_function_definition(self, node):
method visit_table_constructor (line 89) | def visit_table_constructor(self, node):
method visit_array_record (line 92) | def visit_array_record(self, node):
method visit_table_record (line 95) | def visit_table_record(self, node):
method visit_assignment (line 100) | def visit_assignment(self, node):
method visit_binary_operator (line 115) | def visit_binary_operator(self, node):
method visit_unary_operator (line 140) | def visit_unary_operator(self, node):
method visit_statements_list (line 149) | def visit_statements_list(self, node):
method visit_identifiers_list (line 157) | def visit_identifiers_list(self, node):
method visit_records_list (line 161) | def visit_records_list(self, node):
method visit_variables_list (line 178) | def visit_variables_list(self, node):
method visit_expressions_list (line 181) | def visit_expressions_list(self, node):
method visit_identifier (line 186) | def visit_identifier(self, node):
method visit_table_element (line 195) | def visit_table_element(self, node):
method visit_function_call (line 198) | def visit_function_call(self, node):
method visit_if (line 206) | def visit_if(self, node):
method visit_elseif (line 213) | def visit_elseif(self, node):
method visit_block (line 221) | def visit_block(self, node):
method visit_unconditional_warp (line 234) | def visit_unconditional_warp(self, node):
method visit_conditional_warp (line 240) | def visit_conditional_warp(self, node):
method visit_iterator_warp (line 253) | def visit_iterator_warp(self, node):
method visit_numeric_loop_warp (line 264) | def visit_numeric_loop_warp(self, node):
method visit_return (line 275) | def visit_return(self, node):
method visit_while (line 280) | def visit_while(self, node):
method visit_repeat_until (line 286) | def visit_repeat_until(self, node):
method visit_numeric_for (line 292) | def visit_numeric_for(self, node):
method visit_iterator_for (line 299) | def visit_iterator_for(self, node):
method visit_constant (line 308) | def visit_constant(self, node):
method visit_primitive (line 314) | def visit_primitive(self, node):
method _visit (line 321) | def _visit(self, node):
function validate (line 336) | def validate(ast, warped=True):
FILE: ljd/bytecode/constants.py
class Table (line 10) | class Table():
method __init__ (line 11) | def __init__(self):
class Constants (line 19) | class Constants():
method __init__ (line 20) | def __init__(self):
FILE: ljd/bytecode/debuginfo.py
class VariableInfo (line 6) | class VariableInfo():
method __init__ (line 10) | def __init__(self):
class DebugInformation (line 17) | class DebugInformation():
method __init__ (line 18) | def __init__(self):
method lookup_line_number (line 23) | def lookup_line_number(self, addr):
method lookup_local_name (line 29) | def lookup_local_name(self, addr, slot):
method lookup_upvalue_name (line 42) | def lookup_upvalue_name(self, slot):
FILE: ljd/bytecode/helpers.py
function get_jump_destination (line 1) | def get_jump_destination(addr, instruction):
function set_jump_destination (line 5) | def set_jump_destination(addr, instruction, value):
FILE: ljd/bytecode/instructions.py
class _Instruction (line 46) | class _Instruction():
method __init__ (line 47) | def __init__(self, definition):
class _IDef (line 61) | class _IDef():
method __init__ (line 64) | def __init__(self, name, A_type, B_type, CD_type, description):
method __call__ (line 78) | def __call__(self):
FILE: ljd/bytecode/prototype.py
class Flags (line 9) | class Flags():
method __init (line 10) | def __init(self):
class Prototype (line 18) | class Prototype():
method __init__ (line 19) | def __init__(self):
FILE: ljd/lua/writer.py
class _State (line 40) | class _State():
method __init__ (line 41) | def __init__(self):
class Visitor (line 47) | class Visitor(traverse.Visitor):
method __init__ (line 48) | def __init__(self):
method _start_statement (line 58) | def _start_statement(self, statement):
method _end_statement (line 63) | def _end_statement(self, statement):
method _end_line (line 68) | def _end_line(self):
method _start_block (line 71) | def _start_block(self):
method _end_block (line 74) | def _end_block(self):
method _write (line 77) | def _write(self, fmt, *args, **kargs):
method _state (line 80) | def _state(self):
method _push_state (line 83) | def _push_state(self):
method _pop_state (line 86) | def _pop_state(self):
method visit_function_definition (line 91) | def visit_function_definition(self, node):
method visit_table_constructor (line 125) | def visit_table_constructor(self, node):
method visit_table_record (line 166) | def visit_table_record(self, node):
method visit_assignment (line 186) | def visit_assignment(self, node):
method _is_variable (line 223) | def _is_variable(self, node):
method _is_global (line 229) | def _is_global(self, node):
method _is_builtin (line 235) | def _is_builtin(self, node):
method _is_valid_name (line 241) | def _is_valid_name(self, key):
method visit_binary_operator (line 249) | def visit_binary_operator(self, node):
method visit_unary_operator (line 335) | def visit_unary_operator(self, node):
method visit_statements_list (line 356) | def visit_statements_list(self, node):
method leave_statements_list (line 362) | def leave_statements_list(self, node):
method _visit_comma_separated_list (line 368) | def _visit_comma_separated_list(self, node):
method visit_records_list (line 380) | def visit_records_list(self, node):
method visit_identifier (line 398) | def visit_identifier(self, node):
method visit_multres (line 404) | def visit_multres(self, node):
method visit_table_element (line 407) | def visit_table_element(self, node):
method visit_vararg (line 446) | def visit_vararg(self, node):
method visit_function_call (line 449) | def visit_function_call(self, node):
method visit_if (line 494) | def visit_if(self, node):
method visit_elseif (line 522) | def visit_elseif(self, node):
method visit_block (line 535) | def visit_block(self, node):
method visit_unconditional_warp (line 555) | def visit_unconditional_warp(self, node):
method visit_conditional_warp (line 565) | def visit_conditional_warp(self, node):
method visit_iterator_warp (line 590) | def visit_iterator_warp(self, node):
method visit_numeric_loop_warp (line 607) | def visit_numeric_loop_warp(self, node):
method visit_return (line 624) | def visit_return(self, node):
method visit_break (line 633) | def visit_break(self, node):
method visit_while (line 642) | def visit_while(self, node):
method visit_repeat_until (line 656) | def visit_repeat_until(self, node):
method visit_numeric_for (line 669) | def visit_numeric_for(self, node):
method visit_iterator_for (line 687) | def visit_iterator_for(self, node):
method visit_constant (line 705) | def visit_constant(self, node):
method visit_primitive (line 731) | def visit_primitive(self, node):
method _skip (line 739) | def _skip(self, node):
method _visit (line 742) | def _visit(self, node):
function write (line 760) | def write(fd, ast):
function _get_next_significant (line 770) | def _get_next_significant(queue, i):
function _process_queue (line 787) | def _process_queue(fd, queue):
FILE: ljd/pseudoasm/constants.py
function write_tables (line 8) | def write_tables(writer, prototype):
function _write_table (line 18) | def _write_table(writer, index, table):
function _translate_element (line 42) | def _translate_element(element):
FILE: ljd/pseudoasm/instructions.py
class _State (line 16) | class _State():
method __init__ (line 17) | def __init__(self, writer, prototype, instructions):
function write (line 25) | def write(writer, prototype):
function _write_instruction (line 47) | def _write_instruction(writer, addr, line, instruction):
function _write_function (line 61) | def _write_function(writer, addr, line, instruction):
function _translate_description (line 83) | def _translate_description(writer, addr, line, instruction):
function _translate (line 92) | def _translate(writer, addr, value, attr_type):
function _lookup_variable_name (line 134) | def _lookup_variable_name(writer, addr, slot):
function _lookup_variable_name_step (line 146) | def _lookup_variable_name_step(writer, addr, slot):
function _translate_standard (line 207) | def _translate_standard(writer, addr, line, instruction):
function _translate_normal (line 224) | def _translate_normal(writer, description, addr, line, instruction):
function _translate_concat (line 230) | def _translate_concat(writer, description, addr, line, instruction):
function _translate_nil (line 246) | def _translate_nil(writer, description, addr, line, instruction):
function _translate_table_str_op (line 260) | def _translate_table_str_op(writer, description, addr, line, instruction):
function _translate_new_table (line 268) | def _translate_new_table(writer, description, addr, line, instruction):
function _translate_mass_set (line 283) | def _translate_mass_set(writer, description, addr, line, instruction):
function _translate_varg_call (line 297) | def _translate_varg_call(writer, description, addr, line, instruction):
function _translate_call (line 324) | def _translate_call(writer, description, addr, line, instruction):
function _translate_varg_tailcall (line 351) | def _translate_varg_tailcall(writer, description, addr, line, instruction):
function _translate_tailcall (line 370) | def _translate_tailcall(writer, description, addr, line, instruction):
function _translate_iterator (line 389) | def _translate_iterator(writer, description, addr, line, instruction):
function _translate_vararg (line 420) | def _translate_vararg(writer, description, addr, line, instruction):
function _translate_return_mult (line 440) | def _translate_return_mult(writer, description, addr, line, instruction):
function _translate_return_many (line 457) | def _translate_return_many(writer, description, addr, line, instruction):
function _translate_return_one (line 474) | def _translate_return_one(writer, description, addr, line, instruction):
function _translate_for_init (line 480) | def _translate_for_init(writer, description, addr, line, instruction):
function _translate_numeric_loop (line 495) | def _translate_numeric_loop(writer, description, addr, line, instruction):
function _translate_iter_loop (line 511) | def _translate_iter_loop(writer, description, addr, line, instruction):
function _init (line 677) | def _init():
FILE: ljd/pseudoasm/prototype.py
function write (line 9) | def write(writer, prototype):
function _write_header (line 16) | def _write_header(writer, prototype):
function format_header (line 20) | def format_header(writer, prototype):
function write_body (line 33) | def write_body(writer, prototype):
FILE: ljd/pseudoasm/writer.py
class _State (line 10) | class _State():
method __init__ (line 11) | def __init__(self):
function write (line 17) | def write(fd, header, prototype):
function _write_header (line 29) | def _write_header(writer, header):
FILE: ljd/rawdump/code.py
function read (line 173) | def read(parser):
function _set_instruction_operands (line 196) | def _set_instruction_operands(parser, codeword, instruction):
function _process_operand (line 215) | def _process_operand(parser, operand_type, operand):
function _init (line 227) | def _init():
FILE: ljd/rawdump/constants.py
function read (line 28) | def read(parser, constants):
function _read_upvalue_references (line 38) | def _read_upvalue_references(parser, references):
function _read_complex_constants (line 49) | def _read_complex_constants(parser, complex_constants):
function _read_numeric_constants (line 86) | def _read_numeric_constants(parser, numeric_constants):
function _read_number (line 106) | def _read_number(parser):
function _read_signed_int (line 113) | def _read_signed_int(parser):
function _assemble_number (line 119) | def _assemble_number(lo, hi):
function _process_sign (line 129) | def _process_sign(number):
function _read_table (line 136) | def _read_table(parser, table):
function _read_table_item (line 158) | def _read_table_item(parser):
FILE: ljd/rawdump/debuginfo.py
function read (line 31) | def read(parser, line_offset, debuginfo):
function _read_lineinfo (line 41) | def _read_lineinfo(parser, line_offset, lineinfo):
function _read_upvalue_names (line 58) | def _read_upvalue_names(parser, names):
function _read_variable_infos (line 66) | def _read_variable_infos(parser, infos):
FILE: ljd/rawdump/header.py
class Flags (line 17) | class Flags():
method __init__ (line 18) | def __init__(self):
class Header (line 24) | class Header():
method __init__ (line 25) | def __init__(self):
function read (line 32) | def read(state, header):
function _check_magic (line 47) | def _check_magic(state):
function _read_version (line 55) | def _read_version(state, header):
function _read_flags (line 66) | def _read_flags(parser, header):
function _read_name (line 90) | def _read_name(state, header):
FILE: ljd/rawdump/parser.py
class _State (line 16) | class _State():
method __init__ (line 17) | def __init__(self):
function parse (line 23) | def parse(filename):
function _read_header (line 60) | def _read_header(parser, header):
function _read_prototypes (line 73) | def _read_prototypes(state, prototypes):
FILE: ljd/rawdump/prototype.py
class _State (line 21) | class _State():
method __init__ (line 22) | def __init__(self, parser):
function read (line 40) | def read(parser, prototype):
function _read_flags (line 72) | def _read_flags(parser, prototype):
function _read_counts_and_sizes (line 97) | def _read_counts_and_sizes(parser, prototype):
function _read_instructions (line 122) | def _read_instructions(parser, prototype):
function _read_constants (line 151) | def _read_constants(parser, prototype):
function _read_debuginfo (line 155) | def _read_debuginfo(stream, prototype):
FILE: ljd/util/binstream.py
class BinStream (line 10) | class BinStream():
method __init__ (line 11) | def __init__(self):
method open (line 20) | def open(self, filename):
method close (line 25) | def close(self):
method eof (line 30) | def eof(self):
method check_data_available (line 33) | def check_data_available(self, size=1):
method read_bytes (line 36) | def read_bytes(self, size=1):
method read_byte (line 46) | def read_byte(self):
method read_zstring (line 57) | def read_zstring(self):
method read_uleb128 (line 70) | def read_uleb128(self):
method read_uleb128_from33bit (line 88) | def read_uleb128_from33bit(self):
method read_uint (line 109) | def read_uint(self, size=4):
FILE: ljd/util/indentedstream.py
class IndentedStream (line 8) | class IndentedStream():
method __init__ (line 9) | def __init__(self, fd):
method write_multiline (line 15) | def write_multiline(self, fmt, *args, **kargs):
method start_line (line 36) | def start_line(self):
method write (line 42) | def write(self, fmt="", *args, **kargs):
method end_line (line 56) | def end_line(self):
method write_line (line 63) | def write_line(self, *args, **kargs):
method open_block (line 68) | def open_block(self, *args, **kargs):
method close_block (line 74) | def close_block(self, *args, **kargs):
FILE: ljd/util/log.py
function errprint (line 8) | def errprint(*args):
FILE: main.py
function dump (line 40) | def dump(name, obj, level=0):
function main (line 75) | def main():
Condensed preview — 46 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (249K chars).
[
{
"path": ".gitignore",
"chars": 332,
"preview": "*.lua[co]\n*.py[cod]\n\n# C extensions\n*.so\n\n# Packages\n*.egg\n*.egg-info\ndist\nbuild\neggs\nparts\nbin\nvar\nsdist\ndevelop-eggs\n."
},
{
"path": "LICENSE",
"chars": 1079,
"preview": "The MIT License (MIT)\n\nCopyright (c) 2013 Andrian Nord\n\nPermission is hereby granted, free of charge, to any person obta"
},
{
"path": "README.md",
"chars": 503,
"preview": "代码在fork https://github.com/NightNord/ljd 的基础上修改的\n\n主要修正了原版本没有将_read_header函数解析过的header中的flag值带进_read_protoypes。\n\n解决了原代码只能"
},
{
"path": "README.md.txt",
"chars": 2336,
"preview": "LuaJIT raw-bytecode decompiler (LJD)\n===\n\nThe original name was _ljwthgnd_ as in _LuaJIT 'What The Hell is Going On'\nDec"
},
{
"path": "gconfig.py",
"chars": 68,
"preview": "\n#zzw 20180714 support str encode\ngFlagDic = {'strEncode' : 'ascii'}"
},
{
"path": "ljd/__init__.py",
"chars": 72,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\n"
},
{
"path": "ljd/ast/__init__.py",
"chars": 72,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\n"
},
{
"path": "ljd/ast/builder.py",
"chars": 23725,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\nimport ljd.bytecode.instructions as ins\n\nfrom lj"
},
{
"path": "ljd/ast/helpers.py",
"chars": 1991,
"preview": "import ljd.ast.nodes as nodes\nimport ljd.ast.traverse as traverse\n\n\ndef insert_table_record(constructor, key, value):\n\ta"
},
{
"path": "ljd/ast/locals.py",
"chars": 4865,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\n\nimport ljd.ast.nodes as nodes\nimport ljd.ast.tr"
},
{
"path": "ljd/ast/mutator.py",
"chars": 5294,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\nimport copy\n\nfrom ljd.ast.helpers import *\n\n\ncla"
},
{
"path": "ljd/ast/nodes.py",
"chars": 10816,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\n\n# We should visit stuff in it's execution order"
},
{
"path": "ljd/ast/slotworks.py",
"chars": 10423,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\nimport ljd.ast.nodes as nodes\nimport ljd.ast.tra"
},
{
"path": "ljd/ast/traverse.py",
"chars": 3488,
"preview": "class Visitor():\n\tdef __init__(self):\n\t\tpass\n\n\t# ##\n\n\tdef visit_function_definition(self, node):\n\t\tpass\n\n\tdef leave_func"
},
{
"path": "ljd/ast/unwarper.py",
"chars": 37226,
"preview": "import copy\n\nimport ljd.ast.nodes as nodes\nimport ljd.ast.traverse as traverse\nimport ljd.ast.slotworks as slotworks\n\nbi"
},
{
"path": "ljd/ast/validator.py",
"chars": 8310,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\nimport ljd.ast.nodes as nodes\nimport ljd.ast.tra"
},
{
"path": "ljd/bytecode/__init__.py",
"chars": 72,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\n"
},
{
"path": "ljd/bytecode/constants.py",
"chars": 390,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\nT_NIL = 0\nT_FALSE = 1\nT_TRUE = 2\n\n\nclass Table()"
},
{
"path": "ljd/bytecode/debuginfo.py",
"chars": 838,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\n\nclass VariableInfo():\n\tT_VISIBILE = 0\n\tT_INTERN"
},
{
"path": "ljd/bytecode/helpers.py",
"chars": 168,
"preview": "def get_jump_destination(addr, instruction):\n\treturn addr + instruction.CD + 1\n\n\ndef set_jump_destination(addr, instruct"
},
{
"path": "ljd/bytecode/instructions.py",
"chars": 10153,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\n#\n# Almost direct wiki-to-code from\n# http://wik"
},
{
"path": "ljd/bytecode/prototype.py",
"chars": 600,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\nimport ljd.bytecode.constants as constants\nimpor"
},
{
"path": "ljd/lua/__init__.py",
"chars": 72,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\n"
},
{
"path": "ljd/lua/writer.py",
"chars": 16967,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\nimport re\n\nimport ljd.ast.nodes as nodes\nimport "
},
{
"path": "ljd/pseudoasm/__init__.py",
"chars": 72,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\n"
},
{
"path": "ljd/pseudoasm/constants.py",
"chars": 1082,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\nimport ljd.bytecode.constants\n\n\ndef write_tables"
},
{
"path": "ljd/pseudoasm/instructions.py",
"chars": 16726,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\nimport ljd.bytecode.instructions as ins\nfrom ljd"
},
{
"path": "ljd/pseudoasm/prototype.py",
"chars": 1049,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\nimport ljd.pseudoasm.constants\nimport ljd.pseudo"
},
{
"path": "ljd/pseudoasm/writer.py",
"chars": 968,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\nimport ljd.util.indentedstream\n\nimport ljd.pseud"
},
{
"path": "ljd/rawdump/__init__.py",
"chars": 72,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\n"
},
{
"path": "ljd/rawdump/code.py",
"chars": 7085,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\nfrom ljd.util.log import errprint\n\nimport ljd.by"
},
{
"path": "ljd/rawdump/constants.py",
"chars": 3743,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\nimport sys\nimport struct\n\nimport ljd.bytecode.co"
},
{
"path": "ljd/rawdump/debuginfo.py",
"chars": 2014,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\nimport sys\n\nimport ljd.bytecode.debuginfo\n\n\nVARN"
},
{
"path": "ljd/rawdump/header.py",
"chars": 2190,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\nfrom ljd.util.log import errprint\n\n\n_MAGIC = b'\\"
},
{
"path": "ljd/rawdump/parser.py",
"chars": 2069,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\n#!/usr/bin/python3\n\nimport ljd.util.binstream\nfr"
},
{
"path": "ljd/rawdump/prototype.py",
"chars": 4380,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\nfrom ljd.util.log import errprint\n\nimport ljd.by"
},
{
"path": "ljd/util/__init__.py",
"chars": 72,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\n"
},
{
"path": "ljd/util/binstream.py",
"chars": 1993,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\nimport io\nimport os\nimport sys\n\n\nclass BinStream"
},
{
"path": "ljd/util/indentedstream.py",
"chars": 1402,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\n_TAB_WIDTH = \" \" * 8\n\n\nclass IndentedStream():\n\t"
},
{
"path": "ljd/util/log.py",
"chars": 324,
"preview": "#\n# Copyright (C) 2013 Andrian Nord. See Copyright Notice in main.py\n#\n\nimport sys\n\n\ndef errprint(*args):\n\tfmt = None\n\n\t"
},
{
"path": "main.py",
"chars": 3465,
"preview": "#!/usr/bin/python3\n#\n# The MIT License (MIT)\n#\n# Copyright (c) 2013 Andrian Nord\n#\n# Permission is hereby granted, free "
},
{
"path": "test/breaks.lua",
"chars": 2703,
"preview": "--[[\n--]]\n\nfor i=1,2,3 do\n\tif x and y then\n\t\tprint(\"Then\")\n\telse\n\t\tbreak\n\tend\nend\n\nfor i=1,2,3 do\n\tif x then\n\t\tprint(\"Th"
},
{
"path": "test/expression.lua",
"chars": 5544,
"preview": "--[[\n--]]\n\nprint (\"true or true\")\n\nb = true or true\n\nprint (\"false and false\")\n\nb = false and false\n\nprint (\"false and o"
},
{
"path": "test/ifs.lua",
"chars": 11360,
"preview": "--[[\n--]]\n\nif true then\n\tprint(\"if true\")\nend\n\nprint (\"something\")\n\nif false then\n\tprint(\"if false\")\nend\n\nprint (\"someth"
},
{
"path": "test/loop.lua",
"chars": 5765,
"preview": "--[[\n--]]\n\nfor i=1, 100 do\n\tprint (\"Numeric loop\")\nend\n\nfor i=1, 100, 2 do\n\tprint (\"numeric for with step\")\nend\n\nfor i=1"
},
{
"path": "test/primitive.lua",
"chars": 8790,
"preview": "function assignment()\n\tfunction generator()\n\t\treturn 1, 2, 3\n\tend\n\n\ta = 1\n\tb, c, d = nil\n\te, f, g = 1, 2, 3\n\tlocal i, k,"
}
]
About this extraction
This page contains the full source code of the zzwlpx/ljd GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 46 files (217.5 KB), approximately 64.5k tokens, and a symbol index with 584 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.