[ { "path": "README.md", "content": "# `msidump`\n\n**MSI Dump** - a tool that analyzes malicious MSI installation packages, extracts files, streams, binary data and incorporates YARA scanner.\n\nOn Macro-enabled Office documents we can quickly use [oletools mraptor](https://github.com/decalage2/oletools/blob/master/oletools/mraptor.py) to determine whether document is malicious. If we want to dissect it further, we could bring in [oletools olevba](https://github.com/decalage2/oletools/blob/master/oletools/olevba.py) or [oledump](https://github.com/DidierStevens/DidierStevensSuite/blob/master/oledump.py).\n\nTo dissect malicious MSI files, so far we had only one, but reliable and trustworthy [lessmsi](https://github.com/activescott/lessmsi).\nHowever, `lessmsi` doesn't implement features I was looking for:\n\n- quick triage\n- Binary data extraction\n- YARA scanning\n\nHence this is where `msidump` comes into play.\n\n\n## Features\n\nThis tool helps in quick triages as well as detailed examinations of malicious MSIs corpora.\nIt lets us:\n\n- Quickly determine whether file is suspicious or not.\n- List all MSI tables as well as dump specific records\n- Extract Binary data, all files from CABs, scripts from CustomActions\n- scan all inner data and records with YARA rules\n- Uses `file`/MIME type deduction to determine inner data type\n\nIt was created as a companion tool to the blog post I released here:\n\n- [MSI Shenanigans. Part 1 - Offensive Capabilities Overview](https://mgeeky.tech/msi-shenanigans-part-1/)\n\n\n### Limitations\n\n- The program is still in an early alpha version, things are expected to break and triaging/parsing logic to change\n- Due to this tool heavy relience on Win32 COM `WindowsInstaller.Installer` interfaces, currently **it is not possible to support native Linux** platforms. Maybe `wine python msidump.py` could help, but haven't tried that yet.\n\n\n## Use Cases\n\n1. Perform quick triage of a suspicious MSI augmented with YARA rule:\n\n```\ncmd> python msidump.py evil.msi -y rules.yara\n```\n\n![1.png](img/1.png)\n\nHere we can see that input MSI is injected with suspicious **VBScript** and contains numerous executables in it.\n\n\n2. Now we want to take a closer look at this VBScript by extracting only that record. \n\nWe see from the triage table that it was present in `Binary` table. Lets get him:\n\n```\npython msidump.py putty-backdoored.msi -l binary -i UBXtHArj\n```\n\nWe can specify which to record dump either by its name/ID or its index number (here that would be 7).\n\n![2.png](img/2.png)\n\nLets have a look at another example. This time there is executable stored in `Binary` table that will be executed during installation:\n\n![3.png](img/3.png)\n\nTo extract that file we're gonna go with \n\n```\npython msidump.py evil2.msi -x binary -i lmskBju -O extracted\n```\n\nWhere \n- `-x binary` tells to extract contents of `Binary` table\n- `-i lmskBju` specifies which record exactly to extract\n- `-O extracted` sets output directory\n\n![4.png](img/4.png)\n\n\nFor the best output experience, run the tool on a **maximized console window** or redirect output to file:\n\n```\npython msidump.py [...] -o analysis.log\n```\n\n## Full Usage\n\n```\nPS D:\\> python .\\msidump.py --help\noptions:\n -h, --help show this help message and exit\n\nRequired arguments:\n infile Input MSI file (or directory) for analysis.\n\nOptions:\n -q, --quiet Surpress banner and unnecessary information. In triage mode, will display only verdict.\n -v, --verbose Verbose mode.\n -d, --debug Debug mode.\n -N, --nocolor Dont use colors in text output.\n -n PRINT_LEN, --print-len PRINT_LEN\n When previewing data - how many bytes to include in preview/hexdump. Default: 128\n -f {text,json,csv}, --format {text,json,csv}\n Output format: text, json, csv. Default: text\n -o path, --outfile path\n Redirect program output to this file.\n -m, --mime When sniffing inner data type, report MIME types\n\nAnalysis Modes:\n -l what, --list what List specific table contents. See help message to learn what can be listed.\n -x what, --extract what\n Extract data from MSI. For what can be extracted, refer to help message.\n\nAnalysis Specific options:\n -i number|name, --record number|name\n Can be a number or name. In --list mode, specifies which record to dump/display entirely. In --extract mode dumps only this particular record to --outdir\n -O path, --outdir path\n When --extract mode is used, specifies output location where to extract data.\n -y path, --yara path Path to YARA rule/directory with rules. YARA will be matched against Binary data, streams and inner files\n\n------------------------------------------------------\n\n- What can be listed:\n --list CustomAction - Specific table\n --list Registry,File - List multiple tables\n --list stats - Print MSI database statistics\n --list all - All tables and their contents\n --list olestream - Prints all OLE streams & storages.\n To display CABs embedded in MSI try: --list _Streams\n --list cabs - Lists embedded CAB files\n --list binary - Lists binary data embedded in MSI for its own purposes.\n That typically includes EXEs, DLLs, VBS/JS scripts, etc\n\n- What can be extracted:\n --extract all - Extracts Binary data, all files from CABs, scripts from CustomActions\n --extract binary - Extracts Binary data\n --extract files - Extracts files\n --extract cabs - Extracts cabinets\n --extract scripts - Extracts scripts\n\n------------------------------------------------------\n```\n\n## TODO\n\n- Triaging logic is still a bit flakey, I'm not very proud of it. Hence it will be subject for constant redesigns and further ramifications\n- Test it on a wider test samples corpora\n- Add support for input ZIP archives with passwords\n- Add support for ingesting entire directory full of YARA rules instead of working with a single file only\n- Currently, the tool matches malicious `CustomAction Type`s based on assessing their numbers, which is prone to being evaded.\n - It needs to be reworked to properly consume Type number and decompose it [onto flags](https://learn.microsoft.com/en-us/windows/win32/msi/summary-list-of-all-custom-action-types)\n\n\n## Tool's Name\n\nApparently when naming my tool, I didn't think on checking whether it was already taken.\nThere is another tool named `msidump` being part of [msitools](https://gitlab.gnome.org/GNOME/msitools) GNU package:\n\n- [msidump](https://wiki.gnome.org/msitools)\n\n---\n\n### ☕ Show Support ☕\n\nThis and other projects are outcome of sleepless nights and **plenty of hard work**. If you like what I do and appreciate that I always give back to the community,\n[Consider buying me a coffee](https://github.com/sponsors/mgeeky) _(or better a beer)_ just to say thank you! 💪 \n\n---\n\n```\nMariusz Banach / mgeeky, (@mariuszbit)\n\n```\n" }, { "path": "msidump.py", "content": "#!/usr/bin/python3\n#\n# Written by Mariusz Banach , @mariuszbit / mgeeky\n#\n\nimport sys\nimport os\nimport re\nimport glob\nimport pefile\nimport argparse\nimport hashlib\nimport random\nimport string\nimport tempfile\nimport textwrap\nimport cabarchive\nimport shutil\nimport atexit\nimport urllib\nfrom collections import OrderedDict\nfrom textwrap import fill\n\nif sys.platform != 'win32':\n print('\\n\\n[!] FATAL: This script can only be used in Windows system as it works with Win32 COM/OLE interfaces.\\n\\n')\n\nimport pythoncom\nimport win32com.client\nfrom win32com.shell import shell, shellcon\nfrom win32com.client import constants\n\nUSE_SSDEEP = False\n\ntry:\n import ssdeep\n USE_SSDEEP = True\nexcept:\n quiet = False\n # for a in sys.argv:\n # if a == '-q' or a == '--quiet':\n # quiet = True\n # break\n # if not quiet:\n # print(\"[!] 'ssdeep' not installed. Will not use it.\")\n\ntry:\n import colorama\n import magic\n import yara\n import olefile\n from prettytable import PrettyTable\n\nexcept ImportError as e:\n print(f'\\n[!] Requirements not installed: {e}\\n\\tInstall them with:\\n\\tcmd> pip install -r requirements.txt\\n')\n sys.exit(1)\n\n#########################################################\n\nVERSION = '0.2'\n\n#########################################################\n\noptions = {\n 'debug' : False,\n 'verbose' : False,\n 'format' : 'text',\n}\n\nlogger = None\n\ntry:\n colorama.init()\nexcept:\n pass\n\nclass Logger:\n colors_map = {\n 'red': colorama.Fore.RED, \n 'green': colorama.Fore.GREEN, \n 'yellow': colorama.Fore.YELLOW,\n 'blue': colorama.Fore.BLUE, \n 'magenta': colorama.Fore.MAGENTA, \n 'cyan': colorama.Fore.CYAN,\n 'white': colorama.Fore.WHITE, \n 'grey': colorama.Fore.WHITE,\n 'reset': colorama.Style.RESET_ALL,\n }\n \n def __init__(self, opts):\n self.opts = opts\n\n @staticmethod\n def colorize(txt, col):\n if type(txt) is not str:\n txt = str(txt)\n if not col in Logger.colors_map.keys() or options.get('nocolor', False):\n return txt\n return Logger.colors_map[col] + txt + Logger.colors_map['reset']\n\n @staticmethod\n def stripColors(txt):\n ansi_escape = re.compile(r'\\x1B(?:[@-Z\\\\-_]|\\[[0-?]*[ -/]*[@-~])')\n result = ansi_escape.sub('', txt)\n return result\n\n def fatal(self, txt):\n self.text('[!] ' + txt, color='red')\n sys.exit(1)\n\n def info(self, txt):\n self.text('[.] ' + txt, color='yellow')\n\n def err(self, txt):\n self.text('[-] ' + txt, color='red')\n\n def ok(self, txt):\n self.text('[+] ' + txt, color='green')\n\n def verbose(self, txt):\n if self.opts.get('verbose', False) or self.opts.get('debug', False):\n self.text('[>] ' + txt, color='cyan')\n\n def dbg(self, txt):\n if self.opts.get('debug', False):\n self.text('[dbg] ' + txt, color='magenta')\n\n def text(self, txt, color='none'):\n if color != 'none':\n txt = Logger.colorize(txt, color)\n\n if not self.opts.get('quiet', False):\n print(txt)\n\n\nclass MSIDumper:\n # https://learn.microsoft.com/pl-pl/windows/win32/msi/custom-action-return-processing-options?redirectedfrom=MSDN\n CustomActionReturnType = {\n 'check' : 0,\n 'ignore' : 64,\n 'asyncWait' : 128,\n 'asyncNoWait' : 192,\n }\n\n # https://learn.microsoft.com/en-us/windows/win32/msi/custom-action-execution-scheduling-options\n CustomActionExecuteType = {\n 'always' : 0,\n 'firstSequence' : 256,\n 'oncePerProcess' : 512,\n 'clientRepeat' : 768\n }\n\n #\n # https://learn.microsoft.com/en-us/windows/win32/msi/custom-action-in-script-execution-options\n # Deferred, rollback and commit custom actions can only be placed between InstallInitialize and InstallFinalize\n #\n CustomActionInScriptExecute = {\n 'immediate' : 0,\n 'deferred' : 1,\n 'rollback' : 1280,\n 'commit' : 1536,\n 'deferred-no-impersonate' : 3072,\n 'rollback-no-impersonate' : 3328,\n 'commit-no-impersonate' : 3584,\n }\n\n # https://learn.microsoft.com/en-us/windows/win32/msi/summary-list-of-all-custom-action-types\n CustomActionNativeTypes = {\n 'dll-in-binary-table' : 1,\n 'exe-in-binary-table' : 2,\n 'jscript-in-binary-table' : 5,\n 'vbscript-in-binary-table' : 6,\n 'dll-installed-with-product' : 17,\n 'exe-installed-with-product' : 18,\n 'jscript-installed-with-product' : 21,\n 'vbscript-installed-with-product' : 22,\n 'exe-with-directory-path-in-target' : 34,\n 'directory-set' : 35,\n 'jscript-in-sequence-table' : 37,\n 'vbscript-in-sequence-table' : 38,\n 'exe-command-line' : 50,\n 'jscript-with-funcname-in-property' : 53,\n 'vbscript-with-funcname-in-property' : 55,\n }\n\n OpenMode = {\n 'msiOpenDatabaseModeReadOnly' : 0,\n 'msiOpenDatabaseModeTransact' : 1,\n }\n\n SkipColumns = (\n 'extendedtype',\n )\n\n ListModes = (\n 'all', 'olestream', 'cabs', 'binary', 'stats', 'olestreams',\n )\n\n ExtractModes = (\n 'all', 'binary', 'files', 'cabs', 'scripts',\n )\n\n KnownCOMErrors = {\n 0x80004005 : 'Could not process input database',\n }\n\n KnownTables = (\n\t\t'ActionText', 'AdminExecuteSequence', 'AdminUISequence', 'AdvtExecuteSequence', 'AdvtUISequence', \n 'AppId', 'AppSearch', 'BBControl', 'Billboard', 'Binary', 'BindImage', 'CCPSearch', 'CheckBox', \n 'Class', 'ComboBox', 'CompLocator', 'Complus', 'Component', 'Condition', 'Control', 'ControlCondition',\n 'ControlEvent', 'CreateFolder', 'CustomAction', 'Dialog', 'Directory', 'DrLocator', \n 'DuplicateFile', 'Environment', 'Error', 'EventMapping', 'Extension', 'Feature', 'FeatureComponents', \n 'File', 'FileSFPCatalog', 'Font', 'Icon', 'IniFile', 'IniLocator', 'InstallExecuteSequence', \n 'InstallUISequence', 'IsolatedComponent', 'LaunchCondition', 'ListBox', 'ListView', 'LockPermissions', \n 'Media', 'MIME', 'MoveFile', 'MsiAssembly', 'MsiAssemblyName', 'MsiDigitalCertificate', \n 'MsiDigitalSignature', 'MsiEmbeddedChainer', 'MsiEmbeddedUI', 'MsiFileHash', 'MsiLockPermissionsEx', \n 'MsiPackageCertificate', 'MsiPatchCertificate', 'MsiPatchHeaders', 'MsiPatchMetadata', 'MsiPatchOldAssemblyFile', \n 'MsiPatchOldAssemblyName', 'MsiPatchSequence', 'MsiServiceConfig', 'MsiServiceConfigFailureActions', \n 'MsiSFCBypass', 'MsiShortcutProperty', 'ODBCAttribute', 'ODBCDataSource', 'ODBCDriver', 'ODBCSourceAttribute', \n 'ODBCTranslator', 'Patch', 'PatchPackage', 'ProgId', 'Property', 'PublishComponent', 'RadioButton', \n 'Registry', 'RegLocator', 'RemoveFile', 'RemoveIniFile', 'RemoveRegistry', 'ReserveCost', 'SelfReg', \n 'ServiceControl', 'ServiceInstall', 'SFPCatalog', 'Shortcut', 'Signature', 'TextStyle', 'TypeLib', 'UIText', \n 'Upgrade', 'Verb', '_Columns', '_Storages', '_Streams', '_Tables', '_TransformView', '_Validation',\n )\n\n ImportantTables = (\n 'CustomAction', 'InstallExecuteSequence', '_Streams', 'Media', 'InstallUISequence', 'Binary', '_TransformView',\n 'Component', 'Registry', 'Shortcut', 'RemoveFile', 'File',\n )\n\n SuspiciousTables = (\n 'CustomAction', 'Binary', '_Streams', \n )\n\n #\n # Approach based on assessing CustomAction Type numbers is prone to being evaded.\n # TODO: Rework it to properly consume Type number and decompose it onto flags:\n # https://learn.microsoft.com/en-us/windows/win32/msi/summary-list-of-all-custom-action-types\n #\n CustomActionTypes = {\n 'Execute' : {\n 'color' : 'red',\n 'types': (1250, 3298, 226),\n 'desc' : 'Will execute system commands or other executables',\n },\n 'VBScript' : {\n 'color' : 'red',\n 'types': (1126, 102),\n 'desc' : 'Will run VBScript in-memory',\n }, \n 'JScript' : {\n 'color' : 'red',\n 'types': (1125, 101),\n 'desc' : 'Will run JScript in-memory',\n },\n 'Run-Exe' : {\n 'color' : 'red',\n 'types': (1218, 194),\n 'desc' : 'Will extract executable from inner Binary table, drop it to:\\n C:\\\\Windows\\\\Installer\\\\MSIXXXX.tmp\\nand then run it.',\n },\n 'Load-DLL' : {\n 'color' : 'red',\n 'types': (65, ),\n 'desc' : 'Will load DLL in memory and invoke its exported function.\\nThat may also include .NET DLL',\n },\n 'Run-Dropped-File' : {\n 'color' : 'red',\n 'types': (1746,),\n 'desc' : 'Will run file extracted as a result of installation',\n },\n 'Set-Directory' : {\n 'color' : 'cyan',\n 'types': (51,),\n 'desc' : 'Will set Directory to a specific path',\n },\n }\n\n MimeTypesThatIncreasSuspiciousScore = (\n \"application/hta\",\n \"application/js\",\n \"application/msword\",\n \"application/vnd.ms-excel\",\n \"application/vnd.ms-powerpoint\",\n \"application/vns.ms-appx\",\n \"application/x-ms-shortcut\",\n \"application/x-vbs\",\n 'application/vnd.ms-excel', \n 'application/vnd.openxmlformats-officedocument.presentationml.presentation', \n 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', \n 'application/vnd.openxmlformats-officedocument.wordprocessingml.document',\n 'application/x-dosexec',\n )\n\n RecognizedInnerFileTypes = {\n 'cabinet' : {\n 'indicator' : 'MS Cabinet archive (.CAB)',\n 'safe-extension' : '.cab',\n 'color' : 'yellow',\n 'magic' : ('Microsoft Cabinet',)\n },\n 'executable' : {\n 'indicator' : 'PE executable (EXE)',\n 'safe-extension' : '.exe.bin',\n 'color' : 'red',\n 'magic' : (\n 'executable (console)', \n 'executable (GUI)', \n )\n },\n 'dll' : {\n 'indicator' : 'PE executable (DLL)',\n 'safe-extension' : '.dll.bin',\n 'color' : 'red',\n 'magic' : (\n 'executable (DLL)', \n )\n },\n 'unsure-executable' : {\n 'indicator' : 'PE executable (?)',\n 'safe-extension' : '.exe.bin',\n 'color' : 'red',\n 'min-keywords' : 3,\n 'keywords' : (\n 'This program', 'cannot be', 'run in', 'dos mode',\n ),\n },\n 'unsure-cabinet' : {\n 'indicator' : 'CAB archive (?)',\n 'safe-extension' : '.cab',\n 'color' : 'yellow',\n 'min-keywords' : 1,\n 'keywords' : (\n 'MSCF',\n ),\n },\n 'unsure-vbscript' : {\n 'indicator' : 'VBScript (?)',\n 'safe-extension' : '.vbs.bin',\n 'color' : 'red',\n 'printable' : True,\n 'min-keywords' : 3,\n 'keywords' : (\n 'dim', 'function ', 'sub ', 'createobject', 'getobject', 'with', 'string',\n 'object', 'set', 'then', 'end if', 'end function', 'end sub'\n ),\n 'not-keywords' : (\n ''\n\n if type(data) is str:\n data = data.encode()\n\n for i in range(0, num, 16):\n line = ''\n line += '%04x | ' % (addr + i)\n n += 16\n\n for j in range(n-16, n):\n if j >= len(data): break\n line += '%02x ' % (int(data[j]) & 0xff)\n\n line += ' ' * (3 * 16 + 7 - len(line)) + ' | '\n\n for j in range(n-16, n):\n if j >= len(data): break\n c = data[j] if not (data[j] < 0x20 or data[j] > 0x7e) else '.'\n line += '%c' % c\n\n lines.append(line)\n return '\\n'.join(lines)\n\n def parseCOMException(self, message, error, additional=''):\n code = error.hresult + 2**32\n code2 = 0\n\n try:\n code2 = error.excepinfo[-1] + 2**32\n except:\n pass\n\n if code2 != 0:\n if code in MSIDumper.KnownCOMErrors:\n additional += MSIDumper.KnownCOMErrors[code]\n\n if code2 in MSIDumper.KnownCOMErrors:\n additional += MSIDumper.KnownCOMErrors[code2]\n\n self.logger.err(f'''{message}:\n\n {error}\n\n HRESULT 1: 0x{code:08X} <-- General exception code\n\n HRESULT 2: 0x{code2:08X} <-- COM exception code. Google up that error number: \n https://google.com/?q={urllib.parse.quote_plus(f\"COM exception 0x{code2:08X}\")}\n\n {additional}\n''')\n\n else:\n if code in MSIDumper.KnownCOMErrors:\n additional += MSIDumper.KnownCOMErrors[code]\n\n self.logger.err(f'''{message}:\n\n {error}\n\n HRESULT: 0x{code:08X} <-- General exception code\n\n {additional}\n''')\n\n def open(self, infile):\n self.infile = os.path.abspath(os.path.normpath(infile))\n self.outdir = os.path.abspath(os.path.normpath(self.options.get('outdir', '')))\n\n if not os.path.isfile(self.infile):\n self.logger.fatal(f'Input file does not exist: {self.infile}')\n\n mode = MSIDumper.OpenMode['msiOpenDatabaseModeReadOnly']\n\n if self.disinfectionMode:\n self.logger.fatal('MSI Disinfection is not yet implemented.')\n mode = MSIDumper.OpenMode['constants.msiOpenDatabaseModeTransact']\n\n self.initCOM()\n\n try:\n self.logger.dbg(f'Opening database {self.infile} ...')\n self.nativedb = self.installer.OpenDatabase(\n self.infile, \n mode\n )\n\n return True\n\n except pythoncom.com_error as error:\n if self.options['debug']:\n self.parseCOMException(\n message=f\"Could not open MSI database natively via COM\",\n error=error\n )\n\n return False\n\n def close(self):\n if self.nativedb is not None:\n self.nativedb = None\n \n if self.installer is not None:\n try:\n self.installer.Release()\n except:\n pass\n\n self.installer = None\n\n def initCOM(self):\n if self.installer is not None:\n return\n\n try:\n #\n # Logic borrowed from:\n # https://github.com/orestis/python/blob/master/Tools/msi/msilib.py#L60\n #\n\n self.logger.dbg('Initializing COM and instantiating WindowsInstaller.Installer ...')\n pythoncom.CoInitialize()\n\n win32com.client.gencache.EnsureModule('{000C1092-0000-0000-C000-000000000046}', 1033, 1, 0)\n\n self.installer = win32com.client.Dispatch(\n 'WindowsInstaller.Installer',\n resultCLSID='{000C1090-0000-0000-C000-000000000046}'\n )\n\n if self.installer is None:\n self.logger.fatal('Could not instantiate WindowsInstaller.Installer!')\n\n except Exception as e:\n self.logger.fatal(f'Could not instantiate WindowsInstaller.Installer. Exception:\\n\\n\\t{e}')\n\n def collectEntries(self, table, dontSort = False):\n entries = []\n\n try:\n entries = self._collectEntries(\n table, \n dontSort\n )\n except Exception as e:\n self.logger.dbg(f'Error: Table {table} did not contain any records.')\n\n if self.options.get('debug', False) and table.lower() != '_streams':\n raise\n\n return entries\n\n def _collectEntries(self, table, dontSort = False):\n assert self.nativedb is not None, \"Database is not opened\"\n entries = []\n\n view = self.nativedb.OpenView(f\"SELECT * FROM {table}\")\n view.Execute(None)\n\n types = view.ColumnInfo(constants.msiColumnInfoTypes)\n names = view.ColumnInfo(constants.msiColumnInfoNames)\n columns = []\n\n for i in range(1, types.FieldCount+1):\n t = types.StringData(i)\n n = names.StringData(i)\n\n if t[0] in 'slSL':\n columns.append((n, 'str'))\n elif t[0] in 'iI':\n columns.append((n, 'int'))\n elif t[0] == 'v':\n columns.append((n, 'bin'))\n else:\n self.logger.dbg(f'Unsupported column type: table {table}, column: {i}. Type: {t}, Name: {n}')\n columns.append((n, '?'))\n\n while True:\n r = view.Fetch() \n if not r:\n break\n\n rec = OrderedDict()\n for i in range(1, r.FieldCount+1):\n val = None\n name = columns[i-1][0]\n\n if r.IsNull(i):\n val = ''\n\n elif columns[i-1][1] == 'str': \n try:\n val = r.StringData(i)\n\n except Exception as e:\n txt = f'Could not convert {table} column {columns[i-1][0]} value to string (type: {columns[i-1][1]}): {e}'\n if txt not in self.errorsCache:\n self.logger.dbg(txt)\n self.errorsCache.add(txt)\n val = ''\n\n elif columns[i-1][1] == 'int': \n try:\n val = r.IntegerData(i)\n except Exception as e:\n txt = f'Could not convert {table} column {columns[i-1][0]} value to integer (type: {columns[i-1][1]}): {e}'\n if txt not in self.errorsCache:\n self.logger.dbg(txt)\n self.errorsCache.add(txt)\n val = 0\n\n elif columns[i-1][1] == 'bin': \n size = r.DataSize(i)\n val = r.ReadStream(i, size, constants.msiReadStreamBytes)\n\n rec[columns[i-1][0].lower()] = val\n\n entries.append(rec)\n\n view.Close()\n\n if not dontSort and table in MSIDumper.TableSortBy:\n entries = sorted(entries, key=lambda x: list(x.values())[MSIDumper.TableSortBy[table]] )\n\n self.logger.dbg(f'Collected {len(entries)} entries from {table} ...')\n return entries\n\n def getMaxValueFromTable(self, table, columnNum):\n maxVal = -1\n entries = self.collectEntries(table)\n\n for entry in entries:\n if maxVal < entry[columnNum]:\n maxVal = entry[columnNum]\n\n return maxVal\n\n def analyse(self):\n assert self.nativedb is not None, \"Database is not opened\"\n\n try:\n ret = self.analysisWorker()\n\n if self.grade > 0:\n self.verdict = f'[.] Verdict: {Logger.colorize(\"SUSPICIOUS\", \"red\")}'\n\n self.logger.verbose(f'Verdict grade: {self.grade}')\n\n return ret\n\n except Exception as e:\n if self.nativedb is not None:\n self.nativedb = None\n\n if self.options['debug']: \n raise\n else:\n self.logger.err(f'Could not analyse input MSI. Enable --debug to learn more. Exception: {e}')\n\n return False\n\n finally:\n pass\n\n def listTable(self, table):\n if ',' in table:\n output = ''\n tables = table.split(',')\n for t in tables:\n output += f'{Logger.colorize(\"[+]\", \"green\")} Listing: {Logger.colorize(t, \"green\")}\\n\\n'\n\n out = self._listTable(t)\n if out is not None:\n output += str(out) + '\\n'\n\n return output\n else:\n return self._listTable(table)\n\n def _listTable(self, table):\n assert self.nativedb is not None, \"Database is not opened\"\n\n records = None\n\n if table == 'streams': table = '_Streams'\n if table == 'stream': table = '_Streams'\n if table == 'binary': table = 'Binary'\n if table == 'cabs': table = 'Media'\n if table == 'olestreams':table = 'olestream'\n\n if table.lower() not in [x.lower() for x in MSIDumper.KnownTables + MSIDumper.ListModes]:\n tb = PrettyTable(['1','2','3'])\n tb.header = False\n vals = list(MSIDumper.KnownTables + MSIDumper.ListModes)\n i = 0\n while i + 3 < len(vals):\n tb.add_row([vals[i+0], vals[i+1], vals[i+2]])\n i += 3\n\n if i < len(vals):\n for j in range(len(vals)-i):\n tb.add_row([vals[i+j], '', ''])\n\n self.logger.fatal(f'Unsupported --list setting: {table}\\n Pick one/combination of following --list values:\\n\\n{tb}\\n')\n\n if table.lower() in [x.lower() for x in MSIDumper.KnownTables]:\n try:\n if table not in MSIDumper.KnownTables:\n for t in MSIDumper.KnownTables:\n if table.lower() == t.lower():\n table = t\n break\n\n index = self.options.get('record', -1)\n if index != -1:\n records0 = self.collectEntries(table)\n\n try:\n index = int(index)\n if index < 0 or index-1 > len(records0):\n self.logger.fatal(f'Invalid --record specified. There were only {len(records0)} records returned from {table}.\\n\\t\\tUse value between --record 1 and --record {len(records0)}')\n records = [ records0[index-1], ]\n except:\n records = []\n for a in records0:\n vals = list(a.values())\n if len(vals) > 0 and vals[0].lower() == index.lower():\n records.append(a)\n break\n\n if len(records) == 0:\n self.logger.fatal(f'Invalid --record specified. Could not find {table} record entry based on its index number nor ID name.')\n else:\n records = self.collectEntries(table) \n \n except Exception as e:\n self.logger.err(f'Exception occurred while enumerating {table} entries: {e}')\n\n if self.options.get('debug', False):\n raise\n else:\n table = table.lower()\n\n try:\n if table == 'stats':\n records = self.collectStats()\n elif table == 'all':\n return self.collectAll()\n elif table == 'olestream':\n records = self.collectStreams()\n else:\n self.logger.fatal(f'Unsupported --list setting: {table}')\n\n except Exception as e:\n self.logger.err(f'Exception occurred while pulling MSI metadata {table}: {e}')\n\n if self.options.get('debug', False):\n raise\n\n if records is not None:\n self.tableSpecificHighlighting(table, records)\n return self.printTable(table, records)\n\n else:\n if table in MSIDumper.KnownTables:\n return f'No records found in {Logger.colorize(table, \"green\")} table.'\n else:\n return f'No {Logger.colorize(table, \"green\")} metadata was extracted.'\n\n def tableSpecificHighlighting(self, table, records):\n if table.lower() == 'customaction':\n for i in range(len(records)):\n rec = records[i]\n for k, v in rec.items():\n if k == 'type':\n col = ''\n for a, b in MSIDumper.CustomActionTypes.items():\n if v in b['types']:\n col = b['color']\n break\n if col != '':\n records[i][k] = Logger.colorize(v, col)\n records[i]['source'] = Logger.colorize(records[i]['source'], col)\n\n if table.lower() == 'binary':\n for i in range(len(records)):\n records[i]['Magic type'] = self.sniffDataType(records[i]['data'], color=True)\n \n def extract(self, what):\n assert self.nativedb is not None, \"Database is not opened\"\n\n what = what.lower()\n\n if what == 'script':\n what = 'scripts'\n\n if what not in [x.lower() for x in MSIDumper.ExtractModes]:\n self.logger.fatal(f'Unsupported --extract setting: {what}')\n\n self.outdir = os.path.normpath(os.path.abspath(self.options.get('outdir', '')))\n if len(self.outdir) == 0:\n self.outdir = os.getcwd()\n\n if not os.path.isdir(self.outdir):\n os.makedirs(self.outdir)\n\n if what == 'all':\n return self.extractAll()\n elif what == 'binary':\n return self.extractBinary()\n elif what == 'files':\n return self.extractFiles()\n elif what == 'cabs':\n return self.extractCABs()\n elif what == 'scripts':\n return self.extractScripts()\n\n def extractAll(self):\n output = ''\n\n outs = self.extractBinary()\n if len(outs) > 0:\n output += outs + '\\n'\n \n outs = self.extractFiles()\n if len(outs) > 0:\n output += outs + '\\n'\n\n outs = self.extractCABs()\n if len(outs) > 0:\n output += outs + '\\n'\n\n outs = self.extractScripts()\n if len(outs) > 0:\n output += outs + '\\n'\n\n output += f'\\nExtracted in total {self.extractedCount} objects.\\n'\n\n return output\n\n def sanitizeName(self, name):\n windowsNames = (\n 'CON', 'PRN', 'AUX', 'NUL', 'COM1', 'COM2', \n 'COM3', 'COM4', 'COM5', 'COM6', 'COM7', \n 'COM8', 'COM9', 'LPT1', 'LPT2', 'LPT3', 'LPT4', \n 'LPT5', 'LPT6', 'LPT7', 'LPT8', 'LPT9', \n )\n\n for a in ('..', '\\\\', '/', '\"', \"'\", '?', '*', ':'):\n name = name.replace(a, '')\n\n for a in windowsNames:\n name = name.replace(a, '')\n \n if len(name) == 0:\n name = 'bin-' + ''.join(random.choices(string.ascii_uppercase + string.digits, k=5))\n\n return name\n\n def extractBinary(self):\n binary = self.collectEntries('Binary')\n num = 0\n output = ''\n\n self.logger.verbose('Extracting data from Binary table...')\n\n if len(binary) == 0:\n self.logger.err('Input MSI does not contain any embedded Binary data.')\n\n for elem in binary:\n sniffed = self.sniffDataType(elem['data'])\n name = self.sanitizeName(elem['name']) + self.sniffDataExt(sniffed)\n outp = os.path.join(self.outdir, name)\n\n with open(outp, 'wb') as f:\n f.write(elem['data'].encode())\n\n num += 1\n output += f'\\n{Logger.colorize(\"[+]\",\"green\")} Extracted {Logger.colorize(len(elem[\"data\"]),\"green\")} bytes of {Logger.colorize(elem[\"name\"],\"green\")} object to: {Logger.colorize(outp,\"yellow\")}'\n\n self.extractedCount += num\n if num > 0 and self.options.get('extract', '') != 'all':\n output += f'\\n\\nExtracted in total {num} objects.\\n'\n\n return output\n\n def extractCab(self, infile, outdir, files):\n with open(infile, \"rb\") as f:\n arc = cabarchive.CabArchive(f.read())\n\n self.logger.verbose('Extracting Cabinets from MSI...')\n\n output = f'Extracting files from CAB ({infile}):\\n\\n'\n num = 0\n\n for k, v in arc.items():\n fn = v.filename\n\n for _file in files:\n if fn == _file['file']:\n fn = _file['filename']\n\n p, ext = os.path.splitext(fn)\n if ext.lower() in MSIDumper.DangerousExtensions:\n fn += '.bin'\n\n lp = os.path.join(outdir, fn)\n\n lp1 = os.path.join(outdir, os.path.dirname(lp))\n if not os.path.isdir(lp1):\n output += f'\\t{Logger.colorize(\"[+]\",\"green\")} Creating temp dir: {lp1}\\n'\n os.makedirs(lp1, exist_ok=True)\n\n output += f'{Logger.colorize(\"[+]\",\"green\")} {v.filename:20} => {lp}\\n'\n with open(lp, 'wb') as f:\n f.write(v.buf)\n num += 1\n\n return num, output\n\n def extractFiles(self, overrideOutdir=''):\n outdir = self.outdir\n if len(overrideOutdir) > 0:\n dirpath = overrideOutdir\n else:\n dirpath = tempfile.mkdtemp()\n\n self.outdir = dirpath\n self.extractCABs()\n self.outdir = outdir\n\n self.logger.verbose('Extracting files from MSI...')\n\n cabsNum = 0\n num = 0\n output = ''\n files = self.collectEntries('File')\n\n path = os.path.join(dirpath, '*.cab')\n for cab in glob.glob(path, recursive=True):\n cabPath = os.path.join(path, cab)\n cabsNum += 1\n outp = os.path.join(dirpath, os.path.basename(cabPath).replace('.cab', ''))\n\n try:\n num0, output0 = self.extractCab(cabPath, outp, files)\n num += num0\n output += output0\n\n except Exception as e:\n self.logger.err(f'Could not extract files from CABinet: {cabPath}. Error: {e}')\n if self.options.get('debug', False):\n raise\n finally:\n if os.path.isfile(cabPath):\n os.remove(cabPath)\n\n if dirpath != overrideOutdir:\n shutil.rmtree(dirpath)\n\n self.extractedCount += num\n if num > 0 and self.options.get('extract', '') != 'all':\n output += f'\\nExtracted in total {num} files from {cabsNum} cabinets.\\n'\n\n return output\n\n def extractCABs(self):\n binary = self.collectEntries('Binary')\n num = 0\n output = ''\n\n if len(binary) == 0:\n self.logger.err('Input MSI does not contain any embedded Binary data.')\n\n for elem in binary:\n sniffed = self.sniffDataType(elem['data'])\n if '.cab' not in sniffed.lower():\n continue\n\n name = self.sanitizeName(elem['name']) + '.cab'\n outp = os.path.join(self.outdir, name)\n\n with open(outp, 'wb') as f:\n f.write(elem['data'].encode())\n\n num += 1\n\n # source: https://github.com/decalage2/oletools/blob/master/oletools/oledir.py#L245\n ole = olefile.OleFileIO(self.infile)\n for entry in ole.listdir():\n name = entry[-1]\n name = repr(name)[1:-1]\n entry_id = ole._find(entry)\n try:\n size = ole.get_size(entry)\n except:\n size = '-'\n\n data0 = ole.openstream(entry).getvalue()\n data = data0.decode(errors='ignore')\n\n sniffed = self.sniffDataType(data)\n if '.cab' not in sniffed.lower():\n continue\n\n name = f'ole-stream-{entry_id}.cab'\n outp = os.path.join(self.outdir, name)\n\n with open(outp, 'wb') as f:\n f.write(data0)\n\n num += 1\n output += f'\\n{Logger.colorize(\"[+]\",\"green\")} Extracted {Logger.colorize(len(elem[\"data\"]), \"green\")} bytes of {Logger.colorize(elem[\"name\"],\"green\")} object to: {Logger.colorize(outp,\"yellow\")}'\n\n self.extractedCount += num\n if num > 0 and self.options.get('extract', '') != 'all':\n output += f'\\n\\nExtracted in total {num} objects.\\n'\n\n return output\n\n def extractScripts(self):\n binary = self.collectEntries('Binary')\n actions = self.collectEntries('CustomAction')\n num = 0\n output = ''\n\n self.logger.verbose('Extracting scripts from CustomAction and Binary tables...')\n\n if len(binary) == 0:\n self.logger.err('Input MSI does not contain any embedded Binary data.')\n\n for elem in actions:\n sniffed = self.sniffDataType(elem['target'])\n if 'vbscript' not in sniffed.lower() and 'jscript' not in sniffed.lower():\n continue\n\n name = self.sanitizeName(elem['action'])\n outp = os.path.join(self.outdir, name) + self.sniffDataExt(sniffed)\n\n with open(outp, 'wb') as f:\n f.write(elem['target'].encode())\n\n num += 1\n output += f'\\n{Logger.colorize(\"[+]\",\"green\")} Extracted {Logger.colorize(len(elem[\"target\"]),\"green\")} bytes of {Logger.colorize(elem[\"action\"],\"green\")} CustomAction script to: {Logger.colorize(outp,\"yellow\")}'\n\n for elem in binary:\n sniffed = self.sniffDataType(elem['data'])\n if 'vbscript' not in sniffed.lower() and 'jscript' not in sniffed.lower():\n continue\n \n name = self.sanitizeName(elem['name'])\n outp = os.path.join(self.outdir, name) + self.sniffDataExt(sniffed)\n\n with open(outp, 'wb') as f:\n f.write(elem['data'].encode())\n\n num += 1\n output += f'\\n{Logger.colorize(\"[+]\",\"green\")} Extracted {Logger.colorize(len(elem[\"data\"]),\"green\")} bytes of {Logger.colorize(elem[\"name\"],\"green\")} binary object script to: {Logger.colorize(outp,\"yellow\")}'\n\n self.extractedCount += num\n if num > 0 and self.options.get('extract', '') != 'all':\n output += f'\\n\\nExtracted in total {num} objects.\\n'\n\n return output\n\n def formatTable(self, tbl, table, records):\n if self.maxWidth > -1 and len(records) > 0:\n for k in records[0].keys():\n tbl._max_width[k] = self.maxWidth\n\n tbl.align['YARA Results'] = 'l'\n\n if table.lower() in self.specificTableAlignment.keys():\n for k, v in self.specificTableAlignment[table.lower()].items():\n tbl.align[k] = v\n\n if table.lower() in [x.lower() for x in MSIDumper.TableSortBy] and len(records) > 0:\n tbl.sortby = list(records[0].keys())[MSIDumper.TableSortBy[table]]\n\n return tbl\n\n def collectAll(self):\n output = ''\n\n self.logger.info('Dumping all MSI tables...')\n\n for table in MSIDumper.KnownTables:\n recs = self.collectEntries(table)\n\n if not self.options.get('verbose', False) and len(recs) == 0 and table not in MSIDumper.ImportantTables:\n continue\n\n output += '\\n\\n'\n output += Logger.colorize(f'===============[ {table} : {len(recs)} records ]===============', 'green')\n output += '\\n\\n'\n output += self.printTable(table, recs)\n\n return output\n \n def collectStreams(self):\n records = []\n\n ole = olefile.OleFileIO(self.infile)\n for entry in ole.listdir(storages=True):\n name = entry[-1]\n name = repr(name)[1:-1]\n entry_id = ole._find(entry)\n try:\n size = ole.get_size(entry)\n except:\n size = '-'\n typeid = ole.get_type(entry)\n clsid = ole.getclsid(entry)\n \n data0 = ole.openstream(entry).getvalue()\n data = data0.decode(errors='ignore')\n sniffed = self.sniffDataType(data, color=True)\n\n records.append({\n 'entry_id' : entry_id,\n 'data type' : sniffed,\n 'name' : Logger.colorize(name, 'yellow'),\n 'size' : size,\n 'typeid' : typeid,\n 'CLSID' : clsid,\n })\n\n return sorted(records, key=lambda x: x['entry_id'])\n\n def collectStats(self):\n records = []\n hashes = (\n 'md5', 'sha1', 'sha256', 'ssdeep'\n )\n\n self.logger.info('Computing MSI file hashes...')\n\n with open(self.infile, 'rb') as f:\n data = f.read()\n\n for h in hashes:\n if h == 'ssdeep':\n if USE_SSDEEP:\n hsh = ssdeep.hash(data)\n else:\n hsh = 'err: ssdeep module not installed'\n else:\n m = hashlib.new(h)\n m.update(data)\n hsh = m.hexdigest()\n\n records.append({\n 'type' : Logger.colorize(f'Hash {h}', 'cyan'),\n 'value' : Logger.colorize(hsh, 'cyan'),\n })\n\n del data\n\n self.logger.info('Collecting MSI tables stats...')\n\n for table in MSIDumper.KnownTables:\n recs = self.collectEntries(table)\n val = f'{len(recs)} records'\n\n if table in MSIDumper.SuspiciousTables:\n table = Logger.colorize(table, 'red')\n val = Logger.colorize(val, 'red')\n\n elif table in MSIDumper.ImportantTables:\n table = Logger.colorize(table, 'yellow')\n val = Logger.colorize(val, 'yellow')\n\n else:\n if len(recs) == 0 and not self.options.get('verbose', False):\n continue\n\n records.append({\n 'type' : table,\n 'value' : val,\n })\n\n return records\n\n def analysisWorker(self):\n self.processActions()\n self.lookForIOCs()\n\n return self.printReport()\n\n def normalizeDataForOutput(self, val, num=0, table=''):\n if num == 0:\n num = self.options.get('print_len', MSIDumper.DefaultTableWidth)\n\n if num != -1:\n val = val[:num]\n\n printable = MSIDumper.isprintable(val)\n\n if not printable and table not in ('olestream', ):\n printable2 = MSIDumper.isprintable(Logger.stripColors(val))\n if not printable2:\n val = MSIDumper.hexdump(val) + '\\n'\n\n return val\n \n def cleanString(self, txt):\n txt = txt.replace('\\r', '')\n txt = txt.replace('\\t', ' ')\n\n if self.options.get('format', 'text') in ('csv', 'json'):\n txt = Logger.stripColors(txt)\n txt = ''.join(filter(lambda x: x in string.printable, txt))\n txt = txt.replace('\\n', ' ')\n txt = re.sub(r'\\s+', ' ', txt, re.I)\n \n return txt\n\n def printTable(self, table, records):\n if len(records) == 0:\n return f'\\n\\nNo records found in table {Logger.colorize(table, \"green\")}.'\n\n yaraColumn = ''\n self.logger.dbg(f'Dumping {table} table results...')\n\n rules = None\n if len(self.options.get('yara', '')) > 0 and table != 'YARA Results':\n yaraColumn = 'YARA Results'\n matchesReport = []\n rules = self.initYara()\n\n if len(records) == 1 and (self.options.get('record', '') != -1 and len(self.options.get('record', '')) > 0):\n output = ''\n\n for k, v in records[0].items():\n k0 = Logger.colorize(k, \"green\")\n output += f'\\n- {k0:20} : '\n\n if type(v) is str:\n v = self.normalizeDataForOutput(v, -1, table=table)\n\n if len(v) < 50:\n output += v\n else:\n spacer = Logger.colorize('=' * MSIDumper.DefaultTableWidth, 'yellow')\n output += '\\n\\n' + spacer + '\\n\\n' + v + '\\n\\n' + spacer + '\\n'\n else:\n output += str(v)\n\n if table in ('binary', ):\n output += '\\n'\n\n output += '\\n'\n\n if len(yaraColumn) > 0:\n k0 = Logger.colorize(yaraColumn, \"green\")\n output += f'\\n- {k0:20} : '\n\n for k, v in records[0].items(): \n if type(v) is not str:\n continue\n matches = rules.match(data = v)\n if matches:\n ms = ''\n for m in matches:\n ms += f'- {m.rule}\\n'\n output += Logger.colorize(f'YARA rule match on column {k}:', 'green') + '\\n' + ms + '\\n'\n else:\n output = ''\n numCol = ['#',]\n yarCol = []\n if table == 'olestream':\n numCol = []\n\n if len(yaraColumn) > 0:\n yarCol = [yaraColumn, ]\n\n tbl = PrettyTable(numCol + list(records[0].keys()) + yarCol)\n num = 0\n\n index = self.options.get('record', -1)\n if index != -1:\n num = index - 1\n\n tbl = self.formatTable(tbl, table, records)\n\n for rec in records:\n num += 1\n vals = []\n i = 0\n for v in [num, ] + list(rec.values()):\n if i == 0 and 'entry_id' in rec.keys():\n i += 1\n continue\n if type(v) is str:\n v = self.normalizeDataForOutput(v, table=table)\n s = self.cleanString(v).strip()\n n = ''\n\n if table.lower() in ('binary', ):\n n = '\\n'\n\n vals.append(s + n)\n else:\n vals.append(v)\n i += 1\n\n if len(yaraColumn) > 0:\n i = 0\n val = ''\n for v in list(rec.values()): \n if type(v) is not str:\n i += 1\n continue\n matches = rules.match(data = v)\n if matches:\n ms = ''\n for m in matches:\n ms += f'- {m.rule}\\n'\n k = list(rec.keys())[i]\n val += Logger.colorize(f'YARA rule match on column {k}:', 'green') + '\\n' + ms + '\\n'\n i += 1\n vals.append(val)\n\n if self.options['format'] == 'csv':\n tbl.add_row([str(x).replace(self.csvDelim, '') for x in vals])\n else:\n tbl.add_row(vals)\n\n if self.options['format'] == 'text':\n output += str(tbl)\n\n if table != 'YARA Results' and self:\n output += f'\\n\\n[.] Found {Logger.colorize(str(len(records)), \"green\")} records in {Logger.colorize(table, \"green\")} table.'\n\n output += '\\n'\n\n elif self.options['format'] == 'json':\n output += str(tbl.get_json_string())\n \n elif self.options['format'] == 'csv':\n output += str(tbl.get_csv_string(delimiter=self.csvDelim, escapechar='\\\\'))\n \n # elif self.options['format'] == 'html':\n # output += str(tbl.get_html_string())\n \n return output\n\n def printReport(self):\n output = ''\n cols = [\n '#',\n 'threat',\n 'location',\n 'context',\n 'description'\n ]\n tbl = PrettyTable(cols)\n tbl = self.formatTable(tbl, 'report', self.report)\n\n num = 0\n\n for report in self.report:\n num += 1\n rec = [\n num,\n report['name'],\n report['location'],\n report['context'],\n report['desc'],\n ]\n vals = []\n for v in rec:\n if type(v) is str:\n vals.append(self.cleanString(v))\n else:\n vals.append(v)\n\n if self.options['format'] == 'csv':\n tbl.add_row([str(x).replace(self.csvDelim, '') for x in vals])\n else:\n tbl.add_row(vals)\n\n if self.options['format'] == 'text':\n output += str(tbl)\n\n elif self.options['format'] == 'json':\n output += str(tbl.get_json_string())\n \n elif self.options['format'] == 'csv':\n output += str(tbl.get_csv_string(delimiter=self.csvDelim, escapechar='\\\\'))\n \n # elif self.options['format'f] == 'html':\n # output += str(tbl.get_html_string())\n\n return output\n\n def printRecord(self, rec, indent=''):\n out = ''\n keyLen = -1\n\n if type(rec) is str:\n return rec\n\n for k, v in rec.items():\n if len(k) > keyLen:\n keyLen = len(Logger.colorize(k, 'yellow')) + 1\n\n if self.format == 'text':\n for k, v in rec.items():\n if k.lower() in MSIDumper.SkipColumns:\n continue\n\n if type(v) is str or type(v) is bytes:\n printable = MSIDumper.isprintable(v)\n\n if not printable and v[0] != '\\x1b':\n v = '\\n\\n' + MSIDumper.hexdump(v) + '\\n'\n\n if self.options.get('record', -1) == -1 and len(v) > 256: \n v = '\\n\\n' + v[:256].strip() + '\\n\\t[CUT FOR BREVITY]\\n'\n\n k = Logger.colorize(k, 'yellow')\n out += indent + f'- {k:{keyLen}}: {v}\\n'\n\n elif self.format == 'csv':\n out = self.csvDelim.join([str(x).replace(self.csvDelim, '')[:self.maxWidth] for x in rec.values()])\n\n return out\n\n @staticmethod\n def isValidPE(data):\n pe = None\n try:\n pe = pefile.PE(data=data.encode(), fast_load=True)\n _format = MSIDumper.RecognizedInnerFileTypes['executable']['indicator']\n\n if pe.OPTIONAL_HEADER.DllCharacteristics != 0:\n _format = MSIDumper.RecognizedInnerFileTypes['dll']['indicator']\n\n pe.close()\n return (True, _format)\n except pefile.PEFormatError as e:\n logger.dbg(f'pefile error: {e}')\n return (False, '')\n finally:\n if pe:\n pe.close()\n\n def sniffDataExt(self, sniffed):\n for k, v in MSIDumper.RecognizedInnerFileTypes.items():\n if v['indicator'].lower() == sniffed.lower():\n return MSIDumper.RecognizedInnerFileTypes[k]['safe-extension']\n\n return ''\n\n def gradeFoundIndicator(self, indicator, data='', color='', mime=''):\n if color != '':\n if color == 'red':\n return 1\n \n if mime != '' and mime.lower() in MSIDumper.MimeTypesThatIncreasSuspiciousScore:\n return 1\n\n return 0\n\n def sniffDataType(self, data, color=False):\n mime = self.options.get('mime', False)\n magicOut = 'data'\n try:\n magicOut = magic.from_buffer(data, mime=mime)\n except Exception as e:\n self.logger.dbg(f'Magic failed fingerprinting data: {e}')\n\n pe, petype = MSIDumper.isValidPE(data)\n if pe:\n if mime and magicOut in ('data', 'application/octet-stream'):\n indicator = 'application/x-dosexec'\n if color:\n indicator = Logger.colorize(petype, 'red')\n self.grade += self.gradeFoundIndicator(indicator, data, color='red')\n return indicator\n\n for format, predicate in MSIDumper.RecognizedInnerFileTypes.items():\n indicator = predicate.get('indicator', '')\n predColor = predicate.get('color', '')\n\n if format == 'unsure-executable':\n if data[:2] != 'MZ' and data[:2] != 'ZM':\n continue\n elif format == 'unsure-cabinet':\n if data[:4] != 'MSCF':\n continue\n\n if mime:\n indicator = magicOut\n\n if color:\n indicator = Logger.colorize(indicator, predColor)\n \n magicVals = predicate.get('magic', [])\n if len(magicVals) > 0:\n for m in magicVals:\n if m.lower() in magicOut.lower():\n self.grade += self.gradeFoundIndicator(indicator, data, color=predColor)\n return indicator\n\n keywords = predicate.get('keywords', [])\n minkeywords = predicate.get('min-keywords', 0)\n \n printable = predicate.get('printable', 0)\n printableMet = False\n if printable:\n if MSIDumper.isprintable(data):\n printableMet = True\n\n if printable and not printableMet:\n continue\n\n if len(keywords) > 0 and minkeywords > 0:\n skip = False\n found = 0\n for keyword in keywords:\n if re.search(r'\\b' + re.escape(keyword) + r'\\b', data, re.I):\n found += 1\n\n if found >= minkeywords:\n foundNots = 0\n notkeywords = predicate.get('not-keywords', [])\n\n if len(notkeywords) > 0:\n for keyword in notkeywords:\n if re.search(r'\\b' + re.escape(keyword) + r'\\b', data, re.I):\n foundNots += 1\n\n if foundNots == 0:\n self.grade += self.gradeFoundIndicator(indicator, data, color=predColor)\n return indicator\n\n if magicOut == 'data':\n return ''\n\n return magicOut\n\n def lookForIOCs(self):\n binary = self.collectEntries('Binary')\n customActions = self.collectEntries('CustomAction')\n i = 0\n\n streams = self.collectEntries('_Streams')\n if len(streams) == 0:\n self.report.append({\n 'name' : Logger.colorize('Missing _Streams', 'yellow'),\n 'location' : f'_Streams table',\n 'context' : '',\n 'desc' : f'Typically MSIs contain _Streams table referring .CAB archives.\\nThis sample however didn\\'t contain such table, making it unusual/mangled.\\n',\n })\n\n for data in binary:\n i += 1\n sniffed = self.sniffDataType(data['data'], color=True)\n\n if len(sniffed) > 0:\n data['size'] = len(data['data'])\n runByCa = False\n desc = ''\n\n i = 0\n for ca in customActions:\n i += 1\n if ca['source'] == data['name']:\n runByCa = True\n desc = f'\\nThat data will be used during installation by CustomAction {Logger.colorize(i, \"yellow\")}. {Logger.colorize(ca[\"action\"], \"yellow\")}'\n break\n\n if not runByCa:\n self.grade -= 1\n sniffed = Logger.stripColors(sniffed)\n sniffed = Logger.colorize(sniffed, 'yellow')\n desc = '\\nHowever that data doesn\\'t seem to be used in CustomActions, decreasing impact.'\n\n self.report.append({\n 'name' : sniffed,\n 'location' : f'Binary table',\n 'context' : self.printRecord(data),\n 'desc' : f'MSI contains {sniffed} data in Binary table entry {Logger.colorize(str(i), \"yellow\")}. {Logger.colorize(data[\"name\"], \"yellow\")}' + desc,\n })\n\n def processActions(self):\n actions = self.collectEntries('CustomAction')\n execSeq = self.collectEntries('InstallExecuteSequence')\n uiSeq = self.collectEntries('InstallUISequence')\n\n for action in actions:\n self.logger.dbg(f'Parsing CustomAction {action[\"action\"]} ...')\n\n for suspAction, data in MSIDumper.CustomActionTypes.items():\n if action['type'] in data['types']:\n desc = data['desc']\n color = MSIDumper.CustomActionTypes[suspAction].get('color', 'white')\n\n fieldToHighlight = ''\n\n if 'vbscript' in suspAction.lower() or 'jscript' in suspAction.lower():\n if len(action['source']) > 0:\n fieldToHighlight = 'source'\n self.grade += self.gradeFoundIndicator(suspAction, color=color)\n desc += f\".\\nScript is located in {Logger.colorize(action['source'],'yellow')} Binary table record.\"\n\n elif 'run-dll' in suspAction.lower():\n fieldToHighlight = 'source'\n self.grade += self.gradeFoundIndicator(suspAction, color=color)\n desc += f\".\\nDLL is located in {Logger.colorize(action['source'],'yellow')} Binary table record.\"\n \n elif 'run-exe' in suspAction.lower():\n fieldToHighlight = 'source'\n self.grade += self.gradeFoundIndicator(suspAction, color=color)\n desc += f\"\\nEXE is located in {Logger.colorize(action['source'],'yellow')} Binary table record.\"\n\n elif 'set-directory' in suspAction.lower():\n fieldToHighlight = 'target'\n\n elif 'execute' in suspAction.lower():\n fieldToHighlight = 'target'\n self.grade += self.gradeFoundIndicator(suspAction, color=color)\n desc += f\".\\nCommand that will be executed:\\ncmd> {Logger.colorize(action['target'],'red')}\"\n\n foundInSeq = False\n for seq in execSeq:\n if seq['action'] == action['action']:\n foundInSeq = True\n cond = ''\n if len(seq['condition']) > 0:\n cond = f\" with condition:\\n- {Logger.colorize(seq['condition'],'yellow')}\"\n\n desc += f\"\\nThat action is scheduled to run in {Logger.colorize('InstallExecuteSequence','yellow')} table\" + cond + '\\n'\n break\n\n for seq in uiSeq:\n if seq['action'] == action['action']:\n foundInSeq = True\n cond = ''\n if len(seq['condition']) > 0:\n cond = f\" with condition:\\n- {Logger.colorize(seq['condition'],'yellow')}\"\n\n desc += f\"\\nThat action is scheduled to run in {Logger.colorize('InstallUISequence','yellow')} table\" + cond + '\\n'\n break\n\n if not foundInSeq:\n self.grade -= 1\n color = 'yellow'\n desc = '\\nHowever that action doesn\\'t seem to be invoked anywhere, decreasing impact.'\n\n if len(fieldToHighlight) > 0:\n action[fieldToHighlight] = Logger.colorize(action[fieldToHighlight], color)\n\n self.report.append({\n 'name' : Logger.colorize(suspAction, color),\n 'location' : f'CustomAction table',\n 'context' : self.printRecord(action),\n 'desc' : desc\n })\n break\n\n def initYara(self):\n yaraPath = self.options.get('yara', '')\n if len(yaraPath) == 0:\n return None\n\n yaraPath = os.path.abspath(os.path.normpath(yaraPath))\n\n if not os.path.isfile(yaraPath) and not os.path.isdir(yaraPath):\n self.logger.fatal(f'Specified --yara path does not exist.')\n\n rules = None\n try:\n rules = yara.compile(yaraPath)\n except Exception as e:\n self.logger.fatal(f'Could not compile YARA rules. Exception: {e}')\n\n return rules\n\n def yaraScan(self, scanBinary=True, scanActions=True, scanFiles=True):\n matchesReport = []\n rules = self.initYara()\n\n if scanBinary:\n binary = self.collectEntries('Binary')\n output = ''\n\n if len(binary) > 0:\n i = 0\n for elem in binary:\n i += 1\n matches = rules.match(data = elem['data'].encode())\n if matches:\n matchesReport.append({\n 'where' : f'Binary record {Logger.colorize(i, \"yellow\")}. {Logger.colorize(elem[\"name\"], \"yellow\")}',\n 'rules' : '\\n'.join([x.rule for x in matches])\n })\n\n if scanActions:\n actions = self.collectEntries('CustomAction')\n output = ''\n\n if len(actions) > 0:\n i = 0\n for elem in actions:\n sniffed = self.sniffDataType(elem['target'])\n if 'vbscript' not in sniffed.lower() and 'jscript' not in sniffed.lower():\n continue\n i += 1\n matches = rules.match(data = elem['data'])\n if matches:\n matchesReport.append({\n 'where' : f'CustomAction record {Logger.colorize(i, \"yellow\")}. {Logger.colorize(elem[\"name\"], \"yellow\")}',\n 'rules' : '\\n'.join([x.rule for x in matches])\n })\n\n if scanFiles:\n try:\n dirpath = tempfile.mkdtemp()\n self.logger.verbose(f'Extracting all files from MSI into temp dir: {dirpath} ...')\n\n out = self.extractFiles(overrideOutdir = dirpath)\n\n for _file in glob.glob(os.path.join(dirpath, '**/*.*'), recursive=True):\n path = os.path.join(dirpath, _file)\n\n matches = rules.match(path)\n if matches:\n matchesReport.append({\n 'where' : f'File extracted from MSI: {Logger.colorize(os.path.basename(path), \"yellow\")}',\n 'rules' : '\\n'.join([x.rule for x in matches])\n })\n\n except Exception as e:\n self.logger.err(f'Could not extract files from MSI for YARA scanning. Exception: {e}')\n if self.options.get('debug', False):\n raise\n\n finally:\n if os.path.isdir(dirpath):\n shutil.rmtree(dirpath)\n\n if len(matchesReport) > 0:\n output += Logger.colorize(f'[+] Got {len(matchesReport)} YARA rules matches on this MSI:\\n\\n', 'green')\n output += self.printTable('YARA Results', matchesReport)\n\n return output\n\ndef getoptions():\n global logger\n global options\n\n epilog = f'''\n\n------------------------------------------------------\n\n- What can be listed:\n --list CustomAction - Specific table\n --list Registry,File - List multiple tables\n --list stats - Print MSI database statistics\n --list all - All tables and their contents\n --list olestream - Prints all OLE streams & storages. \n To display CABs embedded in MSI try: --list _Streams\n --list cabs - Lists embedded CAB files\n --list binary - Lists binary data embedded in MSI for its own purposes.\n That typically includes EXEs, DLLs, VBS/JS scripts, etc\n\n- What can be extracted:\n --extract all - Extracts Binary data, all files from CABs, scripts from CustomActions\n --extract binary - Extracts Binary data\n --extract files - Extracts files\n --extract cabs - Extracts cabinets\n --extract scripts - Extracts scripts\n\n------------------------------------------------------\n\n'''\n\n usage = '\\nUsage: msidump.py [options] \\n'\n opts = argparse.ArgumentParser(\n usage=usage,\n formatter_class=argparse.RawDescriptionHelpFormatter,\n epilog=textwrap.dedent(epilog)\n )\n\n req = opts.add_argument_group('Required arguments')\n req.add_argument('infile', help='Input MSI file (or directory) for analysis.')\n \n opt = opts.add_argument_group('Options')\n opt.add_argument('-q', '--quiet', default=False, action='store_true', help='Surpress banner and unnecessary information. In triage mode, will display only verdict.')\n opt.add_argument('-v', '--verbose', default=False, action='store_true', help='Verbose mode.')\n opt.add_argument('-d', '--debug', default=False, action='store_true', help='Debug mode.')\n opt.add_argument('-N', '--nocolor', default=False, action='store_true', help='Dont use colors in text output.')\n opt.add_argument('-n', '--print-len', default=MSIDumper.DefaultTableWidth, type=int, help='When previewing data - how many bytes to include in preview/hexdump. Default: 128')\n opt.add_argument('-f', '--format', default='text', choices=['text', 'json', 'csv'], help='Output format: text, json, csv. Default: text')\n opt.add_argument('-o', '--outfile', metavar='path', default='', help='Redirect program output to this file.')\n opt.add_argument('-m', '--mime', default=False, action='store_true', help='When sniffing inner data type, report MIME types')\n \n mod = opts.add_argument_group('Analysis Modes')\n mod.add_argument('-l', '--list', metavar='what', default='', help='List specific table contents. See help message to learn what can be listed.')\n mod.add_argument('-x', '--extract', metavar='what', default='', help='Extract data from MSI. For what can be extracted, refer to help message.')\n\n spec = opts.add_argument_group('Analysis Specific options')\n spec.add_argument('-i', '--record', metavar='number|name', type=str, default=-1, help='Can be a number or name. In --list mode, specifies which record to dump/display entirely. In --extract mode dumps only this particular record to --outdir')\n spec.add_argument('-O', '--outdir', metavar='path', default='', help='When --extract mode is used, specifies output location where to extract data.')\n spec.add_argument('-y', '--yara', metavar='path', default='', help='Path to YARA rule/directory with rules. YARA will be matched against Binary data, streams and inner files')\n\n args = opts.parse_args()\n options.update(vars(args))\n\n logger = Logger(options)\n\n if len(args.list) > 0:\n if args.list.lower() not in [x.lower() for x in MSIDumper.ListModes + MSIDumper.KnownTables] and ',' not in args.list:\n logger.err(f'WARNING: Requested {args.list} table is not recognized: parser will probably crash!')\n\n args.infile = os.path.abspath(os.path.normpath(args.infile))\n\n if not os.path.isfile(args.infile) and not os.path.isdir(args.infile):\n logger.fatal(f'--infile does not exist!')\n\n exclusive = sum([len(args.list) > 0, len(args.extract) > 0])\n if exclusive > 1:\n logger.fatal(f'--list and --extract are mutually exclusive options. Pick one.')\n\n if len(args.extract) > 0 and len(args.outdir) == 0:\n logger.fatal('-O/--outdir telling where to extract files to is required when working in --extract mode.')\n\n options.update(vars(args))\n return args\n\n@atexit.register\ndef goodbye():\n try:\n colorama.deinit()\n except:\n pass\n\ndef terminalWidth():\n n = shutil.get_terminal_size((80, 20)) # pass fallback\n return n.columns\n\ndef banner():\n print(f'''\n _ _ \n _ __ ___ ___(_) __| |_ _ _ __ ___ _ __ \n | '_ ` _ \\/ __| |/ _` | | | | '_ ` _ \\| '_ \\ \n | | | | | \\__ \\ | (_| | |_| | | | | | | |_) |\n |_| |_| |_|___/_|\\__,_|\\__,_|_| |_| |_| .__/ \n |_| \n version: {Logger.colorize(VERSION, \"green\")}\n author : Mariusz Banach (mgeeky, @mariuszbit)\n binary-offensive.com\n''')\n\ndef processFile(args, path):\n msir = MSIDumper(options, logger)\n\n if not msir.open(path):\n logger.err(f'Could not open database (use -d to learn more): {path}')\n return ''\n\n report = ''\n if not args.quiet and args.format == 'text':\n report += f'{Logger.colorize(\"[+]\",\"green\")} Analyzing : {path}\\n\\n'\n\n if len(args.list) > 0:\n report += msir.listTable(args.list)\n\n elif len(args.extract) > 0:\n report += msir.extract(args.extract)\n\n else:\n rep = msir.analyse()\n\n if len(args.yara) > 0:\n rep += '\\n\\n' + msir.yaraScan()\n\n if not args.quiet:\n report += str(rep)\n\n if args.format == 'text':\n report += '\\n\\n' + msir.verdict.strip() + '\\n'\n\n elif args.format == 'text':\n verd = msir.verdict.strip()\n pos = verd.find(':')\n if pos != -1:\n verd = verd[pos+1:].strip()\n\n report += verd + ' : ' + path\n\n if args.format == 'text':\n logger.ok(f'Database processed : {path}')\n msir.close()\n\n return report\n\ndef processDir(args, infile):\n report = ''\n\n logger.verbose(f'Process files from directory: {infile}')\n\n for file in glob.glob(os.path.join(infile, '**/**'), recursive=True):\n path = os.path.join(infile, file)\n if os.path.isfile(path):\n try:\n report += processFile(args, path)\n report += '\\n\\n'\n\n except Exception as e:\n logger.err('Analysis of \"{}\" failed. Exception: {}'.format(\n path, str(e)\n ))\n\n return report\n\ndef main():\n global options\n args = getoptions()\n if not args:\n return False\n\n if not args.quiet and args.format == 'text':\n banner()\n\n if len(args.outfile) > 0:\n options['nocolor'] = True\n\n options['max_width'] = terminalWidth()\n\n if os.path.isfile(args.infile):\n report = processFile(args, args.infile)\n\n else:\n report = processDir(args, args.infile)\n\n if len(args.outfile) > 0:\n with open(args.outfile, 'wb') as f:\n rep = Logger.stripColors(report)\n f.write(rep.encode())\n else:\n print(report)\n\nif __name__ == '__main__':\n main()\n" }, { "path": "requirements.txt", "content": "olefile\ncolorama\nyara-python\nprettytable>=3.5\npefile\ncabarchive\npywin32\npython-magic\npython-magic-bin; sys_platform == \"win32\" or sys_platform == \"darwin\"\n\n# ssdeep is optional\n#ssdeep " }, { "path": "test-cases/README.md", "content": "## msidump test cases\n\n- `sample1-run-autoruns64.msi.bin` - launches MS Sysinternals Autoruns64.exe from `C:\\Windows\\Installer\\MSXXXX.msi`\n- `sample2-run-calc-script.msi.bin` - executes VBScript that runs `calc` over `Wscript.Shell.Exec` method\n- `sample3-run-calc-shellcode-via-dotnet.msi.bin` - bundles specially crafted CustomAction .NET DLL, that when executed, runs shellcode which spawns `calc`\n- `sample4-customaction-run-calc.msi.bin` - simple MSI that runs system commands after installation is complete, here runs `calc`\n- `putty-backdoored.msi.bin` - runs `calc` during PuTTY installation\n\nAll these installers install themselves to `%LOCALAPPDATA%\\VcRedist` directory.\n\nYou can uninstall them with:\n\n```\nmsiexec /q /x file.msi\n```\n" } ]