Repository: roglew/guppy-proxy Branch: master Commit: 01df16be71dd Files: 26 Total size: 30.2 MB Directory structure: gitextract_l9ewhzbc/ ├── .gitignore ├── LICENSE ├── README.md ├── guppyproxy/ │ ├── __init__.py │ ├── config.py │ ├── decoder.py │ ├── gui.py │ ├── gup.py │ ├── hexteditor.py │ ├── interceptor.py │ ├── macros.py │ ├── proxy.py │ ├── repeater.py │ ├── reqlist.py │ ├── reqtree.py │ ├── reqview.py │ ├── settings.py │ ├── shortcuts.py │ └── util.py ├── img/ │ └── shark.icns ├── install.sh ├── puppyrsc/ │ ├── NOTE.md │ ├── puppy.linux32 │ ├── puppy.linux64 │ └── puppy.osx └── setup.py ================================================ FILE CONTENTS ================================================ ================================================ FILE: .gitignore ================================================ *.pyc data.db .coverage .cache tests/.cache .DS_Store TAGS config.json build/* *.egg-info/* .#* *notes* *.org start ================================================ FILE: LICENSE ================================================ MIT License Copyright (c) 2018 Rob Glew Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ================================================ FILE: README.md ================================================ # Guppy Proxy The Guppy Proxy is an intercepting proxy for performing web application security testing. Its features are often similar to, or straight up rippoffs from [Burp Suite](https://portswigger.net/burp/). However, Burp Suite has its own issues (search, licensing) which led to the creation of Guppy. ![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_main.png) # Installation ## Dependencies Make sure the following commands are available: * `python3` * `pip` * `virtualenv` (can be installed with pip) ## Installing ### Mac 1. Download the .app of version of guppy [available here](https://guppydist.s3-us-west-2.amazonaws.com/GuppyProxy-0.0.15.zip) 1. Start the application 1. Add the CA cert in `~/.guppy/certs` to your browser as a CA 1. Configure your browser to use `localhost:8080` as a proxy 1. Navigate to a site and look at the history in the main window ### Linux / Alternative for Mac 1. Clone this repo somewhere it won't get deleted: `git clone https://github.com/roglew/guppy-proxy.git` 1. `cd /path/to/guppy-proxy` 1. `./install.sh` to use pre-built binary or `./install.sh -p` to compile the go component from source (requires a [go installation](https://golang.org/doc/install)) 1. Test that the application starts up and generate certs: `./start` (keep the window open and continue to test it works) 1. Copy/symlink the generated `start` script somewhere in your PATH (i.e. `~/bin` if you have that included) and rename it to `guppy` if you want 1. Add the CA cert in `~/.guppy/certs` to your browser as a CA 1. Configure your browser to use `localhost:8080` as a proxy 1. Navigate to a site and look at the history in the main window ## Updating 1. Navigate to the guppy-proxy folder with this repo in it 1. `git pull` to pull the latest version 1. run `./install.sh` again The same start script as before should still work ## Uninstalling 1. Delete the guppy-proxy directory you made during installation 1. Delete `~/.guppy` 1. Remove the start script from wherever you put it # How to Use Guppy ## History View ![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_main.png) ![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_pretty_view.png) ![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_tree.png) The first thing you see when you open Guppy is the history view. As requests pass through the proxy they are displayed in the lower half of the window. You can click a request to view the full request/response in the windows on the upper half or right click them for more options. The tabs on the upper half will let you view additional information about the selected request: * Messages - The full request/response * Info - A list of values associated with the message * Tags - Lets you view/edit the tags currently associated with the request The bottom half has tabs which relate to all of the requests that have been recorded by the proxy: * List - A list of all of the requests that have been recorded by the proxy * Tree - A site map of all of the endpoints visited * Filters - An advanced search interface which is described below in the Filters section ## Filters and Search ![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_search.png) Guppy's main selling point over other similar proxies is its search. You can search for a wide variety of fields within a request or response and apply more than one search condition at the same time. This allows you to perform complex searches over your history so that you can always find the request that you want. You would be surprised what you can find when searching for paths, headers, and body contents. For example you can find potential CSRF targets by finding requests which are not GET requests and also do not have a header with "CSRF" in it. How to apply a filter to your search: 1. Select the field you want to search by 1. Select how you want to search it (whether it contains a value, matches a regexp, is an exact value, etc) 1. Enter the value to search by in the text box 1. Click "Ok" or press enter in the text box Once you apply a filter, the "list" and "tree" tabs will only include requests which match ALL of the active filters. In addition, you can apply different filters for the key and value of key/value fields (such as headers or request parameters). This can be done by: 1. Select a key/value field such as "Rsp. Header" or "URL Param" 1. Click the "+" button on the right 1. Enter the filter for the key on the left and the filter for the value on the right 1. Click "Ok" or press enter in one of the text boxes ![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_search_kv.png) And that's it! The filter tab has the following additional controls: 1. Clear - Delete all active filters 1. Pop - Delete the most recent filter 1. Scope - Set the active search to your project's scope (see below) 1. Save Scope - Set your project's scope to the currently active filters (see below) 1. Apply a built-in filter dropdown - Guppy has a list of commonly useful filters. Select one from this list to apply it ### Text Filter Entry Along with the provided dropdowns you can manually type in a filter by clicking the `>` button. In some cases it may be faster to type your filter out rather than clicking on dropdowns. In addition it allows you to create filter statements that contain an `OR` and will pass a request that matches any one of the given filters. In fact, all the dropdown input does is generate these strings for you. Most filter strings have the following format: ``` ``` Where `` is some part of the request/response, `` is some comparison to ``. For example, if you wanted a filter that only matches requests to `target.org`, you could use the following filter string: ``` host is target.org field = "host" comparer = "is" value = "target.org" ``` For fields that are a list of key/value pairs (headers, get params, post params, and cookies) you can use the following format: ``` [ ] ``` This is a little more complicated. If you don't give comparer2/value2, the filter will pass any pair where the key or the value matches comparer1 and value1. If you do give comparer2/value2, the key must match comparer1/value1 and the value must match comparer2/value2 For example: ``` Filter A: cookie contains Session Filter B: cookie contains Session contains 456 Filter C: inv cookie contains Ultra Cookie: SuperSession=abc123 Matches A and C but not B Cookie: UltraSession=abc123456 Matches both A and B but not C ``` #### List of fields | Field Name | Aliases | Description | Format | |:--------|:------------|:-----|:------| | all | all | Anywhere in the request, response, or a websocket message | String | | reqbody | reqbody, reqbd, qbd, qdata, qdt | The body of the request | String | | rspbody | rspbody, rspbd, sbd, sdata, sdt | The body of the response | String | | body | body, bd, data, dt | The body in either the request or the response | String | | wsmessage | wsmessage, wsm | In a websocket message | String | | method | method, verb, vb | The request method (GET, POST, etc) | String | | host | host, domain, hs, dm | The host that the request was sent to | String | | path | path, pt | The path of the request | String | | url | url | The full URL of the request | String | | statuscode | statuscode, sc | The status code of the response (200, 404, etc) | String | | tag | tag | Any of the tags of the request | String | | reqheader | reqheader, reqhd, qhd | A header in the request | Key/Value | | rspheader | rspheader, rsphd, shd | A header in the response | Key/Value | | header | header, hd | A header in the request or the response | Key/Value | | param | param, pm | Either a URL or a POST parameter | Key/Value | | urlparam | urlparam, uparam | A URL parameter of the request | Key/Value | | postparam | postparam, pparam | A post parameter of the request | Key/Value | | rspcookie | rspcookie, rspck, sck | A cookie set by the response | Key/Value | | reqcookie | reqcookie, reqck, qck | A cookie submitted by the request | Key/Value | | cookie | cookie, ck | A cookie sent by the request or a cookie set by the response | Key/Value | #### List of comparers | Field Name | Aliases | Description | |:--------|:------------|:-----| | is | is | Exact string match | | contains | contains, ct | A contain B is true if B is a substring of A | | containsr | containsr, ctr | A containr B is true if A matches regexp B | | leneq | leneq | A Leq B if A's length equals B (B must be a number) | | lengt | lengt | A Lgt B if A's length is greater than B (B must be a number ) | | lenlt | lenlt | A Llt B if A's length is less than B (B must be a number) | #### Special form filters A few filters don't conform to the field, comparer, value format. You can still negate these. | Format | Aliases | Description | |:--|:--|:--| | invert | invert, inv | Inverts a filter string. Anything that matches the filter string will not pass the filter. | Examples: ``` Show state-changing requests inv method is GET Show requests without a csrf parameter inv param ct csrf ``` #### Using OR If you want to create a filter that will pass a request if it matches any of one of a few filters you can create `OR` statements. This is done by entering in each filter on the same line and separating them with an `OR` (It's case sensitive!). Examples: ``` Show requests to target.org or example.com: host is target.org OR host is example.com Show requests that either are to /foobar or have foobar in the response or is a 404 path is /foobar OR sbd ct foobar OR sc is 404 ``` ### Scope The scope of your project describes which requests should be recorded as they pass through the proxy. Guppy allows you to define a set of filters which describe which requests are in scope. For example, if your scope is just `host ctr example.com$` only requests to example.com will be recorded in history. To set the scope of your project: 1. Enter the filters you want to be your scope 1. Press the "Save Scope" button And you're done! Requests that do not match this set of filters will no longer be saved. You can also set your current search to your scope by clicking the "Scope" button. The scope can be deleted by pressing the "Clear" button to delete all active filters and then clicking "Save Scope". # Repeater ![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_repeater.png) The repeater lets you repeatedly tweak and submit a request. You can use a request in the repeater by: 1. Find the request you which to resubmit in the history list view 1. Right click the request and click "Send to Repeater" 1. Navigate to the repeater tab 1. Edit the request on the left 1. Click the submit button When you click submit: * The request will be submitted * The request and response will be saved in history * Any tags under the "tag" tab will be applied to the request # Interceptor ![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_interceptor.png) The interceptor lets you edit requests and responses as they pass through the proxy. To use this: 1. Navigate to the interceptor tab 1. Click the "Int. Requests" and/or the "Int. Responses" buttons 1. Wait for a request to pass through the proxy 1. Edit the message in the text box then click "Forward" to forward the edited message or click "Cancel" to just drop the message altogether # Decoder ![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_decoder.png) The decoder allows you to perform common encoding/decoding actions. You use the decoder by: 1. Paste the data that you want to encode/decode 1. Select how you wish to encode/decode it 1. Press "Go!" The text will be processed and it will appear in the same text box. Easy! # Macros Guppy includes support for loading and executing Python scripts in order to allow for more complex attacks than can be performed by hand with the repeater. It is worth noting that **this feature is not user friendly. Use it at your own risk.** No attempt is made to make this feature user-friendly, stable, or good. The main reason it exists is to make it easier to write python scripts which integrate with Guppy history and to provide some way to extend Guppy's features without a pull request. If you haven't been scared away yet, read on. There are two types of macros that you can write: * Active macros: Take requests as an input, make more requests, edit the input requests, etc, then output a new set of requests for review * Intercepting macros: Modify requests and responses as they pass through the proxy Most features that you want will fall into one of those categories. Both macros are created by creating a `.py` file and defining specific functions which will be run when the macro is executed. For example an active macro must define `run_macro` and an intercepting macro must define `mangle_request` and/or `mangle_response`. See their respective sections below for more details. ## The API Unfortunately since this feature was pretty much just thrown together for my own use, the documentation is looking at the source. Hopefully it will become more stable once Guppy development slows down, but for now you'll have to look at the relevant classes to figure out how to do stuff on your own. The following classes are the most important when writing a macro: ### MacroClient `MacroClient` is defined in `(guppyproxy/macro.py)` and is the interface that macros use to submit requests, save requests to history, and produce output. At the time of writing the class provides: ``` MacroClient.submit(req, save=False): Submits a request to the server and sets req.response to the response. req is the HTTPRequest to submit. req.dest_host, req.dest_port, and req.use_tls will be used to determine the location to submit the request to MacroClient.save(req): Permenantly saves an HTTPRequest to history MacroClient.output(s): Prints a string to the output tab in the macros interface MacroClient.output_req(req): Adds a request to the output request table in the macros interface MacroClient.new_request(method="GET", path="/", proto_major=1, proto_minor=1, headers=None, body=bytes(), dest_host="", dest_port=80, use_tls=False, tags=None): Creates a new HTTPRequest from scratch that can be submitted with the client ``` ### HTTPRequest and HTTPResponse `HTTPRequest` and `HTTPResponse` are defined in `guppyproxy/proxy.py`. These classes represent HTTP messages. `HTTPRequest` contains both the contents of the message and information about its intended destination (host, port, whether to use TLS). Below are a few examples on how to use these classes, however for more deails you will need to consult `proxy.py`: ```python req = HTTPRequest() rsp = HTTPResponse() req2 = req.copy() # Copy a request rsp2 = rsp.copy() # Copy a response # Refer to the messages associated with a messages rsp3 = req.response # Response to a request, will be `None` if there was no response unm = req.unmangled # Unmangled version of a request, is `None` if none exist unm2 = rsp.unmangled # Unmangled version of a response, is `None` if none exist # Get the full message of an object (is a bytes()) full_req = req.full_message() full_rsp = rsp.full_message() # Get timing info for a request tstart = req.time_start # datetime.datetime when the request was made tend = req.time_end # datetime.datetime when the request's response was received # Get destination info from a request dest_host = req.dest_host dest_port = req.dest_port use_tls = req.use_tls # Get/set the method of a request m = req.method # Get the method of the request req.method = "POST" # Set the method of the request # Get/set url info of a request requrl = req.full_url() # get the full URL of a request path = req.url.path # get the path of a request req.url.path = "/foo/bar/baz" # set the path of a request v = req.url.get_param("foo") # get the value of the "foo" URL parameter req.url.set_param("foo", "bar") # set the value of the "foo" URL parameter to "bar" req.url.add_param("foo", "bar2") # add a URL parameter allowing duplicates req.url.del_param("foo") # delete a url parameter [(k, v) for k, v in req.url.param_iter] # iterate over all the key/value pairs in the URL parameters frag = req.url.fragment # get the fragment of the url (the bit after the #) req.url.fragment = "frag" # set the url fragment of the request # Manage headers in a message req.headers.set("Foo", "Bar") # set a header, repalcing existing value hd = req.headers.get("Foo") # get the value of a header (for duplicates, returns first value) req.headers.add("Foo", "Bar2") # add a header without replacing an existing one pairs = req.headers.pairs() # returns all the key/value pairs of the headers in the message req.headers.delete("Foo") # delete a header req.headers.dict() # Returns a dict of the headers in the form of {"key1": ["val1", "val2"], "key2": ["val3", "val4"]} # Same for responses rsp.headers.set("Foo", "Bar") hd = rsp.headers.get("Foo") rsp.headers.add("Foo", "Bar2") pairs = rsp.headers.pairs() rsp.headers.delete("Foo") rsp.headers.dict() # Manage body of a message req.body = "foo=bar" # set the body of the message to a string req.body = b"\x01\x02\x03" # set the body to bytes bd = req.body # Get the value of the body (always is bytes()) # Same for responses rsp.body = "foo=bar" rsp.body = b"\x01\x02\x03" bd = rsp.body # Manage POST parameters of a request params = req.parameters() # Returns a dict of the POST parameters in the form of {"key1": ["val1", "val2"], "key2": ["val3", "val4"]} [(k, v) for k, v in req.param_iter()] # Iterate through all the key/value pairs of the request parameters req.set_param("Foo", "Bar") # Set the "Foo" parameter to "Bar" req.add_param("Foo", "Bar2") # Add a POST parameter to the request allowing duplicates req.del_param("Foo") # Delete a parameter from the request # NOTE: Setting a POST parameter will not change the request method to POST # Managing the cookies of a message cookie = req.cookies() # Returns an http.cookies.BaseCookie representing the request's cookies req.set_cookie("foo", "bar") # set a cookie in the request req.del_cookie("foo") # delete a cookie from the request [(k, v) for k, v in req.cookie_iter()] # Iterate over the key/value pairs of the cookies in a request req.set_cookies({"cookie1": "val1", "cookie2": "val2"}) # Set the cookies in the request req.set_cookies(req2) # Set the requests on req to the cookies in req2 req.add_cookies({"cookie1": "val1", "cookie2": "val2"}) # Add cookies to the request replacing existing values req.add_cookies(req2) # Add cookies from req2 to the request replacing existing values # Same for responses cookie = rsp.cookies() rsp.set_cookie("foo", "bar") rsp.del_cookie("foo") [(k, v) for k, v in rsp.cookie_iter()] rsp.set_cookies({"cookie1": "val1", "cookie2": "val2"}) rsp.set_cookies(rsp2) rsp.add_cookies({"cookie1": "val1", "cookie2": "val2"}) rsp.add_cookies(rsp2) # Manage tags of a request hastag = ("tagname" in req.tags) # check if a request has a tag req.tags.add("tagname") # add a tag to a request req.tags.remove("tagname") # remove a tag from the request # NOTE: req.tags is a regular set() and you can do whatever you want to it ``` ## Macro Arguments ![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_macro_args.png) Both active and intercepting macros can optionally have Guppy prompt for a set of arguments before running. These arguments will be passed as a dict in the `args` variable when calling the relevant function. A macro can request arguments by defining a `get_args` function and returning a list of strings. For example if a macro defines the following `get_args` function: ```python def get_args(): return ["foo", "bar"] ``` the proxy will prompt for values for foo and bar. If the user enters "FOOARG" and "BARARG" for the values `args` will have a value of: ```python {"foo": "FOOARG", "bar": "BARARG"} ``` See below for examples on how to use arguments in macros. If `get_args` is not defined, `None` will be passed in for `args`. ## Active Macros ![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_macro_active_in.png) Active macros are a Python script that define a `run_macro` function that takes in two arguments. A `MacroClient` (as defined in `guppyproxy/macros.py`) and a list of requests (`HTTPRequest` and `HTTPResponse` are defined in `guppyproxy/proxy.py`). The following is an example of a macro that resubmits all of the input requests but adds a new header: ```python # addheader.py def get_args(): return ["header_key", "header_val"] def run_macro(client, args, reqs): for req in reqs: client.output("Submitting request to %s..." % req.full_url()) req.headers.set(args["header_key"], args["header_val"]) client.submit(req) client.output_req(req) ``` Macros such as this can be used for things such as testing auth controls or brute forcing paths/filenames. ## Intercepting Macros ![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_macro_int.png) Intercepting macros are used to look at/modify requests as they pass through the proxy. This is done by defining `mangle_request` and/or `mangle_response`: ``` mangle_request(client, req): Takes in a client and an HTTPRequest and returns an HTTPRequest. The returned HTTPRequest will be sent to the server instead of the original. mangle_response(client, req, rsp): Takes in a client, HTTPRequest, and HTTResponse and returns an HTTPResponse. The returned HTTPResponse will be sent to the browser instead of the original. ``` As an example, the following macro will ask for a find/replace value. When run, it will set the `session` cookie in the request to `bar` before submitting it to the server and then perform the given find and replace on the body of the response. ```python # intexample.py def get_args(): return ["find", "replace"] def mangle_request(client, args, req): req.set_cookie("session", "bar") return req def mangle_response(client, args, req, rsp): rsp.body = rsp.body.replace(args['find'].encode(), args['replace'].encode()) return rsp ``` # Settings ![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_settings.png) This tab allows you to edit your proxy settings. It lets you select a file to store your history/settings in and configure what ports the proxy listens on. It also allows you to configure an upstream proxy to use. You can add a listener by entering the interface and port into the text boxes and clicking the "+" button. They can be deleted by selecting them from the list and clicking the "-" button. You can also specify settings for an upstream proxy by checking the "Use Proxy" box, filling out the appropriate info, and clicking "confirm". ## Data Files Your entire request history and your settings can be stored in a data file on disk. This allows you to save your work for later and even send your work to someone else. You can start a new project with a new data file by clicking the "New" button in the settings tab. Once you do this, your settings, scope, and all the messages that pass through the proxy will be saved to the specified file. You can also load an existing project by using the "Open" button. Finally, you can specify a data file by typing the path into the text box and clicking "Go!" # Keybindings Guppy has the following keybindings: | Key | Action | |:--------|:------------| | `Ctrl+J` | Navigate to request list | | `Ctrl+T` | Navigate to tree view | | `Ctrl+R` | Navigate to repeater | | `Ctrl+N` | Navigate to interceptor | | `Ctrl+D` | Navigate to decoder | | `Ctrl+U` | Navigate to filter text input | | `Ctrl+I` | Navigate to filter dropdown input | | `Ctrl+P` | Navigate to filters and pop most recent filter | | `Ctrl+Shift+D` | Navigate to decoder and fill with clipboard | | `Ctrl+Shift+N` | Create new datafile | | `Ctrl+Shift+O` | Open existing datafile | ================================================ FILE: guppyproxy/__init__.py ================================================ ================================================ FILE: guppyproxy/config.py ================================================ import copy import json default_config = """{ "listeners": [ {"iface": "127.0.0.1", "port": 8080} ], "proxy": {"use_proxy": false, "host": "", "port": 0, "is_socks": false} }""" class ProxyConfig: PLUGIN_KEY = "guppy_config" def __init__(self): self._listeners = [('127.0.0.1', 8080, None)] self._proxy = {'use_proxy': False, 'host': '', 'port': 0, 'is_socks': False} def loads(self, js): config_info = json.loads(js) self._set_config(config_info) def dumps(self): listeners = [] for l in self._listeners: listener = {"host": l[0], "port": l[1]} listeners.append(listener) _config_info = {"listeners": listeners, "proxy": self._proxy} return json.dumps(_config_info) def load(self, fname): try: with open(fname, 'r') as f: config_info = json.loads(f.read()) except IOError: config_info = json.loads(default_config) with open(fname, 'w') as f: f.write(default_config) self._set_config(config_info) def _set_config(self, config_info): # Listeners if 'listeners' in config_info: self._parse_listeners(config_info['listeners']) if 'proxy' in config_info: self._proxy = config_info['proxy'] def _parse_listeners(self, listeners): self._listeners = [] for info in listeners: if 'port' in info: port = info['port'] else: port = 8080 if 'interface' in info: iface = info['interface'] elif 'iface' in info: iface = info['iface'] else: iface = '127.0.0.1' if "transparent" in info: trans_info = info['transparent'] transparent_dest = (trans_info.get('host', ""), trans_info.get('port', 0), trans_info.get('use_tls', False)) else: transparent_dest = None self._listeners.append((iface, port, transparent_dest)) def set_listeners(self, listeners): self._listeners = listeners @property def listeners(self): return copy.deepcopy(self._listeners) @listeners.setter def listeners(self, val): self._parse_listeners(val) @property def proxy(self): # don't use this, use the getters to get the parsed values return self._proxy @proxy.setter def proxy(self, val): self._proxy = val @property def use_proxy(self): if self._proxy is None: return False if 'use_proxy' in self._proxy: if self._proxy['use_proxy']: return True return False @property def proxy_host(self): if self._proxy is None: return '' if 'host' in self._proxy: return self._proxy['host'] return '' @property def proxy_port(self): if self._proxy is None: return '' if 'port' in self._proxy: return self._proxy['port'] return '' @property def proxy_username(self): if self._proxy is None: return '' if 'username' in self._proxy: return self._proxy['username'] return '' @property def proxy_password(self): if self._proxy is None: return '' if 'password' in self._proxy: return self._proxy['password'] return '' @property def use_proxy_creds(self): return ('username' in self._proxy or 'password' in self._proxy) @property def is_socks_proxy(self): if self._proxy is None: return False if 'is_socks' in self._proxy: if self._proxy['is_socks']: return True return False ================================================ FILE: guppyproxy/decoder.py ================================================ import html import base64 import urllib import json from guppyproxy.util import display_error_box from guppyproxy.hexteditor import ComboEditor from PyQt5.QtWidgets import QWidget, QHBoxLayout, QVBoxLayout, QComboBox, QPlainTextEdit, QPushButton from PyQt5.QtCore import pyqtSlot, pyqtSignal from datetime import datetime class DecodeError(Exception): pass def asciihex_encode_helper(s): return ''.join('{0:x}'.format(c) for c in s).encode() def asciihex_decode_helper(s): ret = [] try: for a, b in zip(s[0::2], s[1::2]): c = chr(a) + chr(b) ret.append(chr(int(c, 16))) return ''.join(ret).encode() except Exception as e: raise DecodeError("Unable to decode asciihex") def base64_decode_helper(s): s = s.decode() for i in range(0, 8): try: s_padded = base64.b64decode(s + '=' * i) return s_padded except Exception as e2: pass raise DecodeError("Unable to base64 decode string: %s" % s) def url_decode_helper(s): bs = s.decode() return urllib.parse.unquote(bs).encode() def url_encode_helper(s): bs = s.decode() return urllib.parse.quote_plus(bs).encode() def html_encode_helper(s): return ''.join(['&#x{0:x};'.format(c) for c in s]).encode() def html_decode_helper(s): return html.unescape(s.decode()).encode() def pp_json(s): d = json.loads(s.strip()) return json.dumps(d, indent=2, sort_keys=True).encode() def decode_jwt(s): # in case they paste the whole auth header or the token with "bearer" s = s.strip() fields = s.split(b' ') s = fields[-1].strip() parts = s.split(b'.') ret = b'' for part in parts: try: ret += base64_decode_helper(part.decode()) + b'\n\n' except: ret += b"[error decoding]\n\n" return ret def decode_unixtime(s): ts = int(s) dfmt = '%b %d, %Y %I:%M:%S %p' try: return datetime.utcfromtimestamp(ts).strftime(dfmt).encode() except ValueError: ts = ts/1000 return datetime.utcfromtimestamp(ts).strftime(dfmt).encode() class DecoderWidget(QWidget): def __init__(self): QWidget.__init__(self) layout = QVBoxLayout() self.decoder_input = DecoderInput() layout.addWidget(self.decoder_input) self.setLayout(layout) self.layout().setContentsMargins(0, 0, 0, 0) class DecoderInput(QWidget): decodeRun = pyqtSignal(bytes) decoders = { "encode_b64": ("Encode Base64", base64.b64encode), "decode_b64": ("Decode Base64", base64_decode_helper), "encode_ah": ("Encode Asciihex", asciihex_encode_helper), "decode_ah": ("Decode Asciihex", asciihex_decode_helper), "encode_url": ("URL Encode", url_encode_helper), "decode_url": ("URL Decode", url_decode_helper), "encode_html": ("HTML Encode", html_encode_helper), "decode_html": ("HTML Decode", html_decode_helper), "decode_unixtime": ("Format Unix Timestamp", decode_unixtime), "pp_json": ("Pretty-Print JSON", pp_json), "decode_jwt": ("Decode JWT Token", decode_jwt), } def __init__(self, *args, **kwargs): QWidget.__init__(self) layout = QVBoxLayout() tool_layout = QHBoxLayout() self.editor = ComboEditor(pretty_tab=False, enable_pretty=False) self.encode_entry = QComboBox() encode_button = QPushButton("Go!") encode_button.clicked.connect(self.encode) for k, v in self.decoders.items(): self.encode_entry.addItem(v[0], k) layout.addWidget(self.editor) tool_layout.addWidget(self.encode_entry) tool_layout.addWidget(encode_button) tool_layout.addStretch() layout.addLayout(tool_layout) self.setLayout(layout) self.layout().setContentsMargins(0, 0, 0, 0) @pyqtSlot() def encode(self): text = self.editor.get_bytes() encode_type = self.encode_entry.itemData(self.encode_entry.currentIndex()) encode_func = DecoderInput.decoders[encode_type][1] try: encoded = encode_func(text) except Exception as e: display_error_box("Error processing string:\n" + str(e)) return self.editor.set_bytes(encoded) ================================================ FILE: guppyproxy/gui.py ================================================ import random from guppyproxy.reqlist import ReqBrowser, ReqListModel from guppyproxy.repeater import RepeaterWidget from guppyproxy.interceptor import InterceptorWidget from guppyproxy.decoder import DecoderWidget from guppyproxy.settings import SettingsWidget from guppyproxy.shortcuts import GuppyShortcuts from guppyproxy.macros import MacroWidget from PyQt5.QtWidgets import QWidget, QTabWidget, QVBoxLayout, QTableView from PyQt5.QtCore import Qt, QTimer, QObject, pyqtSlot class GuppyWindow(QWidget): titles = ( "Guppy Proxy", ) def __init__(self, client): QWidget.__init__(self) self.client = client self.delayTimeout = 100 self._resizeTimer = QTimer(self) self._resizeTimer.timeout.connect(self._delayedUpdate) self.setFocusPolicy(Qt.StrongFocus) self.shortcuts = GuppyShortcuts(self) self.tabWidget = QTabWidget() self.repeaterWidget = RepeaterWidget(self.client) self.interceptorWidget = InterceptorWidget(self.client) self.macroWidget = MacroWidget(self.client) self.historyWidget = ReqBrowser(self.client, repeater_widget=self.repeaterWidget, macro_widget=self.macroWidget, is_client_context=True, update=True) self.decoderWidget = DecoderWidget() self.settingsWidget = SettingsWidget(self.client) self.settingsWidget.datafileLoaded.connect(self.historyWidget.reset_to_scope) self.history_ind = self.tabWidget.count() self.tabWidget.addTab(self.historyWidget, "History") self.repeater_ind = self.tabWidget.count() self.tabWidget.addTab(self.repeaterWidget, "Repeater") self.interceptor_ind = self.tabWidget.count() self.tabWidget.addTab(self.interceptorWidget, "Interceptor") self.decoder_ind = self.tabWidget.count() self.tabWidget.addTab(self.decoderWidget, "Decoder") self.macro_ind = self.tabWidget.count() self.tabWidget.addTab(self.macroWidget, "Macros") self.settings_ind = self.tabWidget.count() self.tabWidget.addTab(self.settingsWidget, "Settings") self.mainLayout = QVBoxLayout() self.mainLayout.addWidget(self.tabWidget) self.mainWidget = QWidget() self.mainWidget.setLayout(self.mainLayout) self.wrapperLayout = QVBoxLayout() self.wrapperLayout.addWidget(self.mainWidget) self.wrapperLayout.setContentsMargins(0, 0, 0, 0) self.setLayout(self.wrapperLayout) self.setWindowTitle(random.choice(GuppyWindow.titles)) self.show() def show_hist_tab(self): self.tabWidget.setCurrentIndex(self.history_ind) def show_repeater_tab(self): self.tabWidget.setCurrentIndex(self.repeater_ind) def show_interceptor_tab(self): self.tabWidget.setCurrentIndex(self.interceptor_ind) def show_decoder_tab(self): self.tabWidget.setCurrentIndex(self.decoder_ind) def show_active_macro_tab(self): self.tabWidget.setCurrentIndex(self.macro_ind) self.macroWidget.show_active() def show_int_macro_tab(self): self.tabWidget.setCurrentIndex(self.macro_ind) self.macroWidget.show_int() def resizeEvent(self, event): QWidget.resizeEvent(self, event) self._resizeTimer.stop() self._resizeTimer.start(self.delayTimeout) self.mainWidget.setVisible(False) @pyqtSlot() def _delayedUpdate(self): self._resizeTimer.stop() self.mainWidget.setVisible(True) def close(self): self.interceptorWidget.close() ================================================ FILE: guppyproxy/gup.py ================================================ import argparse import sys import os from PyQt5.QtWidgets import QApplication from PyQt5.QtCore import Qt from guppyproxy.gui import GuppyWindow from guppyproxy.proxy import ProxyClient, MessageError, ProxyThread from guppyproxy.util import confirm, set_running_as_app from guppyproxy.macros import MacroClient def load_certificates(client, path): client.load_certificates(os.path.join(path, "server.pem"), os.path.join(path, "server.key")) def generate_certificates(client, path): try: os.makedirs(path, 0o755) except os.error as e: if not os.path.isdir(path): raise e pkey_file = os.path.join(path, 'server.key') cert_file = os.path.join(path, 'server.pem') client.generate_certificates(pkey_file, cert_file) def main(): parser = argparse.ArgumentParser(description="Guppy debug flags. Don't worry about most of these") parser.add_argument("--binary", nargs=1, help="location of the backend binary") parser.add_argument("--attach", nargs=1, help="attach to an already running backend") parser.add_argument("--dbgattach", nargs=1, help="attach to an already running backend and also perform setup") parser.add_argument('--debug', help='run in debug mode', action='store_true') parser.add_argument('--dog', help='dog', action='store_true') args = parser.parse_args() if args.binary is not None and args.attach is not None: print("Cannot provide both a binary location and an address to connect to") exit(1) data_dir = os.path.join(os.path.expanduser('~'), '.guppy') if args.binary is not None: binloc = args.binary[0] msg_addr = None elif args.attach is not None or args.dbgattach: binloc = None if args.attach is not None: msg_addr = args.attach[0] if args.dbgattach is not None: msg_addr = args.dbgattach[0] else: msg_addr = None binloc = os.path.join(data_dir, "puppy") if 'RESOURCEPATH' in os.environ: rpath = os.environ['RESOURCEPATH'] checkloc = os.path.join(rpath, 'puppyrsc', 'puppy.osx') if os.path.exists(checkloc): set_running_as_app(True) binloc = checkloc if not os.path.exists(binloc): print("Could not find puppy binary. Please ensure that it has been compiled and placed in ~/.guppy/, or pass in the binary location from the command line") exit(1) cert_dir = os.path.join(data_dir, "certs") with ProxyClient(binary=binloc, conn_addr=msg_addr, debug=args.debug) as client: try: load_certificates(client, cert_dir) except MessageError as e: generate_certificates(client, cert_dir) print("Certificates generated to {}".format(cert_dir)) print("Be sure to add {} to your trusted CAs in your browser!".format(os.path.join(cert_dir, "server.pem"))) load_certificates(client, cert_dir) try: # Only try and listen/set default storage if we're not attaching if args.attach is None: storage = client.add_in_memory_storage("") client.disk_storage = storage client.inmem_storage = client.add_in_memory_storage("m") client.set_proxy_storage(storage.storage_id) app = QApplication(sys.argv) window = GuppyWindow(client) try: app.exec_() finally: window.close() except MessageError as e: print(str(e)) MacroClient._ded = True # pray this kills the threads ProxyThread.waitall() def start(): main() if __name__ == '__main__': main() ================================================ FILE: guppyproxy/hexteditor.py ================================================ import base64 from guppyproxy.util import printable_data, qtprintable, textedit_highlight, DisableUpdates from guppyproxy.proxy import _parse_message, Headers from itertools import count from PyQt5.QtWidgets import QWidget, QTextEdit, QTableWidget, QVBoxLayout, QTableWidgetItem, QTabWidget, QStackedLayout, QLabel, QComboBox from PyQt5.QtGui import QTextCursor, QTextCharFormat, QImage, QColor, QTextImageFormat, QTextDocument, QTextDocumentFragment, QTextBlockFormat from PyQt5.QtCore import Qt, pyqtSlot, QUrl from pygments import highlight from pygments.formatters import HtmlFormatter from pygments.lexers import get_lexer_for_mimetype, TextLexer from pygments.lexers.data import JsonLexer from pygments.lexers.html import HtmlLexer from pygments.styles import get_style_by_name class PrettyPrintWidget(QWidget): VIEW_NONE = 0 VIEW_HIGHLIGHTED = 1 VIEW_JSON = 2 VIEW_HTMLXML = 3 def __init__(self, *args, **kwargs): QWidget.__init__(self, *args, **kwargs) self.headers = Headers() self.data = b'' self.view = 0 self.setLayout(QVBoxLayout()) self.layout().setContentsMargins(0, 0, 0, 0) self.stack = QStackedLayout() self.stack.setContentsMargins(0, 0, 0, 0) self.nopp_widg = QLabel("No pretty version available") self.stack.addWidget(self.nopp_widg) self.highlighted_widg = QTextEdit() self.highlighted_widg.setReadOnly(True) self.stack.addWidget(self.highlighted_widg) self.json_widg = QTextEdit() self.json_widg.setReadOnly(True) self.stack.addWidget(self.json_widg) self.htmlxml_widg = QTextEdit() self.htmlxml_widg.setReadOnly(True) self.stack.addWidget(self.htmlxml_widg) self.selector = QComboBox() self.selector.addItem("Manually Select Printer", self.VIEW_NONE) self.selector.addItem("Highlighted", self.VIEW_HIGHLIGHTED) self.selector.addItem("JSON", self.VIEW_JSON) self.selector.addItem("HTML/XML", self.VIEW_HTMLXML) self.selector.currentIndexChanged.connect(self._combo_changed) self.layout().addWidget(self.selector) self.layout().addLayout(self.stack) def guess_format(self): if 'Content-Type' in self.headers: ct = self.headers.get('Content-Type').lower() if 'json' in ct: self.set_view(self.VIEW_JSON) elif 'html' in ct or 'xml' in ct: self.set_view(self.VIEW_HTMLXML) else: self.set_view(self.VIEW_HIGHLIGHTED) else: self.set_view(self.VIEW_NONE) @pyqtSlot() def _combo_changed(self): field = self.selector.itemData(self.selector.currentIndex()) old = self.selector.blockSignals(True) self.set_view(field) self.selector.blockSignals(old) def set_view(self, view): if view == self.VIEW_NONE: self.clear_output() self.stack.setCurrentIndex(self.VIEW_NONE) elif view == self.VIEW_JSON: self.clear_output() self.fill_json() self.stack.setCurrentIndex(self.VIEW_JSON) elif view == self.VIEW_HTMLXML: self.clear_output() self.fill_htmlxml() self.stack.setCurrentIndex(self.VIEW_HTMLXML) elif view == self.VIEW_HIGHLIGHTED: self.clear_output() self.fill_highlighted() self.stack.setCurrentIndex(self.VIEW_HIGHLIGHTED) else: return self.selector.setCurrentIndex(view) self.view = view def clear_output(self): self.json_widg.setPlainText("") self.htmlxml_widg.setPlainText("") def set_bytes(self, bs): self.clear_output() self.headers = Headers() self.data = b'' if not bs: return _, h, body = _parse_message(bs, lambda x: None) self.headers = h self.data = body def fill_json(self): from .decoder import pp_json with DisableUpdates(self.json_widg): self.json_widg.setPlainText("") if not self.data: return try: j = pp_json(self.data.decode()) except Exception: return highlighted = textedit_highlight(j, JsonLexer()) self.json_widg.setHtml(highlighted) def fill_htmlxml(self): from lxml import etree, html with DisableUpdates(self.htmlxml_widg): self.htmlxml_widg.setPlainText("") if not self.data: return try: fragments = html.fragments_fromstring(self.data.decode()) parsed_frags = [] for f in fragments: parsed_frags.append(etree.tostring(f, pretty_print=True)) pretty = b''.join(parsed_frags) except Exception: return highlighted = textedit_highlight(pretty, HtmlLexer()) self.htmlxml_widg.setHtml(highlighted) def fill_highlighted(self): with DisableUpdates(self.htmlxml_widg): self.highlighted_widg.setPlainText("") if not self.data: return ct = self.headers.get('Content-Type').lower() if ";" in ct: ct = ct.split(";")[0] try: lexer = get_lexer_for_mimetype(ct) highlighted = textedit_highlight(self.data, lexer) except: highlighted = printable_data(self.data) self.highlighted_widg.setHtml(highlighted) class HextEditor(QWidget): byte_image = QImage() byte_image.loadFromData(base64.b64decode("iVBORw0KGgoAAAANSUhEUgAAAAgAAAAICAYAAADED76LAAAAQklEQVQYlYWPMQoAMAgD82jpSzL613RpqG2FBly8QwywQlJ1UENSipAilIAS2FKFziFZ8LIVOjg6ocJx/+ELD/zVnJcqe5vHUAJgAAAAAElFTkSuQmCC")) byte_url = "data://byte.png" byte_property = 0x100000 + 1 nonce_property = 0x100000 + 2 byte_nonce = count() def __init__(self, enable_pretty=True): QWidget.__init__(self) layout = QVBoxLayout() self.enable_pretty = enable_pretty self.setLayout(layout) self.layout().setSpacing(0) self.layout().setContentsMargins(0, 0, 0, 0) self.lexer = TextLexer() self.textedit = QTextEdit() self.textedit.setAcceptRichText(False) doc = self.textedit.document() font = doc.defaultFont() font.setFamily("Courier New") font.setPointSize(10) doc.setDefaultFont(font) doc.addResource(QTextDocument.ImageResource, QUrl(self.byte_url), HextEditor.byte_image) self.textedit.focusInEvent = self.focus_in_event self.textedit.focusOutEvent = self.focus_left_event self.data = b'' self.pretty_mode = False self.layout().addWidget(self.textedit) def focus_in_event(self, e): QTextEdit.focusInEvent(self.textedit, e) if not self.textedit.isReadOnly(): self.set_bytes(self.data) self.pretty_mode = False def focus_left_event(self, e): QTextEdit.focusOutEvent(self.textedit, e) if not self.textedit.isReadOnly(): self.data = self.get_bytes() self.set_bytes_highlighted(self.data) self.pretty_mode = True def setReadOnly(self, ro): self.textedit.setReadOnly(ro) def _insert_byte(self, cursor, b): f = QTextImageFormat() f2 = QTextCursor().charFormat() cursor.document().addResource(QTextDocument.ImageResource, QUrl(self.byte_url), HextEditor.byte_image) f.setName(self.byte_url) f.setProperty(HextEditor.byte_property, b + 1) f.setProperty(HextEditor.nonce_property, next(self.byte_nonce)) cursor.insertImage(f) cursor.setCharFormat(QTextCursor().charFormat()) def clear(self): self.textedit.setPlainText("") def set_lexer(self, lexer): self.lexer = lexer def set_bytes(self, bs): with DisableUpdates(self.textedit): self.pretty_mode = False self.data = bs chunks = HextEditor._split_by_printables(bs) self.clear() cursor = QTextCursor(self.textedit.document()) cursor.beginEditBlock() try: cursor.select(QTextCursor.Document) cursor.setCharFormat(QTextCharFormat()) cursor.clearSelection() for chunk in chunks: if chr(chunk[0]) in qtprintable: cursor.insertText(chunk.decode()) else: for b in chunk: self._insert_byte(cursor, b) finally: cursor.endEditBlock() self.repaint() # needed to fix issue with py2app def set_bytes_highlighted(self, bs, lexer=None): if not self.enable_pretty: self.set_bytes(bs) return with DisableUpdates(self.textedit): self.pretty_mode = True self.clear() self.data = bs if lexer: self.lexer = lexer printable = printable_data(bs) highlighted = textedit_highlight(printable, self.lexer) self.textedit.setHtml(highlighted) self.repaint() # needed to fix issue with py2app def get_bytes(self): if not self.pretty_mode: self.data = self._get_bytes() return self.data def _get_bytes(self): from .util import hexdump bs = bytearray() block = self.textedit.document().firstBlock() newline = False while block.length() > 0: if newline: bs.append(ord('\n')) newline = True it = block.begin() while not it.atEnd(): f = it.fragment() fmt = f.charFormat() byte = fmt.intProperty(HextEditor.byte_property) if byte > 0: text = f.text().encode() if text == b"\xef\xbf\xbc": bs.append(byte - 1) else: bs += text else: text = f.text() bs += text.encode() it += 1 block = block.next() return bytes(bs) @classmethod def _split_by_printables(cls, bs): if len(bs) == 0: return [] def is_printable(c): return c in qtprintable chunks = [] printable = is_printable(chr(bs[0])) a = 0 b = 1 while b < len(bs): if is_printable(chr(bs[b])) != printable: chunks.append(bs[a:b]) a = b printable = not printable b += 1 chunks.append(bs[a:b]) return chunks class HexEditor(QWidget): def __init__(self): QWidget.__init__(self) self.setLayout(QVBoxLayout()) self.layout().setContentsMargins(0, 0, 0, 0) self.layout().setSpacing(0) self.data = bytearray() self.datatable = QTableWidget() self.datatable.cellChanged.connect(self._cell_changed) self.datatable.horizontalHeader().setStretchLastSection(True) self.row_size = 16 self.read_only = False self.redraw_table() self.layout().addWidget(self.datatable) def set_bytes(self, bs): self.data = bytearray(bs) self.redraw_table() def get_bytes(self): return bytes(self.data) def setReadOnly(self, ro): self.read_only = ro self.redraw_table() def _redraw_strcol(self, row): start = self.row_size * row end = start + self.row_size data = self.data[start:end] print_data = printable_data(data, include_newline=False) item = QTableWidgetItem(print_data) item.setFlags(item.flags() ^ Qt.ItemIsEditable) self.datatable.setItem(row, self.str_col, item) def redraw_table(self, length=None): with DisableUpdates(self.datatable): oldsig = self.datatable.blockSignals(True) self.row_size = length or self.row_size self.datatable.setColumnCount(self.row_size + 1) self.datatable.setRowCount(0) self.str_col = self.row_size self.datatable.horizontalHeader().hide() self.datatable.verticalHeader().hide() rows = int(len(self.data) / self.row_size) if len(self.data) % self.row_size > 0: rows += 1 self.datatable.setRowCount(rows) for i in range(rows * self.row_size): row = i / self.row_size col = i % self.row_size if i < len(self.data): dataval = "%02x" % self.data[i] item = QTableWidgetItem(dataval) if self.read_only: item.setFlags(item.flags() ^ Qt.ItemIsEditable) else: item = QTableWidgetItem("") item.setFlags(item.flags() ^ Qt.ItemIsEditable) self.datatable.setItem(row, col, item) for row in range(rows): self._redraw_strcol(row) self.datatable.blockSignals(oldsig) self.datatable.resizeColumnsToContents() self.datatable.resizeRowsToContents() @classmethod def _format_hex(cls, n): return ("%02x" % n).upper() @pyqtSlot(int, int) def _cell_changed(self, row, col): oldsig = self.datatable.blockSignals(True) if col == self.str_col: return if len(self.data) == 0: return data_ind = self.row_size * row + col if data_ind >= len(self.data): return data_text = self.datatable.item(row, col).text() try: data_val = int(data_text, 16) if data_val < 0x0 or data_val > 0xff: raise Exception() except Exception as e: item = QTableWidgetItem(self._format_hex(self.data[data_ind])) self.datatable.setItem(row, col, item) self.datatable.blockSignals(oldsig) return if data_text != self._format_hex(data_val): self.datatable.setItem(row, col, QTableWidgetItem(self._format_hex(data_val))) self.data[data_ind] = data_val self._redraw_strcol(row) self.datatable.blockSignals(oldsig) class ComboEditor(QWidget): def __init__(self, pretty_tab=True, enable_pretty=True): QWidget.__init__(self) self.setLayout(QVBoxLayout()) self.layout().setSpacing(0) self.layout().setContentsMargins(0, 0, 0, 0) self.data = b'' self.enable_pretty = enable_pretty self.tabWidget = QTabWidget() self.hexteditor = HextEditor(enable_pretty=self.enable_pretty) self.hexeditor = HexEditor() self.ppwidg = PrettyPrintWidget() self.hexteditor_ind = self.tabWidget.count() self.tabWidget.addTab(self.hexteditor, "Text") self.hexeditor_ind = self.tabWidget.count() self.tabWidget.addTab(self.hexeditor, "Hex") self.pp_ind = -1 if pretty_tab: self.pp_ind = self.tabWidget.count() self.tabWidget.addTab(self.ppwidg, "Pretty") self.tabWidget.currentChanged.connect(self._tab_changed) self.previous_tab = self.tabWidget.currentIndex() self.layout().addWidget(self.tabWidget) @pyqtSlot(int) def _tab_changed(self, i): # commit data from old tab if self.previous_tab == self.hexteditor_ind: self.data = self.hexteditor.get_bytes() if self.previous_tab == self.hexeditor_ind: self.data = self.hexeditor.get_bytes() # set up new tab if i == self.hexteditor_ind: if self.hexteditor.pretty_mode: self.hexteditor.set_bytes_highlighted(self.data) else: self.hexteditor.set_bytes(self.data) if i == self.hexeditor_ind: self.hexeditor.set_bytes(self.data) if i == self.pp_ind: self.ppwidg.set_bytes(self.data) self.ppwidg.guess_format() # update previous tab self.previous_tab = self.tabWidget.currentIndex() @pyqtSlot(bytes) def set_bytes(self, bs): self.data = bs self.tabWidget.setCurrentIndex(0) if self.tabWidget.currentIndex() == self.hexteditor_ind: self.hexteditor.set_bytes(bs) elif self.tabWidget.currentIndex() == self.hexeditor_ind: self.hexeditor.set_bytes(bs) elif self.tabWidget.currentIndex() == self.pp_ind: self.ppwidg.set_bytes(bs) @pyqtSlot(bytes) def set_bytes_highlighted(self, bs, lexer=None): self.data = bs self.tabWidget.setCurrentIndex(0) if self.enable_pretty: self.hexteditor.set_bytes_highlighted(bs, lexer=lexer) else: self.set_bytes(bs) def get_bytes(self): if self.tabWidget.currentIndex() == self.hexteditor_ind: self.data = self.hexteditor.get_bytes() elif self.tabWidget.currentIndex() == self.hexeditor_ind: self.data = self.hexeditor.get_bytes() return self.data def setReadOnly(self, ro): self.hexteditor.setReadOnly(ro) self.hexeditor.setReadOnly(ro) ================================================ FILE: guppyproxy/interceptor.py ================================================ from guppyproxy.util import display_error_box from guppyproxy.proxy import InterceptMacro, parse_request, parse_response from guppyproxy.hexteditor import ComboEditor from PyQt5.QtWidgets import QWidget, QVBoxLayout, QHBoxLayout, QPushButton from PyQt5.QtCore import pyqtSlot, pyqtSignal, QObject import threading edit_queue = [] class InterceptEvent: def __init__(self): self.e = threading.Event() self.canceled = False self.message = None def wait(self): self.e.wait() return self.message def set(self, message): self.message = message self.e.set() def cancel(self): self.canceled = True self.set(None) class InterceptedMessage: def __init__(self, request=None, response=None, wsmessage=None): self.request = request self.response = response self.wsmessage = wsmessage self.event = InterceptEvent() self.message_type = None if self.request: self.message_type = "request" elif self.response: self.message_type = "response" elif self.wsmessage: self.message_type = "wsmessage" class InterceptorMacro(InterceptMacro, QObject): """ A class representing a macro that modifies requests as they pass through the proxy """ messageReceived = pyqtSignal(InterceptedMessage) def __init__(self, int_widget): InterceptMacro.__init__(self) QObject.__init__(self) self.int_widget = int_widget self.messageReceived.connect(self.int_widget.message_received) self.name = "InterceptorMacro" def mangle_request(self, request): int_msg = InterceptedMessage(request=request) self.messageReceived.emit(int_msg) req = int_msg.event.wait() if int_msg.event.canceled: return request req.dest_host = request.dest_host req.dest_port = request.dest_port req.use_tls = request.use_tls return req def mangle_response(self, request, response): int_msg = InterceptedMessage(response=response) self.messageReceived.emit(int_msg) rsp = int_msg.event.wait() if int_msg.event.canceled: return response return rsp def mangle_websocket(self, request, response, message): # just don't do this right now pass class InterceptorWidget(QWidget): def __init__(self, client): QWidget.__init__(self) self.client = client self.int_conn = None self.queued_messages = [] self.editing_message = None self.editing = False self.int_req = False self.int_rsp = False self.int_ws = False # layouts self.setLayout(QVBoxLayout()) self.layout().setSpacing(0) self.layout().setContentsMargins(0, 0, 0, 0) buttons = QHBoxLayout() buttons.setContentsMargins(0, 0, 0, 0) buttons.setSpacing(10) # widgets intReqButton = QPushButton("Int. Requests") intRspButton = QPushButton("Int. Responses") intWsButton = QPushButton("Int. Websocket") forwardButton = QPushButton("Forward") cancelButton = QPushButton("Cancel") self.editor = ComboEditor() intReqButton.setCheckable(True) intRspButton.setCheckable(True) intWsButton.setCheckable(True) intWsButton.setEnabled(False) forwardButton.clicked.connect(self.forward_message) cancelButton.clicked.connect(self.cancel_edit) intReqButton.toggled.connect(self.int_req_toggled) intRspButton.toggled.connect(self.int_rsp_toggled) intWsButton.toggled.connect(self.int_ws_toggled) buttons.addWidget(forwardButton) buttons.addWidget(cancelButton) buttons.addWidget(intReqButton) buttons.addWidget(intRspButton) buttons.addWidget(intWsButton) # checkbox for req/rsp/ws self.layout().addLayout(buttons) self.layout().addWidget(self.editor) @pyqtSlot(bool) def int_req_toggled(self, state): self.int_req = state self.restart_intercept() @pyqtSlot(bool) def int_rsp_toggled(self, state): self.int_rsp = state self.restart_intercept() @pyqtSlot(bool) def int_ws_toggled(self, state): self.int_ws = state self.restart_intercept() @pyqtSlot(InterceptedMessage) def message_received(self, msg): self.queued_messages.append(msg) # Update queue list self.edit_next_message() def set_edited_message(self, msg): if msg.message_type == "request": self.editor.set_bytes(msg.request.full_message()) elif msg.message_type == "response": self.editor.set_bytes(msg.response.full_message()) elif msg.message_type == "wsmessage": # this is not gonna work self.editor.set_bytes(msg.wsmessage.message) def edit_next_message(self): if self.editing: return self.editor.set_bytes(b"") if not self.queued_messages: return self.editing_message = self.queued_messages.pop() self.set_edited_message(self.editing_message) self.editing = True @pyqtSlot() def forward_message(self): if not self.editing: return if self.editing_message.message_type == "request": try: req = parse_request(self.editor.get_bytes()) except Exception: display_error_box("Could not parse request") return self.editing_message.event.set(req) elif self.editing_message.message_type == "response": try: rsp = parse_response(self.editor.get_bytes()) except Exception: display_error_box("Could not parse response") return self.editing_message.event.set(rsp) elif self.editing_message.message_type == "wsmessage": pass self.editing = False self.edit_next_message() @pyqtSlot() def cancel_edit(self): if self.editing_message: self.editing_message.event.cancel() self.editing = False self.edit_next_message() def clear_edit_queue(self): while self.queued_messages or self.editing_message: if self.editing_message: self.editing_message.event.cancel() self.editing_message = False if self.queued_messages: self.editing_message = self.queued_messages.pop() def restart_intercept(self): self.close() self.editor.set_bytes("") self.editing = False if not (self.int_req or self.int_rsp or self.int_ws): return mangle_macro = InterceptorMacro(self) mangle_macro.intercept_requests = self.int_req mangle_macro.intercept_responses = self.int_rsp mangle_macro.intercept_ws = self.int_ws self.int_conn = self.client.new_conn() self.int_conn.intercept(mangle_macro) def close(self): if self.int_conn: self.int_conn.close() self.int_conn = None self.clear_edit_queue() ================================================ FILE: guppyproxy/macros.py ================================================ import glob import imp import os import random import re import stat import sys import traceback from guppyproxy.proxy import InterceptMacro, HTTPRequest, ProxyThread from guppyproxy.util import display_error_box, qtprintable, set_default_dialog_dir, default_dialog_dir, open_dialog, save_dialog, display_info_box from collections import namedtuple from itertools import count from PyQt5.QtWidgets import QWidget, QTableWidget, QTableWidgetItem, QGridLayout, QHeaderView, QAbstractItemView, QVBoxLayout, QHBoxLayout, QComboBox, QTabWidget, QPushButton, QLineEdit, QStackedLayout, QToolButton, QCheckBox, QLabel, QTableView, QPlainTextEdit, QFormLayout, QSizePolicy, QDialog, QPlainTextEdit, QTextEdit from PyQt5.QtCore import pyqtSlot, pyqtSignal, QObject, QVariant, Qt, QAbstractTableModel, QModelIndex, QItemSelection, QSortFilterProxyModel errwins = set() class MacroException(Exception): pass class MacroClient(QObject): # A wrapper around proxy.ProxyClient that provides a simplified interface # to a macro to prevent it from accidentally making the proxy unstable. # Will add to it as needed/requested _macroOutput = pyqtSignal(str) _requestOutput = pyqtSignal(HTTPRequest) _ded = False def __init__(self, client): QObject.__init__(self) self._client = client def check_dead(self): """ Raises an exception if the program is trying to close. Use this in your loops so that your macro doesn't keep the program from quitting """ if self._ded: raise Exception("program over=very yes") def submit(self, req, save=False): """ Submit a request. If save == True, it will be saved to history """ self.check_dead() self._client.submit(req, save=save) def save(self, req): """ Manually save a request to history. This can be used to perform a request and only save requests with interesting responses """ self.check_dead() self._client.save_new(req) def output(self, s): """ Write text to the "output" tab """ self.check_dead() self._macroOutput.emit(str(s)+"\n") def output_req(self, req): """ Add a request/response to the list of outputted requests """ self.check_dead() self._requestOutput.emit(req) def new_request(self, method="GET", path="/", proto_major=1, proto_minor=1, headers=None, body=bytes(), dest_host="", dest_port=80, use_tls=False, tags=None): """ Manually create a request object that can be submitted with client.submit() """ self.check_dead() return HTTPRequest(method=method, path=path, proto_major=proto_major, proto_minor=proto_minor, headers=headers, body=body, dest_host=dest_host, dest_port=dest_port, use_tls=use_tls, tags=tags) class FileInterceptMacro(InterceptMacro, QObject): """ An intercepting macro that loads a macro from a file. """ macroError = pyqtSignal(str) def __init__(self, parent, client, filename): InterceptMacro.__init__(self) QObject.__init__(self) self.fname = filename or None # name from the file self.source = None self.client = client self.parent = parent self.mclient = MacroClient(self.client) self.cached_args = {} self.used_args = {} if filename: self.load(filename) def __repr__(self): s = self.fname or "(No loaded macro)" return "" % s def load(self, fname): if fname: self.fname = fname # yes there's a race condition here, but it's better than nothing st = os.stat(self.fname) if (st.st_mode & stat.S_IWOTH): raise MacroException("Refusing to load world-writable macro: %s" % self.fname) module_name = self.fname try: if module_name in sys.modules and self.source != None: del sys.modules[module_name] del self.source self.source = imp.load_source(module_name, self.fname) except Exception as e: self.macroError.emit(make_err_str(self, e)) else: self.fname = None self.source = None # Update what we can do if self.source and hasattr(self.source, 'mangle_request'): self.intercept_requests = True else: self.intercept_requests = False if self.source and hasattr(self.source, 'mangle_response'): self.intercept_responses = True else: self.intercept_responses = False if self.source and hasattr(self.source, 'mangle_websocket'): self.intercept_ws = True else: self.intercept_ws = False def prompt_args(self): if not hasattr(self.source, "get_args"): self.used_args = {} return True try: spec = self.source.get_args() except Exception as e: self.macroError.emit(make_err_str(self, e)) return False args = get_macro_args(self.parent, spec, cached=self.cached_args) if args is None: return False self.cached_args = args self.used_args = args return True def init(self, args): if hasattr(self.source, 'init'): try: self.source.init(self.mclient, args) except Exception as e: self.macroError.emit(make_err_str(self, e)) return False return True def mangle_request(self, request): if hasattr(self.source, 'mangle_request'): try: return self.source.mangle_request(self.mclient, self.used_args, request) except Exception as e: self.macroError.emit(make_err_str(self, e)) return request def mangle_response(self, request, response): if hasattr(self.source, 'mangle_response'): try: return self.source.mangle_response(self.mclient, self.used_args, request, response) except Exception as e: self.macroError.emit(make_err_str(self, e)) return response def mangle_websocket(self, request, response, message): if hasattr(self.source, 'mangle_websocket'): try: return self.source.mangle_websocket(self.mclient, self.used_args, request, response, message) except Exception as e: self.macroError.emit(make_err_str(self, e)) return message class FileMacro(QObject): macroError = pyqtSignal(str) macroComplete = pyqtSignal(str) requestOutput = pyqtSignal(HTTPRequest) macroOutput = pyqtSignal(str) def __init__(self, parent, filename='', resultSlot=None): QObject.__init__(self) self.fname = filename or None # filename we load from self.source = None self.parent = parent self.cached_args = {} self.load() def load(self): if self.fname: st = os.stat(self.fname) if (st.st_mode & stat.S_IWOTH): raise MacroException("Refusing to load world-writable macro: %s" % self.fname) module_name = self.fname try: if module_name in sys.modules and self.source != None: del sys.modules[module_name] del self.source self.source = imp.load_source('%s'%module_name, self.fname) except Exception as e: self.macroError.emit(make_err_str(self, e)) def execute(self, client, reqs): self.load() # Execute the macro if self.source: args = None if hasattr(self.source, "get_args"): try: spec = self.source.get_args() except Exception as e: self.macroError.emit(make_err_str(self, e)) return args = get_macro_args(self.parent, spec, cached=self.cached_args) if args is None: return self.cached_args = args def perform_macro(): mclient = MacroClient(client) mclient._macroOutput.connect(self.macroOutput) mclient._requestOutput.connect(self.requestOutput) try: self.source.run_macro(mclient, args, reqs) _, fname = os.path.split(self.fname) self.macroComplete.emit("%s has finished running" % fname) except Exception as e: self.macroError.emit(make_err_str(self, e)) ProxyThread(target=perform_macro).start() class MacroWidget(QWidget): # Tabs containing both int and active macros def __init__(self, client, *args, **kwargs): self.client = client QWidget.__init__(self, *args, **kwargs) self.setLayout(QVBoxLayout()) self.layout().setContentsMargins(0, 0, 0, 0) self.tab_widg = QTabWidget() self.active_widg = ActiveMacroWidget(client) self.active_ind = self.tab_widg.count() self.tab_widg.addTab(self.active_widg, "Active") self.int_widg = IntMacroWidget(client) self.int_ind = self.tab_widg.count() self.tab_widg.addTab(self.int_widg, "Intercepting") self.warning_widg = QLabel("

Warning! Macros may cause instability

Macros load and run python files into the Guppy process. If you're not careful when you write them you may cause Guppy to crash. If an active macro ends up in an infinite loop you may need to force kill the application when you quit.

PROCEED WITH CAUTION

") self.warning_widg.setWordWrap(True) self.tab_widg.addTab(self.warning_widg, "Warning") self.layout().addWidget(self.tab_widg) def show_active(self): self.tab_widg.setCurrentIndex(self.active_ind) def show_int(self): self.tab_widg.setCurrentIndex(self.int_ind) def add_requests(self, reqs): # Add requests to active macro inputw self.active_widg.add_requests(reqs) class IntMacroListModel(QAbstractTableModel): err_window = None def __init__(self, parent, client, *args, **kwargs): self.client = client QAbstractTableModel.__init__(self, *args, **kwargs) self.macros = [] self.int_conns = {} self.conn_ids = count() self.parent = parent self.headers = ["On", "Path"] def _emit_all_data(self): self.dataChanged.emit(self.createIndex(0, 0), self.createIndex(self.columnCount(None), self.rowCount(None))) def headerData(self, section, orientation, role): if role == Qt.DisplayRole and orientation == Qt.Horizontal: return self.headers[section] return QVariant() def rowCount(self, parent): return len(self.macros) def columnCount(self, parent): return len(self.headers) def data(self, index, role): if role == Qt.DisplayRole: if index.column() == 1: rowdata = self.macros[index.row()] macro = rowdata[index.column()] return macro.fname if role == Qt.CheckStateRole: if index.column() == 0: if self.macros[index.row()][0]: return 2 return 0 return QVariant() def flags(self, index): f = Qt.ItemIsEnabled | Qt.ItemIsSelectable if index.column() == 0: f = f | Qt.ItemIsUserCheckable | Qt.ItemIsEditable return f def setData(self, index, value, role): if role == Qt.CheckStateRole and index.column() == 0: if value: self.enable_macro(index.row()) else: self.disable_macro(index.row()) return True return False # Non model functions @pyqtSlot(str) def add_macro_exception(self, estr): if not self.err_window: self.err_window = MacroErrWindow() self.err_window.add_error(estr) def add_macro(self, macro_path): self.beginResetModel() macro = FileInterceptMacro(self.parent, self.client, macro_path) macro.macroError.connect(self.add_macro_exception) self.macros.append([False, macro, -1]) self._emit_all_data() self.endResetModel() def remove_macro(self, ind): self.beginResetModel() self.disable_macro(ind) self.macros = self.macros[:ind] + self.macros[ind+1:] self._emit_all_data() self.endResetModel() def enable_macro(self, ind): self.beginResetModel() macro = self.macros[ind][1] if not macro.init(None): return try: macro.load(macro.fname) except MacroException as e: display_error_box("Macro could not be loaded: %s", e) return except Exception as e: self.add_macro_exception(make_err_str(macro, e)) return if not (macro.intercept_requests or macro.intercept_responses or macro.intercept_ws): display_error_box("Macro must implement mangle_request or mangle_response") return if not macro.prompt_args(): return conn = self.client.new_conn() conn_id = next(self.conn_ids) self.macros[ind][2] = conn_id self.int_conns[conn_id] = conn conn.intercept(macro) self.macros[ind][0] = True self._emit_all_data() self.endResetModel() def disable_macro(self, ind): self.beginResetModel() conn_id = self.macros[ind][2] if conn_id >= 0: conn = self.int_conns[conn_id] conn.close() del self.int_conns[conn_id] self.macros[ind][2] = -1 self.macros[ind][0] = False self._emit_all_data() self.endResetModel() class IntMacroWidget(QWidget): # Lets the user enable/disable int. macros def __init__(self, client, *args, **kwargs): self.client = client self.macros = [] QWidget.__init__(self, *args, **kwargs) self.setLayout(QVBoxLayout()) self.layout().setContentsMargins(0, 0, 0, 0) buttonLayout = QHBoxLayout() new_button = QPushButton("New") add_button = QPushButton("Add...") remove_button = QPushButton("Remove") new_button.clicked.connect(self.new_macro) add_button.clicked.connect(self.browse_macro) remove_button.clicked.connect(self.remove_selected) # Set up table self.macroListModel = IntMacroListModel(self, self.client) self.macroListView = QTableView() self.macroListView.setModel(self.macroListModel) self.macroListView.verticalHeader().setSectionResizeMode(QHeaderView.ResizeToContents) self.macroListView.verticalHeader().hide() self.macroListView.horizontalHeader().setSectionResizeMode(QHeaderView.ResizeToContents) self.macroListView.horizontalHeader().hide() self.macroListView.horizontalHeader().setStretchLastSection(True) self.macroListView.setSelectionBehavior(QAbstractItemView.SelectRows) self.macroListView.setSelectionMode(QAbstractItemView.SingleSelection) buttonLayout.addWidget(new_button) buttonLayout.addWidget(add_button) buttonLayout.addWidget(remove_button) buttonLayout.addStretch() self.layout().addWidget(self.macroListView) self.layout().addLayout(buttonLayout) def add_macro(self, fname): self.macroListModel.add_macro(fname) def reload_macros(self): self.macroListModel.reload_macros() @pyqtSlot() def new_macro(self): fname = save_dialog(self, filter_string="Python File (*.py)") if not fname: return with open(fname, 'w') as f: contents = new_int_macro() f.write(contents) self.add_macro(fname) @pyqtSlot() def browse_macro(self): fname = open_dialog(self, filter_string="Python File (*.py)") if not fname: return self.add_macro(fname) @pyqtSlot() def remove_selected(self): rows = self.macroListView.selectionModel().selectedRows() if len(rows) == 0: return for idx in rows: row = idx.row() self.macroListModel.remove_macro(row) return class ActiveMacroModel(QAbstractTableModel): err_window = None requestOutput = pyqtSignal(HTTPRequest) macroOutput = pyqtSignal(str) def __init__(self, parent, client, *args, **kwargs): QAbstractTableModel.__init__(self, *args, **kwargs) self.client = client self.parent = parent self.headers = ["Path"] self.macros = [] def _emit_all_data(self): self.dataChanged.emit(self.createIndex(0, 0), self.createIndex(self.columnCount(None), self.rowCount(None))) def headerData(self, section, orientation, role): if role == Qt.DisplayRole and orientation == Qt.Horizontal: return self.headers[section] return QVariant() def rowCount(self, parent): return len(self.macros) def columnCount(self, parent): return len(self.headers) def data(self, index, role): if role == Qt.DisplayRole: return self.macros[index.row()][0] return QVariant() def flags(self, index): return Qt.ItemIsEnabled | Qt.ItemIsSelectable def add_macro(self, path): self.beginResetModel() self._emit_all_data() fileMacro = FileMacro(self.parent, filename=path) fileMacro.macroOutput.connect(self.macroOutput) fileMacro.macroError.connect(self.add_macro_exception) fileMacro.requestOutput.connect(self.requestOutput) fileMacro.macroComplete.connect(self.display_macro_complete) self.macros.append((path, fileMacro)) self.endResetModel() def run_macro(self, ind, reqs=None): path, macro = self.macros[ind] reqs = reqs or [] macro.execute(self.client, reqs) def remove_macro(self, ind): self.beginResetModel() self._emit_all_data() self.macros = self.macros[:ind] + self.macros[ind+1:] self.endResetModel() @pyqtSlot(str) def add_macro_exception(self, estr): if not self.err_window: self.err_window = MacroErrWindow() self.err_window.add_error(estr) @pyqtSlot(str) def display_macro_complete(self, msg): display_info_box(msg, title="Macro complete") class ActiveMacroWidget(QWidget): # Provides an interface to send a set of requests to python scripts def __init__(self, client, *args, **kwargs): from .reqlist import ReqTableWidget, ReqBrowser QWidget.__init__(self, *args, **kwargs) self.client = client self.setLayout(QVBoxLayout()) tab_widg = QTabWidget() # Input inputLayout = QVBoxLayout() inputLayout.setContentsMargins(0, 0, 0, 0) inputLayout.addWidget(QLabel("Input")) inputLayout.setSpacing(8) self.reqlist = ReqTableWidget(self.client) butlayout = QHBoxLayout() delButton = QPushButton("Remove") clearButton = QPushButton("Clear") importAllButton = QPushButton("Import Currently Filtered Requests") delButton.clicked.connect(self.reqlist.delete_selected) clearButton.clicked.connect(self.reqlist.clear) importAllButton.clicked.connect(self.import_all_reqs) butlayout.addWidget(delButton) butlayout.addWidget(clearButton) butlayout.addWidget(importAllButton) butlayout.addStretch() inputLayout.addWidget(self.reqlist) inputLayout.addLayout(butlayout) # Macro selection listLayout = QVBoxLayout() listLayout.addWidget(QLabel("Macros")) listLayout.setContentsMargins(0, 0, 0, 0) listLayout.setSpacing(8) self.tableModel = ActiveMacroModel(self, self.client) self.tableModel.macroOutput.connect(self.add_macro_output) self.tableView = QTableView() self.tableModel.requestOutput.connect(self.add_request_output) self.tableView.setModel(self.tableModel) self.tableView.verticalHeader().setSectionResizeMode(QHeaderView.ResizeToContents) self.tableView.verticalHeader().hide() self.tableView.horizontalHeader().setSectionResizeMode(QHeaderView.ResizeToContents) self.tableView.horizontalHeader().setStretchLastSection(True) self.tableView.horizontalHeader().hide() self.tableView.setSelectionMode(QAbstractItemView.SingleSelection) butlayout2 = QHBoxLayout() newButton = QPushButton("New") newButton.clicked.connect(self.new_macro) addButton2 = QPushButton("Add...") addButton2.clicked.connect(self.browse_macro) delButton2 = QPushButton("Remove") delButton2.clicked.connect(self.remove_selected) runButton2 = QPushButton("Run") runButton2.clicked.connect(self.run_selected_macro) butlayout2.addWidget(newButton) butlayout2.addWidget(addButton2) butlayout2.addWidget(delButton2) butlayout2.addWidget(runButton2) butlayout2.addStretch() listLayout.addWidget(self.tableView) listLayout.addLayout(butlayout2) # Output outputLayout = QVBoxLayout() outputLayout.setContentsMargins(0, 0, 0, 0) outputLayout.setSpacing(8) self.outreqlist = ReqBrowser(self.client, reload_reqs=False, filter_tab=False) self.outreqlist.listWidg.allow_save = True outbutlayout = QHBoxLayout() delButton = QPushButton("Clear") delButton.clicked.connect(self.clear_output) outbutlayout.addWidget(delButton) outbutlayout.addStretch() outputLayout.addWidget(self.outreqlist) outputLayout.addLayout(outbutlayout) text_out_layout = QVBoxLayout() text_out_layout.setContentsMargins(0, 0, 0, 0) self.macro_text_out = QPlainTextEdit() text_out_layout.addWidget(self.macro_text_out) text_out_butlayout = QHBoxLayout() clearBut = QPushButton("Clear") clearBut.clicked.connect(self.clear_text_output) text_out_butlayout.addWidget(clearBut) text_out_butlayout.addStretch() text_out_layout.addLayout(text_out_butlayout) # Tabs intab = QWidget() intabLayout = QVBoxLayout() intabLayout.setContentsMargins(0, 0, 0, 0) intabLayout.addLayout(listLayout) intabLayout.addLayout(inputLayout) intab.setLayout(intabLayout) tab_widg.addTab(intab, "Input") reqOutputWidg = QWidget() reqOutputWidg.setLayout(outputLayout) tab_widg.addTab(reqOutputWidg, "Req. Output") textOutputWidg = QWidget() textOutputWidg.setLayout(text_out_layout) tab_widg.addTab(textOutputWidg, "Text Output") self.layout().addWidget(tab_widg) @pyqtSlot(list) def add_requests(self, reqs): # Add requests to active macro input for req in reqs: self.reqlist.add_request(req) @pyqtSlot() def new_macro(self): fname = save_dialog(self, filter_string="Python File (*.py)") if not fname: return with open(fname, 'w') as f: contents = new_active_macro() f.write(contents) self.tableModel.add_macro(fname) @pyqtSlot() def browse_macro(self): fname = open_dialog(self, filter_string="Python File (*.py)") if not fname: return self.tableModel.add_macro(fname) @pyqtSlot() def remove_selected(self): rows = self.tableView.selectionModel().selectedRows() if len(rows) == 0: return for idx in rows: row = idx.row() self.tableModel.remove_macro(row) return @pyqtSlot() def run_selected_macro(self): rows = self.tableView.selectionModel().selectedRows() if len(rows) == 0: return for idx in rows: row = idx.row() reqs = self.reqlist.get_all_requests() self.tableModel.run_macro(row, reqs) return @pyqtSlot(HTTPRequest) def add_request_output(self, req): self.outreqlist.listWidg.add_request(req) @pyqtSlot() def clear_output(self): self.outreqlist.set_requests([]) @pyqtSlot() def clear_text_output(self): self.macro_text_out.setPlainText("") @pyqtSlot(str) def add_macro_output(self, s): t = self.macro_text_out.toPlainText() t += s self.macro_text_out.setPlainText(t) @pyqtSlot() def import_all_reqs(self): reqs = self.client.in_context_requests(headers_only=True) self.add_requests(reqs) class MacroErrWindow(QWidget): def __init__(self, *args, **kwargs): QObject.__init__(self, *args, **kwargs) self.msg = "" self.setLayout(QVBoxLayout()) self.msgwidg = QPlainTextEdit() self.layout().addWidget(self.msgwidg) def add_error(self, msg): self.msg += msg + "\n\n" self.msgwidg.setPlainText(self.msg) self.show() def closeEvent(self, event): self.msgwidg.setPlainText("") IntMacroListModel.err_window = None ActiveMacroModel.err_window = None def make_err_str(macro, e): estr = "Exception in macro %s:\n" % macro.fname estr += str(e) + '\n' estr += str(traceback.format_exc()) return estr class ArgWindow(QDialog): def __init__(self, parent, argspec, cached=None): QDialog.__init__(self, parent) winLayout = QVBoxLayout() formLayout = QFormLayout() self.shownargs = [] self.canceled = False argnames = set() for spec in argspec: name = None argtype = None argval = None if isinstance(spec, str): name = spec argtype = "str" else: if len(spec) > 0: name = spec[0] if len(spec) > 1: argtype = spec[1] if len(spec) > 2: argval = spec[2] if not name: continue if not argtype: continue if name in argnames: continue widg = None if argtype.lower() in ("str", "string"): argtype = "str" widg = QLineEdit() if name in cached: widg.setText(cached[name]) else: return formLayout.addRow(QLabel(name), widg) self.shownargs.append(((name, argtype, argval), widg)) argnames.add(name) butlayout = QHBoxLayout() okbut = QPushButton("Ok") okbut.clicked.connect(self.accept) cancelbut = QPushButton("Cancel") cancelbut.clicked.connect(self.reject) self.rejected.connect(self._set_canceled) butlayout.addWidget(okbut) butlayout.addWidget(cancelbut) butlayout.addStretch() winLayout.addLayout(formLayout) winLayout.addLayout(butlayout) self.setLayout(winLayout) @pyqtSlot() def _set_canceled(self): self.canceled = True def get_args(self): if self.canceled: return None retargs = {} for shownarg in self.shownargs: spec, widg = shownarg name, argtype, typeargs = spec if argtype == "str": retargs[name] = widg.text() return retargs def get_macro_args(parent, argspec, cached=None): if not isinstance(argspec, list): return argwin = ArgWindow(parent, argspec, cached=cached) argwin.exec_() return argwin.get_args() def req_python_def(varname, req): method = req.method path = req.url.geturl() pmajor = req.proto_major pminor = req.proto_minor headers = req.headers.dict().items() dest_host = req.dest_host dest_port = req.dest_port if req.use_tls: use_tls = "True" else: use_tls = "False" body = "" if len(req.body) > 0: s = '"' if b'\n' in req.body: s = '"""' for c in req.body: if chr(c) in qtprintable: body += chr(c) else: body += "\\x%02x" % c body = "%s%s%s" % (s, body, s) ret = '' ret += '%s = HTTPRequest(' % varname ret += 'proto_major=%d, proto_minor=%d,\n' % (pmajor, pminor) ret += ' use_tls=%s, dest_host="%s", dest_port=%d,\n' % (use_tls, dest_host, dest_port) ret += ' method="%s", path="%s", headers={\n' % (method, path) for k, vs in headers: qvs = [] for v in vs: qvs.append('"%s"' % v) vstr = "[" + ", ".join(qvs) + "]" ret += ' "%s": %s,\n' % (k, vstr) ret += ' },\n' if len(body) > 0: ret += ' body=%s\n' % body ret += ")" return ret def create_macro_template(reqs): ret = "from guppyproxy.proxy import HTTPRequest\n\n" i = 0 for req in reqs: ret += req_python_def("req%d"%i, req) ret += "\n\n" i += 1 ret += "def run_macro(client, args, reqs):\n" if i == 0: ret += " pass\n" for ii in range(i): ret += " client.submit(req%d)\n" % ii ret += " client.output_req(req%d)\n\n" % ii return ret def new_active_macro(): return "def run_macro(client, args, reqs):\n # Macro code goes here\n pass" def new_int_macro(): return """def mangle_request(client, args, req): # modify request here return req def mangle_response(client, args, req, rsp): # modify response here return rsp """ ================================================ FILE: guppyproxy/proxy.py ================================================ #!/usr/bin/env python3 import base64 import copy import datetime import json import math import re import socket import threading from collections import namedtuple from itertools import count from urllib.parse import urlparse, ParseResult, parse_qs, urlencode from subprocess import Popen, PIPE from http import cookies as hcookies from PyQt5.QtCore import QThread, QObject, pyqtSlot class MessageError(Exception): pass class ProxyException(Exception): pass class InvalidQuery(Exception): pass class SocketClosed(Exception): pass class SockBuffer: # I can't believe I have to implement this def __init__(self, sock): self.buf = [] # a list of chunks of strings self.s = sock self.closed = False def close(self): try: self.s.shutdown(socket.SHUT_RDWR) self.s.close() except OSError: # already closed pass finally: self.closed = True def _check_newline(self): for chunk in self.buf: if '\n' in chunk: return True return False def readline(self): # Receive until we get a newline, raise SocketClosed if socket is closed while True: try: data = self.s.recv(256) except OSError: raise SocketClosed() if not data: raise SocketClosed() self.buf.append(data) if b'\n' in data: break # Combine chunks allbytes = b''.join(self.buf) head, tail = allbytes.split(b'\n', 1) self.buf = [tail] return head.decode() def send(self, data): try: self.s.send(data) except OSError: raise SocketClosed() class ProxyThread(QThread): threads = {} tiditer = count() def __init__(self, target=None, args=tuple()): global mainWidg QThread.__init__(self) self.f = target self.args = args self.tid = next(ProxyThread.tiditer) ProxyThread.threads[self.tid] = self self.finished.connect(clean_thread(self.tid)) def run(self): self.f(*self.args) def wait(self): QThread.wait(self) @classmethod def waitall(cls): ts = [(tid, thread) for tid, thread in cls.threads.items()] for tid, thread in ts: thread.wait() def clean_thread(tid): @pyqtSlot() def clean(): del ProxyThread.threads[tid] return clean class Headers: def __init__(self, headers=None): self.headers = {} if headers is not None: if isinstance(headers, Headers): for _, pairs in headers.headers.items(): for k, v in pairs: self.add(k, v) else: for k, vs in headers.items(): for v in vs: self.add(k, v) def __contains__(self, hd): for k, _ in self.headers.items(): if k.lower() == hd.lower(): return True return False def add(self, k, v): try: lst = self.headers[k.lower()] lst.append((k, v)) except KeyError: self.headers[k.lower()] = [(k, v)] def set(self, k, v): self.headers[k.lower()] = [(k, v)] def get(self, k): return self.headers[k.lower()][0][1] def delete(self, k): try: del self.headers[k.lower()] except KeyError: pass def pairs(self, key=None): for _, kvs in self.headers.items(): for k, v in kvs: if key is None or k.lower() == key.lower(): yield (k, v) def dict(self): retdict = {} for _, kvs in self.headers.items(): for k, v in kvs: if k in retdict: retdict[k].append(v) else: retdict[k] = [v] return retdict class RequestContext: def __init__(self, client, query=None): self._current_query = [] self.client = client if query is not None: self._current_query = query def _validate(self, query): self.client.validate_query(query) def set_query(self, query): self._validate(query) self._current_query = query def apply_phrase(self, phrase): self._validate([phrase]) self._current_query.append(phrase) def pop_phrase(self): if len(self._current_query) > 0: self._current_query.pop() def apply_filter(self, filt): self._validate([[filt]]) self._current_query.append([filt]) @property def query(self): return copy.deepcopy(self._current_query) class URL: def __init__(self, url): parsed = urlparse(url) if url is not None: parsed = urlparse(url) self.scheme = parsed.scheme self.netloc = parsed.netloc self.path = parsed.path self.params = parsed.params self.query = parsed.query self.fragment = parsed.fragment else: self.scheme = "" self.netloc = "" self.path = "/" self.params = "" self.query = "" self.fragment = "" def geturl(self, include_params=True): params = self.params query = self.query fragment = self.fragment if not include_params: params = "" query = "" fragment = "" r = ParseResult(scheme=self.scheme, netloc=self.netloc, path=self.path, params=params, query=query, fragment=fragment) return r.geturl() def parameters(self): try: return parse_qs(self.query, keep_blank_values=True) except Exception: return {} def param_iter(self): for k, vs in self.parameters().items(): for v in vs: yield k, v def set_param(self, key, val): params = self.parameters() params[key] = [val] self.query = urlencode(params) def add_param(self, key, val): params = self.parameters() if key in params: params[key].append(val) else: params[key] = [val] self.query = urlencode(params) def del_param(self, key): params = self.parameters() del params[key] self.query = urlencode(params) def set_params(self, params): self.query = urlencode(params) class InterceptMacro: """ A class representing a macro that modifies requests as they pass through the proxy """ def __init__(self): self.name = '' self.intercept_requests = False self.intercept_responses = False self.intercept_ws = False def __repr__(self): return "" % self.name def mangle_request(self, request): return request def mangle_response(self, request, response): return response def mangle_websocket(self, request, response, message): return message class HTTPRequest: def __init__(self, method="GET", path="/", proto_major=1, proto_minor=1, headers=None, body=bytes(), dest_host="", dest_port=80, use_tls=False, time_start=None, time_end=None, db_id="", tags=None, headers_only=False, storage_id=0): # http info self.method = method self.url = URL(path) self.proto_major = proto_major self.proto_minor = proto_minor self.headers = Headers(headers) self.headers_only = headers_only self._body = bytes() if not headers_only: self.body = body # metadata self.dest_host = dest_host self.dest_port = dest_port self.use_tls = use_tls self.time_start = time_start self.time_end = time_end self.response = None self.unmangled = None self.ws_messages = [] self.db_id = db_id self.storage_id = storage_id if tags is not None: self.tags = set(tags) else: self.tags = set() @property def body(self): return self._body @body.setter def body(self, bs): self.headers_only = False if type(bs) is str: self._body = bs.encode() elif type(bs) is bytes: self._body = bs else: raise Exception("invalid body type: {}".format(type(bs))) self.headers.set("Content-Length", str(len(self._body))) @property def content_length(self): if 'content-length' in self.headers: return int(self.headers.get('content-length')) return len(self.body) def status_line(self): sline = "{method} {path} HTTP/{proto_major}.{proto_minor}".format( method=self.method, path=self.url.geturl(), proto_major=self.proto_major, proto_minor=self.proto_minor).encode() return sline def headers_section(self): message = self.status_line() + b"\r\n" for k, v in self.headers.pairs(): message += "{}: {}\r\n".format(k, v).encode() return message def full_message(self): message = self.headers_section() message += b"\r\n" message += self.body return message def parameters(self): try: return parse_qs(self.body.decode(), keep_blank_values=True) except Exception: return {} def param_iter(self, ignore_content_type=False): if not ignore_content_type: if "content-type" not in self.headers: return if "www-form-urlencoded" not in self.headers.get("content-type").lower(): return for k, vs in self.parameters().items(): for v in vs: yield k, v def set_param(self, key, val): params = self.parameters() params[key] = [val] self.body = urlencode(params) def add_param(self, key, val): params = self.parameters() if key in params: params[key].append(val) else: params[key] = [val] self.body = urlencode(params) def del_param(self, key): params = self.parameters() del params[key] self.body = urlencode(params) def set_params(self, params): self.body = urlencode(params) def cookies(self): try: cookie = hcookies.BaseCookie() cookie.load(self.headers.get("cookie")) return cookie except Exception as e: return hcookies.BaseCookie() def cookie_iter(self): c = self.cookies() for k in c: yield k, c[k].value def set_cookie(self, key, val): c = self.cookies() c[key] = val self.set_cookies(c) def del_cookie(self, key): c = self.cookies() del c[key] self.set_cookies(c) def set_cookies(self, c): if isinstance(c, hcookies.BaseCookie): # it's a basecookie cookie_pairs = [] for k in c: cookie_pairs.append('{}={}'.format(k, c[k].value)) header_str = '; '.join(cookie_pairs) elif isinstance(c, HTTPRequest): # it's a request we should copy cookies from try: header_str = c.headers.get("Cookie") except KeyError: header_str = "" else: # it's a dictionary cookie_pairs = [] for k, v in c.items(): cookie_pairs.append('{}={}'.format(k, v)) header_str = '; '.join(cookie_pairs) if header_str == '': try: self.headers.delete("Cookie") except KeyError: pass else: self.headers.set("Cookie", header_str) def add_cookies(self, c): new_cookies = self.cookies() if isinstance(c, hcookies.BaseCookie): for k in c: new_cookies[k] = c[k].value elif isinstance(c, HTTPRequest): for k, v in c.cookie_iter(): new_cookies[k] = v elif isinstance(c, HTTPResponse): for k, v in c.cookie_iter(): new_cookies[k] = v else: for k, v in c.items(): new_cookies[k] = v self.set_cookies(new_cookies) def full_url(self): return get_full_url(self) def copy(self): return HTTPRequest( method=self.method, path=self.url.geturl(), proto_major=self.proto_major, proto_minor=self.proto_minor, headers=self.headers, body=self.body, dest_host=self.dest_host, dest_port=self.dest_port, use_tls=self.use_tls, tags=copy.deepcopy(self.tags), headers_only=self.headers_only, ) class HTTPResponse: def __init__(self, status_code=200, reason="OK", proto_major=1, proto_minor=1, headers=None, body=bytes(), db_id="", headers_only=False, storage_id=0): self.status_code = status_code self.reason = reason self.proto_major = proto_major self.proto_minor = proto_minor self.headers = Headers() if headers is not None: for k, vs in headers.items(): for v in vs: self.headers.add(k, v) self.headers_only = headers_only self._body = bytes() if not headers_only: self.body = body self.unmangled = None self.db_id = db_id self.storage = storage_id @property def body(self): return self._body @body.setter def body(self, bs): self.headers_only = False if type(bs) is str: self._body = bs.encode() elif type(bs) is bytes: self._body = bs else: raise Exception("invalid body type: {}".format(type(bs))) self.headers.set("Content-Length", str(len(self._body))) @property def content_length(self): if 'content-length' in self.headers: return int(self.headers.get('content-length')) return len(self.body) def status_line(self): sline = "HTTP/{proto_major}.{proto_minor} {status_code} {reason}".format( proto_major=self.proto_major, proto_minor=self.proto_minor, status_code=self.status_code, reason=self.reason).encode() return sline def headers_section(self): message = self.status_line() + b"\r\n" for k, v in self.headers.pairs(): message += "{}: {}\r\n".format(k, v).encode() return message def full_message(self): message = self.headers_section() message += b"\r\n" message += self.body return message def cookies(self): try: cookie = hcookies.BaseCookie() for _, v in self.headers.pairs('set-cookie'): cookie.load(v) return cookie except Exception as e: return hcookies.BaseCookie() def cookie_iter(self): c = self.cookies() for k in c: yield k, c[k].value def set_cookie(self, key, val): c = self.cookies() c[key] = val self.set_cookies(c) def del_cookie(self, key): c = self.cookies() del c[key] self.set_cookies(c) def set_cookies(self, c): self.headers.delete("set-cookie") if isinstance(c, hcookies.BaseCookie): cookies = c else: cookies = hcookies.BaseCookie() for k, v in c.items(): cookies[k] = v for _, m in c.items(): self.headers.add("Set-Cookie", m.OutputString()) def copy(self): return HTTPResponse( status_code=self.status_code, reason=self.reason, proto_major=self.proto_major, proto_minor=self.proto_minor, headers=self.headers.headers, body=self.body, headers_only=self.headers_only, ) class WSMessage: def __init__(self, is_binary=True, message=bytes(), to_server=True, timestamp=None, db_id="", storage_id=0): self.is_binary = is_binary self.message = message self.to_server = to_server self.timestamp = timestamp or datetime.datetime(1970, 1, 1) self.unmangled = None self.db_id = db_id self.storage = storage_id def copy(self): return WSMessage( is_binary=self.is_binary, message=self.message, to_server=self.to_server, ) ScopeResult = namedtuple("ScopeResult", ["is_custom", "filter"]) ListenerResult = namedtuple("ListenerResult", ["lid", "addr"]) GenPemCertsResult = namedtuple("GenPemCertsResult", ["key_pem", "cert_pem"]) SavedQuery = namedtuple("SavedQuery", ["name", "query"]) SavedStorage = namedtuple("SavedStorage", ["storage_id", "description"]) def messagingFunction(func): def f(self, *args, **kwargs): if self.is_interactive: raise MessageError("cannot be called while other message is interactive") if self.closed: raise MessageError("connection is closed") with self.message_lock: return func(self, *args, **kwargs) return f class ProxyConnection: next_id = 1 def __init__(self, kind="", addr=""): self.connid = ProxyConnection.next_id ProxyConnection.next_id += 1 self.sbuf = None self.buf = bytes() self.parent_client = None self.debug = False self.is_interactive = False self.closed = True self.message_lock = threading.Lock() self.kind = None self.addr = None self.int_thread = None if kind.lower() == "tcp": tcpaddr, port = addr.rsplit(":", 1) self.connect_tcp(tcpaddr, int(port)) elif kind.lower() == "unix": self.connect_unix(addr) def __enter__(self): return self def __exit__(self, exc_type, exc_value, traceback): self.close() def connect_tcp(self, addr, port): s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.connect((addr, port)) self.sbuf = SockBuffer(s) self.closed = False self.kind = "tcp" self.addr = "{}:{}".format(addr, port) def connect_unix(self, addr): s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) s.connect(addr) self.sbuf = SockBuffer(s) self.closed = False self.kind = "unix" self.addr = addr @property def maddr(self): if self.kind is not None: return "{}:{}".format(self.kind, self.addr) else: return None def close(self): self.sbuf.close() if self.parent_client is not None: try: self.parent_client.conns.remove(self) except KeyError: pass self.closed = True def read_message(self): ln = self.sbuf.readline() if self.debug: print("<({}) {}".format(self.connid, ln)) j = json.loads(ln) if ("Success" in j) and (j["Success"] is False): if "Reason" in j: raise MessageError(j["Reason"]) raise MessageError("unknown error") return j def submit_command(self, cmd): ln = json.dumps(cmd).encode() + b"\n" if self.debug: print(">({}) {}".format(self.connid, ln.decode()[:-1])) self.sbuf.send(ln) def reqrsp_cmd(self, cmd): self.submit_command(cmd) ret = self.read_message() if ret is None: raise Exception() return ret ########### # Commands @messagingFunction def ping(self): cmd = {"Command": "Ping"} result = self.reqrsp_cmd(cmd) return result["Ping"] @messagingFunction def submit(self, req, storage=0): cmd = { "Command": "Submit", "Request": encode_req(req), "Storage": 0, } if storage is not None: cmd["Storage"] = storage result = self.reqrsp_cmd(cmd) if "SubmittedRequest" not in result: raise MessageError("no request returned") newreq = decode_req(result["SubmittedRequest"], storage=storage) req.response = newreq.response req.unmangled = newreq.unmangled req.time_start = newreq.time_start req.time_end = newreq.time_end req.db_id = newreq.db_id req.storage_id = storage @messagingFunction def save_new(self, req, storage): cmd = { "Command": "SaveNew", "Request": encode_req(req), "Storage": storage, } result = self.reqrsp_cmd(cmd) req.db_id = result["DbId"] req.storage_id = storage return result["DbId"] def _query_storage(self, q, storage, headers_only=False, max_results=0): cmd = { "Command": "StorageQuery", "Query": q, "HeadersOnly": headers_only, "MaxResults": max_results, "Storage": storage, } result = self.reqrsp_cmd(cmd) reqs = [] unmangled = set() for reqd in result["Results"]: req = decode_req(reqd, headers_only=headers_only, storage=storage) req.storage_id = storage reqs.append(req) if req.unmangled is not None: unmangled.add(req.unmangled.db_id) return [r for r in reqs if r.db_id not in unmangled] @messagingFunction def query_storage(self, q, storage, max_results=0, headers_only=False): return self._query_storage(q, storage, headers_only=headers_only, max_results=max_results) @messagingFunction def req_by_id(self, reqid, storage, headers_only=False): results = self._query_storage([[["dbid", "is", reqid]]], storage, headers_only=headers_only, max_results=1) if len(results) == 0: raise MessageError("request with id {} does not exist".format(reqid)) return results[0] @messagingFunction def set_scope(self, filt): cmd = { "Command": "SetScope", "Query": filt, } self.reqrsp_cmd(cmd) @messagingFunction def get_scope(self): cmd = { "Command": "ViewScope", } result = self.reqrsp_cmd(cmd) ret = ScopeResult(result["IsCustom"], result["Query"]) return ret @messagingFunction def add_tag(self, reqid, tag, storage): cmd = { "Command": "AddTag", "ReqId": reqid, "Tag": tag, "Storage": storage, } self.reqrsp_cmd(cmd) @messagingFunction def remove_tag(self, reqid, tag, storage): cmd = { "Command": "RemoveTag", "ReqId": reqid, "Tag": tag, "Storage": storage, } self.reqrsp_cmd(cmd) @messagingFunction def clear_tag(self, reqid, storage): cmd = { "Command": "ClearTag", "ReqId": reqid, "Storage": storage, } self.reqrsp_cmd(cmd) @messagingFunction def all_saved_queries(self, storage): cmd = { "Command": "AllSavedQueries", "Storage": storage, } results = self.reqrsp_cmd(cmd) queries = [] for result in results["Queries"]: queries.append(SavedQuery(name=result["Name"], query=result["Query"])) return queries @messagingFunction def save_query(self, name, filt, storage): cmd = { "Command": "SaveQuery", "Name": name, "Query": filt, "Storage": storage, } self.reqrsp_cmd(cmd) @messagingFunction def load_query(self, name, storage): cmd = { "Command": "LoadQuery", "Name": name, "Storage": storage, } result = self.reqrsp_cmd(cmd) return result["Query"] @messagingFunction def delete_query(self, name, storage): cmd = { "Command": "DeleteQuery", "Name": name, "Storage": storage, } self.reqrsp_cmd(cmd) @messagingFunction def add_listener(self, addr, port, transparent=False, destHost="", destPort=0, destUseTLS=False): laddr = "{}:{}".format(addr, port) cmd = { "Command": "AddListener", "Type": "tcp", "Addr": laddr, "TransparentMode": transparent, "DestHost": destHost, "DestPort": destPort, "DestUseTLS": destUseTLS, } result = self.reqrsp_cmd(cmd) lid = result["Id"] return lid @messagingFunction def remove_listener(self, lid): cmd = { "Command": "RemoveListener", "Id": lid, } self.reqrsp_cmd(cmd) @messagingFunction def get_listeners(self): cmd = { "Command": "GetListeners", } result = self.reqrsp_cmd(cmd) results = [] for r in result["Results"]: results.append((r["Id"], r["Addr"])) return results @messagingFunction def load_certificates(self, cert_file, pkey_file): cmd = { "Command": "LoadCerts", "KeyFile": pkey_file, "CertificateFile": cert_file, } self.reqrsp_cmd(cmd) @messagingFunction def set_certificates(self, pkey_pem, cert_pem): cmd = { "Command": "SetCerts", "KeyPEMData": pkey_pem, "CertificatePEMData": cert_pem, } self.reqrsp_cmd(cmd) @messagingFunction def clear_certificates(self): cmd = { "Command": "ClearCerts", } self.reqrsp_cmd(cmd) @messagingFunction def generate_certificates(self, pkey_file, cert_file): cmd = { "Command": "GenCerts", "KeyFile": pkey_file, "CertFile": cert_file, } self.reqrsp_cmd(cmd) @messagingFunction def generate_pem_certificates(self): cmd = { "Command": "GenPEMCerts", } result = self.reqrsp_cmd(cmd) ret = GenPemCertsResult(result["KeyPEMData"], result["CertificatePEMData"]) return ret @messagingFunction def validate_query(self, query): cmd = { "Command": "ValidateQuery", "Query": query, } try: self.reqrsp_cmd(cmd) except MessageError as e: raise InvalidQuery(str(e)) @messagingFunction def check_request(self, query, req=None, storage_id=-1, db_id=""): cmd = { "Command": "checkrequest", "Query": query, } if req: cmd["Request"] = encode_req(req) if db_id != "": cmd["DbId"] = db_id cmd["StorageId"] = storage_id result = self.reqrsp_cmd(cmd) return result["Result"] @messagingFunction def add_sqlite_storage(self, path, desc): cmd = { "Command": "AddSQLiteStorage", "Path": path, "Description": desc } result = self.reqrsp_cmd(cmd) return result["StorageId"] @messagingFunction def add_in_memory_storage(self, desc): cmd = { "Command": "AddInMemoryStorage", "Description": desc } result = self.reqrsp_cmd(cmd) return result["StorageId"] @messagingFunction def close_storage(self, storage_id): cmd = { "Command": "CloseStorage", "StorageId": storage_id, } self.reqrsp_cmd(cmd) @messagingFunction def set_proxy_storage(self, storage_id): cmd = { "Command": "SetProxyStorage", "StorageId": storage_id, } self.reqrsp_cmd(cmd) @messagingFunction def list_storage(self): cmd = { "Command": "ListStorage", } result = self.reqrsp_cmd(cmd) ret = [] for ss in result["Storages"]: ret.append(SavedStorage(ss["Id"], ss["Description"])) return ret @messagingFunction def set_proxy(self, use_proxy=False, proxy_host="", proxy_port=0, use_creds=False, username="", password="", is_socks=False): cmd = { "Command": "SetProxy", "UseProxy": use_proxy, "ProxyHost": proxy_host, "ProxyPort": proxy_port, "ProxyIsSOCKS": is_socks, "UseCredentials": use_creds, "Username": username, "Password": password, } self.reqrsp_cmd(cmd) @messagingFunction def intercept(self, macro): # Run an intercepting macro until closed # Start intercepting self.is_interactive = True cmd = { "Command": "Intercept", "InterceptRequests": macro.intercept_requests, "InterceptResponses": macro.intercept_responses, "InterceptWS": macro.intercept_ws, } try: self.reqrsp_cmd(cmd) except Exception as e: self.is_interactive = False raise e def run_macro(): iditer = count() threads = {} while True: try: msg = self.read_message() except MessageError as e: return except SocketClosed: return def mangle_and_respond(msg): retCmd = None if msg["Type"] == "httprequest": req = decode_req(msg["Request"]) newReq = macro.mangle_request(req) if newReq is None: retCmd = { "Id": msg["Id"], "Dropped": True, } else: newReq.unmangled = None newReq.response = None newReq.ws_messages = [] retCmd = { "Id": msg["Id"], "Dropped": False, "Request": encode_req(newReq), } elif msg["Type"] == "httpresponse": req = decode_req(msg["Request"]) rsp = decode_rsp(msg["Response"]) newRsp = macro.mangle_response(req, rsp) if newRsp is None: retCmd = { "Id": msg["Id"], "Dropped": True, } else: newRsp.unmangled = None retCmd = { "Id": msg["Id"], "Dropped": False, "Response": encode_rsp(newRsp), } elif msg["Type"] == "wstoserver" or msg["Type"] == "wstoclient": req = decode_req(msg["Request"]) rsp = decode_rsp(msg["Response"]) wsm = decode_ws(msg["WSMessage"]) newWsm = macro.mangle_websocket(req, rsp, wsm) if newWsm is None: retCmd = { "Id": msg["Id"], "Dropped": True, } else: newWsm.unmangled = None retCmd = { "Id": msg["Id"], "Dropped": False, "WSMessage": encode_ws(newWsm), } else: raise Exception("Unknown message type: " + msg["Type"]) if retCmd is not None: try: self.submit_command(retCmd) except SocketClosed: return tid = next(iditer) mangle_thread = ProxyThread(target=mangle_and_respond, args=(msg,)) threads[tid] = mangle_thread mangle_thread.start() self.int_thread = ProxyThread(target=run_macro) self.int_thread.start() @messagingFunction def watch_storage(self, storage_id=-1, headers_only=True): # Generator that generates request, response, and wsmessages as they # are stored by the proxy cmd = { "Command": "WatchStorage", "StorageId": storage_id, "HeadersOnly": headers_only, } try: self.reqrsp_cmd(cmd) except Exception as e: self.is_interactive = False raise e while True: msg = self.read_message() if msg["Request"]: msg["Request"] = decode_req(msg["Request"], storage=msg["StorageId"], headers_only=headers_only) if msg["Response"]: msg["Response"] = decode_rsp(msg["Response"], storage=msg["StorageId"], headers_only=headers_only) if msg["WSMessage"]: msg["WSMessage"] = decode_ws(msg["WSMessage"], storage=msg["StorageId"], headers_only=headers_only) yield msg @messagingFunction def set_plugin_value(self, key, value, storage_id): cmd = { "Command": "SetPluginValue", "Storage": storage_id, "Key": key, "Value": value, } self.reqrsp_cmd(cmd) @messagingFunction def get_plugin_value(self, key, storage_id): cmd = { "Command": "GetPluginValue", "Storage": storage_id, "Key": key, } result = self.reqrsp_cmd(cmd) return result["Value"] ActiveStorage = namedtuple("ActiveStorage", ["type", "storage_id", "prefix"]) def _serialize_storage(stype, prefix): return "{}|{}".format(stype, prefix) class ProxyClient: def __init__(self, binary=None, debug=False, conn_addr=None): self.binloc = binary self.proxy_proc = None self.ltype = None self.laddr = None self.debug = debug self.conn_addr = conn_addr self.conns = set() self.msg_conn = None # conn for single req/rsp messages self.context = RequestContext(self) self.storage_by_id = {} self.storage_by_prefix = {} self.proxy_storage = None self.inmem_storage = None self.reqrsp_methods = { "submit_command", # "reqrsp_cmd", "ping", # "submit", # "save_new", # "query_storage", # "req_by_id", "set_scope", "get_scope", # "add_tag", # "remove_tag", # "clear_tag", "all_saved_queries", "save_query", "load_query", "delete_query", "add_listener", "remove_listener", "get_listeners", "load_certificates", "set_certificates", "clear_certificates", "generate_certificates", "generate_pem_certificates", "validate_query", # "check_request", "list_storage", # "add_sqlite_storage", # "add_in_memory_storage", # "close_storage", # "set_proxy_storage", "set_proxy", # "set_plugin_value", # "get_plugin_value", } def __enter__(self): if self.conn_addr is not None: self.msg_connect(self.conn_addr) else: self.execute_binary(binary=self.binloc, debug=self.debug) return self def __exit__(self, exc_type, exc_value, traceback): self.close() def __getattr__(self, name): if name in self.reqrsp_methods: return getattr(self.msg_conn, name) raise NotImplementedError(name) @property def maddr(self): if self.ltype is not None: return "{}:{}".format(self.ltype, self.laddr) else: return None def execute_binary(self, binary=None, debug=False, listen_addr=None): self.binloc = binary args = [self.binloc] if listen_addr is not None: args += ["--msglisten", listen_addr] else: args += ["--msgauto"] if debug: args += ["--dbg"] self.proxy_proc = Popen(args, stdout=PIPE, stderr=PIPE) # Wait for it to start and make connection listenstr = self.proxy_proc.stdout.readline().rstrip() self.msg_connect(listenstr.decode()) def msg_connect(self, addr): self.ltype, self.laddr = addr.split(":", 1) self.msg_conn = self.new_conn() self._get_storage() def close(self): conns = list(self.conns) for conn in conns: conn.close() if self.proxy_proc is not None: self.proxy_proc.terminate() def new_conn(self): conn = ProxyConnection(kind=self.ltype, addr=self.laddr) conn.parent_client = self conn.debug = self.debug self.conns.add(conn) return conn # functions involving storage def _add_storage(self, storage, prefix): self.storage_by_prefix[prefix] = storage self.storage_by_id[storage.storage_id] = storage def _clear_storage(self): self.storage_by_prefix = {} self.storage_by_id = {} def _get_storage(self): self._clear_storage() storages = self.list_storage() for s in storages: stype, prefix = s.description.split("|") storage = ActiveStorage(stype, s.storage_id, prefix) self._add_storage(storage, prefix) def parse_reqid(self, reqid): if reqid[0].isalpha(): prefix = reqid[0] realid = reqid[1:] else: prefix = "" realid = reqid # `u`, `s` are special cases for the unmangled version of req and rsp if prefix == 'u': req = self.req_by_id(realid) if req.unmangled is None: raise MessageError("request %s was not mangled" % reqid) ureq = req.unmangled return self.storage_by_id[ureq.storage_id], ureq.db_id elif prefix == 's': req = self.req_by_id(realid) if req.response is None: raise MessageError("response %s was not mangled" % reqid) if req.response.unmangled is None: raise MessageError("response %s was not mangled" % reqid) return self.storage_by_id[req.storage_id], req.db_id else: storage = self.storage_by_prefix[prefix] return storage, realid def storage_iter(self): for _, s in self.storage_by_id.items(): yield s def _stg_or_def(self, storage): if storage is None: return self.proxy_storage return storage def is_in_context(self, req): return self.check_request(self.context.query, req) def in_context_requests(self, headers_only=False, max_results=0): return self.query_storage(self.context.query, headers_only=headers_only, max_results=max_results) def in_context_requests_async(self, slot, headers_only=False, max_results=0, *args, **kwargs): return self.query_storage(slot, self.context.query, headers_only=headers_only, max_results=max_results) def in_context_requests_iter(self, headers_only=False, max_results=0): results = self.query_storage(self.context.query, headers_only=headers_only, max_results=max_results) ret = results if max_results > 0 and len(results) > max_results: ret = results[:max_results] for reqh in ret: req = self.req_by_id(reqh.db_id, storage_id=reqh.storage_id) yield req def get_reqid(self, req): prefix = "" if req.storage_id in self.storage_by_id: s = self.storage_by_id[req.storage_id] prefix = s.prefix return "{}{}".format(prefix, req.db_id) def load_by_reqheaders(self, req): reqid = self.get_reqid(req) return self.req_by_id(reqid) # functions that don't just pass through to underlying conn def add_sqlite_storage(self, path, prefix): desc = _serialize_storage("sqlite", prefix) sid = self.msg_conn.add_sqlite_storage(path, desc) s = ActiveStorage(type="sqlite", storage_id=sid, prefix=prefix) self._add_storage(s, prefix) return s def add_in_memory_storage(self, prefix): desc = _serialize_storage("inmem", prefix) sid = self.msg_conn.add_in_memory_storage(desc) s = ActiveStorage(type="inmem", storage_id=sid, prefix=prefix) self._add_storage(s, prefix) return s def close_storage(self, storage_id): s = self.storage_by_id[storage_id] self.msg_conn.close_storage(s.storage_id) del self.storage_by_id[s.storage_id] del self.storage_by_prefix[s.prefix] def set_proxy_storage(self, storage_id): s = self.storage_by_id[storage_id] self.msg_conn.set_proxy_storage(s.storage_id) self.proxy_storage = storage_id def set_storage_prefix(self, storage_id, prefix): if prefix in self.storage_by_prefix: raise Exception("prefix already exists") s = self.storage_by_id[storage_id] del self.storage_by_prefix[s.prefix] news = ActiveStorage(type=s.type, prefix=prefix, storage_id=s.storage_id) self.storage_by_prefix[news.prefix] = news self.storage_by_id[storage_id] = news def save_new(self, req, inmem=False, storage=None): if inmem: storage = self.inmem_storage else: storage = self._stg_or_def(storage) self.msg_conn.save_new(req, storage=storage) def submit(self, req, save=False, inmem=False, storage=None): if save: storage = self._stg_or_def(storage) if inmem: storage = self.inmem_storage self.msg_conn.submit(req, storage=storage) def query_storage(self, q, max_results=0, headers_only=False, storage=None, conn=None): results = [] conn = conn or self.msg_conn if storage is None: for s in self.storage_iter(): results += conn.query_storage(q, max_results=max_results, headers_only=headers_only, storage=s.storage_id) else: results += conn.query_storage(q, max_results=max_results, headers_only=headers_only, storage=storage) def kfunc(req): if req.time_start is None: return datetime.datetime.utcfromtimestamp(0) return req.time_start results.sort(key=kfunc) results = [r for r in reversed(results)] return results def query_storage_async(self, slot, *args, **kwargs): def perform_query(): try: with self.new_conn() as c: r = self.query_storage(*args, conn=c, **kwargs) slot.emit(r) except Exception: pass ProxyThread(target=perform_query).start() def req_by_id(self, reqid, storage_id=None, headers_only=False): if storage_id is None: storage, db_id = self.parse_reqid(reqid) storage_id = storage.storage_id else: db_id = reqid retreq = self.msg_conn.req_by_id(db_id, headers_only=headers_only, storage=storage_id) if reqid[0] == 's': # `u` is handled by parse_reqid retreq.response = retreq.response.unmangled return retreq def check_request(self, query, req=None, reqid=""): if req is not None: return self.msg_conn.check_request(query, req=req) else: storage, db_id = self.parse_reqid(reqid) storage_id = storage.storage_id return self.msg_conn.check_request(query, storage_id=storage_id, db_id=db_id) raise Exception("check_request requires either a request or reqid") # for these and submit, might need storage stored on the request itself def add_tag(self, reqid, tag, storage=None): self.msg_conn.add_tag(reqid, tag, storage=self._stg_or_def(storage)) def remove_tag(self, reqid, tag, storage=None): self.msg_conn.remove_tag(reqid, tag, storage=self._stg_or_def(storage)) def clear_tag(self, reqid, storage=None): self.msg_conn.clear_tag(reqid, storage=self._stg_or_def(storage)) def all_saved_queries(self, storage=None): self.msg_conn.all_saved_queries(storage=None) def save_query(self, name, filt, storage=None): self.msg_conn.save_query(name, filt, storage=self._stg_or_def(storage)) def load_query(self, name, storage=None): self.msg_conn.load_query(name, storage=self._stg_or_def(storage)) def delete_query(self, name, storage=None): self.msg_conn.delete_query(name, storage=self._stg_or_def(storage)) def set_plugin_value(self, key, value, storage=None): self.msg_conn.set_plugin_value(key, value, self._stg_or_def(storage)) def get_plugin_value(self, key, storage=None): return self.msg_conn.get_plugin_value(key, self._stg_or_def(storage)) def decode_req(result, headers_only=False, storage=0): if "StartTime" in result and result["StartTime"] > 0: time_start = time_from_nsecs(result["StartTime"]) else: time_start = None if "EndTime" in result and result["EndTime"] > 0: time_end = time_from_nsecs(result["EndTime"]) else: time_end = None if "DbId" in result: db_id = result["DbId"] else: db_id = "" if "Tags" in result: tags = result["Tags"] else: tags = "" ret = HTTPRequest( method=result["Method"], path=result["Path"], proto_major=result["ProtoMajor"], proto_minor=result["ProtoMinor"], headers=copy.deepcopy(result["Headers"]), body=base64.b64decode(result["Body"]), dest_host=result["DestHost"], dest_port=result["DestPort"], use_tls=result["UseTLS"], time_start=time_start, time_end=time_end, tags=tags, headers_only=headers_only, db_id=db_id, storage_id=storage) if "Unmangled" in result: ret.unmangled = decode_req(result["Unmangled"], headers_only=headers_only, storage=storage) if "Response" in result: ret.response = decode_rsp(result["Response"], headers_only=headers_only, storage=storage) if "WSMessages" in result: for wsm in result["WSMessages"]: ret.ws_messages.append(decode_ws(wsm, storage=storage)) return ret def decode_rsp(result, headers_only=False, storage=0): ret = HTTPResponse( status_code=result["StatusCode"], reason=result["Reason"], proto_major=result["ProtoMajor"], proto_minor=result["ProtoMinor"], headers=copy.deepcopy(result["Headers"]), body=base64.b64decode(result["Body"]), headers_only=headers_only, storage_id=storage, ) if "Unmangled" in result: ret.unmangled = decode_rsp(result["Unmangled"], headers_only=headers_only, storage=storage) return ret def decode_ws(result, storage=0): timestamp = None db_id = "" if "Timestamp" in result: timestamp = time_from_nsecs(result["Timestamp"]) if "DbId" in result: db_id = result["DbId"] ret = WSMessage( is_binary=result["IsBinary"], message=base64.b64decode(result["Message"]), to_server=result["ToServer"], timestamp=timestamp, db_id=db_id, storage_id=storage, ) if "Unmangled" in result: ret.unmangled = decode_ws(result["Unmangled"], storage=storage) return ret def encode_req(req, int_rsp=False): msg = { "DestHost": req.dest_host, "DestPort": req.dest_port, "UseTLS": req.use_tls, "Method": req.method, "Path": req.url.geturl(), "ProtoMajor": req.proto_major, "ProtoMinor": req.proto_major, "Headers": req.headers.dict(), "Tags": list(req.tags), "Body": base64.b64encode(copy.copy(req.body)).decode(), } if not int_rsp: msg["StartTime"] = time_to_nsecs(req.time_start) msg["EndTime"] = time_to_nsecs(req.time_end) if req.unmangled is not None: msg["Unmangled"] = encode_req(req.unmangled) if req.response is not None: msg["Response"] = encode_rsp(req.response) msg["WSMessages"] = [] for wsm in req.ws_messages: msg["WSMessages"].append(encode_ws(wsm)) return msg def encode_rsp(rsp, int_rsp=False): msg = { "ProtoMajor": rsp.proto_major, "ProtoMinor": rsp.proto_minor, "StatusCode": rsp.status_code, "Reason": rsp.reason, "Headers": rsp.headers.dict(), "Body": base64.b64encode(copy.copy(rsp.body)).decode(), } if not int_rsp: if rsp.unmangled is not None: msg["Unmangled"] = encode_rsp(rsp.unmangled) return msg def encode_ws(ws, int_rsp=False): msg = { "Message": base64.b64encode(ws.message).decode(), "IsBinary": ws.is_binary, "toServer": ws.to_server, } if not int_rsp: if ws.unmangled is not None: msg["Unmangled"] = encode_ws(ws.unmangled) msg["Timestamp"] = time_to_nsecs(ws.timestamp) msg["DbId"] = ws.db_id return msg def time_from_nsecs(nsecs): secs = nsecs / 1000000000 t = datetime.datetime.utcfromtimestamp(secs) return t def time_to_nsecs(t): if t is None: return None secs = (t - datetime.datetime(1970, 1, 1)).total_seconds() return int(math.floor(secs * 1000000000)) RequestStatusLine = namedtuple("RequestStatusLine", ["method", "path", "proto_major", "proto_minor"]) ResponseStatusLine = namedtuple("ResponseStatusLine", ["proto_major", "proto_minor", "status_code", "reason"]) def parse_req_sline(sline): if len(sline.split(b' ')) == 3: verb, path, version = sline.split(b' ') elif len(sline.split(b' ')) == 2: verb, version = sline.split(b' ') path = b'' else: raise Exception("malformed statusline") raw_version = version[5:] # strip HTTP/ pmajor, pminor = raw_version.split(b'.', 1) return RequestStatusLine(verb.decode(), path.decode(), int(pmajor), int(pminor)) def parse_rsp_sline(sline): if len(sline.split(b' ')) > 2: version, status_code, reason = sline.split(b' ', 2) else: version, status_code = sline.split(b' ', 1) reason = '' raw_version = version[5:] # strip HTTP/ pmajor, pminor = raw_version.split(b'.', 1) return ResponseStatusLine(int(pmajor), int(pminor), int(status_code), reason.decode()) def _parse_message(bs, sline_parser): header_env, body = re.split(br"(?:\r\n|\n)(?:\r\n|\n)", bs, 1) status_line, header_bytes = re.split(b"\r?\n", header_env, 1) h = Headers() for l in re.split(br"\r?\n", header_bytes): k, v = l.split(b": ", 1) if k.lower != 'content-length': h.add(k.decode(), v.decode()) h.add("Content-Length", str(len(body))) return (sline_parser(status_line), h, body) def parse_request(bs, dest_host='', dest_port=80, use_tls=False): req_sline, headers, body = _parse_message(bs, parse_req_sline) req = HTTPRequest( method=req_sline.method, path=req_sline.path, proto_major=req_sline.proto_major, proto_minor=req_sline.proto_minor, headers=headers.dict(), body=body, dest_host=dest_host, dest_port=dest_port, use_tls=use_tls) return req def parse_response(bs): rsp_sline, headers, body = _parse_message(bs, parse_rsp_sline) rsp = HTTPResponse( status_code=rsp_sline.status_code, reason=rsp_sline.reason, proto_major=rsp_sline.proto_major, proto_minor=rsp_sline.proto_minor, headers=headers.dict(), body=body) return rsp def get_full_url(req): netloc = req.dest_host if req.use_tls: scheme = "https" if req.dest_port != 443: netloc = "%s:%d" % (req.dest_host, req.dest_port) else: scheme = "http" if req.dest_port != 80: netloc = "%s:%d" % (req.dest_host, req.dest_port) rpath = req.url u = URL("") u.scheme = scheme u.netloc = netloc u.path = rpath.path u.params = rpath.params u.query = rpath.query u.fragment = rpath.fragment return u.geturl() ================================================ FILE: guppyproxy/repeater.py ================================================ from guppyproxy.util import display_error_box from guppyproxy.reqview import ReqViewWidget from PyQt5.QtWidgets import QWidget, QVBoxLayout, QHBoxLayout, QPushButton, QLineEdit, QCheckBox, QLabel, QSizePolicy, QToolButton from PyQt5.QtCore import pyqtSlot class RepeaterWidget(QWidget): def __init__(self, client): QWidget.__init__(self) self.client = client self.history = [] self.history_pos = 0 self.setLayout(QVBoxLayout()) self.layout().setSpacing(0) self.layout().setContentsMargins(0, 0, 0, 0) buttons = QHBoxLayout() buttons.setContentsMargins(0, 0, 0, 0) buttons.setSpacing(8) submitButton = QPushButton("Submit") submitButton.clicked.connect(self.submit) self.dest_host_input = QLineEdit() self.dest_port_input = QLineEdit() self.dest_port_input.setMaxLength(5) self.dest_port_input.setSizePolicy(QSizePolicy.Minimum, QSizePolicy.Preferred) self.dest_usetls_input = QCheckBox() self.back_button = QToolButton() self.back_button.setText("<") self.back_button.clicked.connect(self.back) self.forward_button = QToolButton() self.forward_button.setText(">") self.forward_button.clicked.connect(self.forward) buttons.addWidget(self.back_button) buttons.addWidget(self.forward_button) buttons.addWidget(submitButton) buttons.addWidget(QLabel("Host:")) buttons.addWidget(self.dest_host_input) buttons.addWidget(QLabel("Port:")) buttons.addWidget(self.dest_port_input) buttons.addWidget(QLabel("Use TLS:")) buttons.addWidget(self.dest_usetls_input) buttons.addStretch() self.reqview = ReqViewWidget(tag_tab=True) self.reqview.set_read_only(False) self.reqview.set_tags_read_only(False) self.layout().addLayout(buttons) self.layout().addWidget(self.reqview) self.req = None self.dest_host = "" self.dest_port = 80 self.use_tls = False self._update_buttons() def _set_host(self, host): self.dest_host_input.setText(host) def _set_port(self, port): if port is None or port <= 0: self.dest_port_input.setText("") else: self.dest_port_input.setText(str(port)) def _set_usetls(self, usetls): if usetls: self.dest_usetls_input.setCheckState(2) else: self.dest_usetls_input.setCheckState(0) def _set_dest_info(self, host, port, usetls): self._set_host(host) self._set_port(port) self._set_usetls(usetls) def _get_dest_info(self): host = self.dest_host_input.text() try: port = int(self.dest_port_input.text()) except: port = -1 if self.dest_usetls_input.checkState() == 0: usetls = False else: usetls = True return (host, port, usetls) def set_request(self, req, update_history=True): self._set_dest_info("", -1, False) if update_history: self.history.append(req) self.history_pos = len(self.history)-1 self._update_buttons() if req: self.req = req self.req.tags = set(["repeater"]) self._set_dest_info(req.dest_host, req.dest_port, req.use_tls) self.reqview.set_request(self.req) @pyqtSlot(set) def update_req_tags(self, tags): if self.req: self.req.tags = tags @pyqtSlot() def submit(self): try: req = self.reqview.get_request() if not req: display_error_box("Could not parse request") return except: display_error_box("Could not parse request") return req.tags.add("repeater") host, port, usetls = self._get_dest_info() if port is None: display_error_box("Invalid port") return req.dest_host = host req.dest_port = port req.dest_usetls = usetls try: self.client.submit(req, save=True) self.req = req self.set_request(req) except Exception as e: errmsg = "Error submitting request:\n%s" % str(e) display_error_box(errmsg) return @pyqtSlot() def back(self): if self.history_pos > 0: self.history_pos -= 1 self.set_request(self.history[self.history_pos], update_history=False) self._update_buttons() @pyqtSlot() def forward(self): if self.history_pos < len(self.history)-1: self.history_pos += 1 self.set_request(self.history[self.history_pos], update_history=False) self._update_buttons() def _update_buttons(self): self.forward_button.setEnabled(True) self.back_button.setEnabled(True) if len(self.history) == 0 or self.history_pos == len(self.history)-1: self.forward_button.setEnabled(False) if self.history_pos == 0: self.back_button.setEnabled(False) ================================================ FILE: guppyproxy/reqlist.py ================================================ import threading import shlex from guppyproxy.util import max_len_str, query_to_str, display_error_box, display_info_box, display_req_context, display_multi_req_context, hostport, method_color, sc_color, DisableUpdates, host_color from guppyproxy.proxy import HTTPRequest, RequestContext, InvalidQuery, SocketClosed, time_to_nsecs, ProxyThread from guppyproxy.reqview import ReqViewWidget from guppyproxy.reqtree import ReqTreeView from PyQt5.QtWidgets import QWidget, QTableWidget, QTableWidgetItem, QGridLayout, QHeaderView, QAbstractItemView, QVBoxLayout, QHBoxLayout, QComboBox, QTabWidget, QPushButton, QLineEdit, QStackedLayout, QToolButton, QCheckBox, QLabel, QTableView, QMenu from PyQt5.QtCore import pyqtSlot, pyqtSignal, QObject, QVariant, Qt, QAbstractTableModel, QModelIndex, QItemSelection, QSortFilterProxyModel from itertools import groupby, count def get_field_entry(): dropdown = QComboBox() dropdown.addItem("Anywhere", "all") dropdown.addItem("Req. Body", "reqbody") dropdown.addItem("Rsp. Body", "rspbody") dropdown.addItem("Any Body", "body") # dropdown.addItem("WSMessage", "wsmessage") dropdown.addItem("Req. Header", "reqheader") dropdown.addItem("Rsp. Header", "rspheader") dropdown.addItem("Any Header", "header") dropdown.addItem("Method", "method") dropdown.addItem("Host", "host") dropdown.addItem("Path", "path") dropdown.addItem("URL", "url") dropdown.addItem("Status", "statuscode") dropdown.addItem("Tag", "tag") dropdown.addItem("Any Param", "param") dropdown.addItem("URL Param", "urlparam") dropdown.addItem("Post Param", "postparam") dropdown.addItem("Rsp. Cookie", "rspcookie") dropdown.addItem("Req. Cookie", "reqcookie") dropdown.addItem("Any Cookie", "cookie") # dropdown.addItem("After", "") # dropdown.addItem("Before", "") # dropdown.addItem("TimeRange", "") # dropdown.addItem("Id", "") return dropdown def get_string_cmp_entry(): dropdown = QComboBox() dropdown.addItem("cnt.", "contains") dropdown.addItem("cnt. (rgx)", "containsregexp") dropdown.addItem("is", "is") dropdown.addItem("len. >", "lengt") dropdown.addItem("len. <", "lenlt") dropdown.addItem("len. =", "leneq") return dropdown class StringCmpWidget(QWidget): returnPressed = pyqtSignal() def __init__(self, *args, **kwargs): QWidget.__init__(self, *args, **kwargs) layout = QHBoxLayout() self.cmp_entry = get_string_cmp_entry() self.text_entry = QLineEdit() self.text_entry.returnPressed.connect(self.returnPressed) layout.addWidget(self.cmp_entry) layout.addWidget(self.text_entry) self.setLayout(layout) self.layout().setContentsMargins(0, 0, 0, 0) def get_value(self): str_cmp = self.cmp_entry.itemData(self.cmp_entry.currentIndex()) str_val = self.text_entry.text() return [str_cmp, str_val] def reset(self): self.cmp_entry.setCurrentIndex(0) self.text_entry.setText("") def dt_sort_key(r): if r.time_start: return time_to_nsecs(r.time_start) return 0 class StringKVWidget(QWidget): returnPressed = pyqtSignal() def __init__(self, *args, **kwargs): QWidget.__init__(self, *args, **kwargs) self.str2_shown = False self.str1 = StringCmpWidget() self.str2 = StringCmpWidget() self.str1.returnPressed.connect(self.returnPressed) self.str2.returnPressed.connect(self.returnPressed) self.toggle_button = QToolButton() self.toggle_button.setText("+") self.toggle_button.clicked.connect(self._show_hide_str2) layout = QHBoxLayout() layout.addWidget(self.str1) layout.addWidget(self.str2) layout.addWidget(self.toggle_button) self.str2.setVisible(self.str2_shown) self.setLayout(layout) self.layout().setContentsMargins(0, 0, 0, 0) @pyqtSlot() def _show_hide_str2(self): if self.str2_shown: self.toggle_button.setText("+") self.str2_shown = False else: self.toggle_button.setText("-") self.str2_shown = True self.str2.setVisible(self.str2_shown) def get_value(self): retval = self.str1.get_value() if self.str2_shown: retval += self.str2.get_value() return retval def reset(self): self.str1.reset() self.str2.reset() class DropdownFilterEntry(QWidget): # a widget that lets you enter filters using ezpz dropdowns/text boxes filterEntered = pyqtSignal(list) def __init__(self, *args, **kwargs): QWidget.__init__(self, *args, **kwargs) layout = QHBoxLayout() confirm = QToolButton() confirm.setText("OK") confirm.setToolTip("Apply the entered filter") self.field_entry = get_field_entry() # stack containing widgets for string, k/v, date, daterange self.str_cmp_entry = StringCmpWidget() self.kv_cmp_entry = StringKVWidget() self.inv_entry = QCheckBox("inv") # date # daterange self.entry_layout = QStackedLayout() self.entry_layout.setContentsMargins(0, 0, 0, 0) self.current_entry = 0 self.entry_layout.addWidget(self.str_cmp_entry) self.entry_layout.addWidget(self.kv_cmp_entry) # add date # 2 # add daterange # 3 confirm.clicked.connect(self.confirm_entry) self.str_cmp_entry.returnPressed.connect(self.confirm_entry) self.kv_cmp_entry.returnPressed.connect(self.confirm_entry) self.field_entry.currentIndexChanged.connect(self._display_value_widget) layout.addWidget(confirm) layout.addWidget(self.inv_entry) layout.addWidget(self.field_entry) layout.addLayout(self.entry_layout) self.setLayout(layout) self.setContentsMargins(0, 0, 0, 0) self._display_value_widget() @pyqtSlot() def _display_value_widget(self): # show the correct value widget in the value stack layout field = self.field_entry.itemData(self.field_entry.currentIndex()) self.current_entry = 0 if field in ("all", "reqbody", "rspbody", "body", "wsmessage", "method", "host", "path", "url", "statuscode", "tag"): self.current_entry = 0 elif field in ("reqheader", "rspheader", "header", "param", "urlparam" "postparam", "rspcookie", "reqcookie", "cookie"): self.current_entry = 1 # elif for date # elif for daterange self.entry_layout.setCurrentIndex(self.current_entry) def get_value(self): val = [] if self.inv_entry.isChecked(): val.append("inv") field = self.field_entry.itemData(self.field_entry.currentIndex()) val.append(field) if self.current_entry == 0: val += self.str_cmp_entry.get_value() elif self.current_entry == 1: val += self.kv_cmp_entry.get_value() # elif for date # elif for daterange return [val] # no support for OR @pyqtSlot() def confirm_entry(self): phrases = self.get_value() self.filterEntered.emit(phrases) self.str_cmp_entry.reset() self.kv_cmp_entry.reset() # reset date # reset date range class TextFilterEntry(QWidget): # a text box that can be used to enter filters filterEntered = pyqtSignal(list) def __init__(self, *args, **kwargs): QWidget.__init__(self, *args, **kwargs) layout = QHBoxLayout() self.textEntry = QLineEdit() self.textEntry.returnPressed.connect(self.confirm_entry) self.textEntry.setToolTip("Enter the filter here and press return to apply it") layout.addWidget(self.textEntry) self.setLayout(layout) self.layout().setContentsMargins(0, 0, 0, 0) @pyqtSlot() def confirm_entry(self): args = shlex.split(self.textEntry.text()) phrases = [list(group) for k, group in groupby(args, lambda x: x == "OR") if not k] self.filterEntered.emit(phrases) self.textEntry.setText("") class FilterEntry(QWidget): # a widget that lets you switch between filter entries filterEntered = pyqtSignal(list) def __init__(self, *args, **kwargs): QWidget.__init__(self, *args, **kwargs) self.current_entry = 0 self.max_entries = 2 self.text_entry = TextFilterEntry() dropdown_entry = DropdownFilterEntry() self.text_entry.filterEntered.connect(self.filterEntered) dropdown_entry.filterEntered.connect(self.filterEntered) self.entry_layout = QStackedLayout() self.entry_layout.addWidget(dropdown_entry) self.entry_layout.addWidget(self.text_entry) swap_button = QToolButton() swap_button.setText(">") swap_button.setToolTip("Switch between dropdown and text entry") swap_button.clicked.connect(self.next_entry) hlayout = QHBoxLayout() hlayout.addWidget(swap_button) hlayout.addLayout(self.entry_layout) self.setLayout(hlayout) self.layout().setContentsMargins(0, 0, 0, 0) self.layout().setSpacing(0) @pyqtSlot() def next_entry(self): self.current_entry += 1 self.current_entry = self.current_entry % self.max_entries self.entry_layout.setCurrentIndex(self.current_entry) def set_entry(self, entry): self.current_entry = entry self.current_entry = self.current_entry % self.max_entries self.entry_layout.setCurrentIndex(self.current_entry) class FilterListWidget(QTableWidget): # list part of the filter tab def __init__(self, *args, **kwargs): self.client = kwargs.pop("client") QTableWidget.__init__(self, *args, **kwargs) self.context = RequestContext(self.client) # Set up table self.setColumnCount(1) self.horizontalHeader().hide() self.horizontalHeader().setSectionResizeMode(QHeaderView.Stretch) self.verticalHeader().hide() self.verticalHeader().setSectionResizeMode(QHeaderView.ResizeToContents) #self.setSelectionMode(QAbstractItemView.NoSelection) #self.setEditTriggers(QAbstractItemView.NoEditTriggers) def append_fstr(self, fstr): args = shlex.split(fstr) phrase = [list(group) for k, group in groupby(args, lambda x: x == "OR") if not k] self.context.apply_phrase(phrase) self._append_fstr_row(fstr) def set_query(self, query): self.context.set_query(query) self.redraw_table() def pop_phrase(self): self.context.pop_phrase() self.redraw_table() def clear_phrases(self): self.context.set_query([]) self.redraw_table() def _append_fstr_row(self, fstr): row = self.rowCount() self.insertRow(row) self.setItem(row, 0, QTableWidgetItem(fstr)) def redraw_table(self): self.setRowCount(0) query = self.context.query for p in query: condstrs = [' '.join(l) for l in p] fstr = ' OR '.join(condstrs) self._append_fstr_row(fstr) def get_query(self): return self.context.query class FilterEditor(QWidget): # a widget containing a list of filters and the ability to edit the filters in the list filtersEdited = pyqtSignal(list) builtin_filters = ( ('No Images', ['inv', 'path', 'containsregexp', r'(\.png$|\.jpg$|\.jpeg$|\.gif$|\.ico$|\.bmp$|\.svg$)']), ('No JavaScript/CSS/Fonts', ['inv', 'path', 'containsregexp', r'(\.js$|\.css$|\.woff$)']), ) def __init__(self, *args, **kwargs): self.client = kwargs.pop("client") QWidget.__init__(self, *args, **kwargs) layout = QVBoxLayout() # Manage bar manage_bar = QHBoxLayout() pop_button = QPushButton("Pop") pop_button.setToolTip("Remove the most recently applied filter") clear_button = QPushButton("Clear") clear_button.setToolTip("Remove all active filters") scope_reset_button = QPushButton("Scope") scope_reset_button.setToolTip("Set the active filters to the current scope") scope_save_button = QPushButton("Save Scope") scope_save_button.setToolTip("Set the scope to the current filters. Any messages that don't match the active filters will be ignored by the proxy.") self.builtin_combo = QComboBox() self.builtin_combo.addItem("Apply a built-in filter", None) for desc, filt in FilterEditor.builtin_filters: self.builtin_combo.addItem(desc, filt) self.builtin_combo.currentIndexChanged.connect(self._apply_builtin_filter) manage_bar.addWidget(clear_button) manage_bar.addWidget(pop_button) manage_bar.addWidget(scope_reset_button) manage_bar.addWidget(scope_save_button) manage_bar.addWidget(self.builtin_combo) manage_bar.addStretch() mbar_widget = QWidget() mbar_widget.setLayout(manage_bar) pop_button.clicked.connect(self.pop_phrase) clear_button.clicked.connect(self.clear_phrases) scope_reset_button.clicked.connect(self.reset_to_scope) scope_save_button.clicked.connect(self.save_scope) # Filter list self.filter_list = FilterListWidget(client=self.client) # Filter entry self.entry = FilterEntry() self.entry.setMaximumHeight(self.entry.sizeHint().height()) self.entry.filterEntered.connect(self.apply_phrase) layout.addWidget(mbar_widget) layout.addWidget(self.filter_list) layout.addWidget(self.entry) self.setLayout(layout) self.layout().setSpacing(0) self.layout().setContentsMargins(0, 0, 0, 0) @pyqtSlot() def save_scope(self): query = self.filter_list.get_query() self.client.set_scope(query) display_info_box("Scope updated") @pyqtSlot() def reset_to_scope(self): query = self.client.get_scope().filter self.filter_list.set_query(query) self.filtersEdited.emit(self.filter_list.get_query()) @pyqtSlot() def clear_phrases(self): self.filter_list.clear_phrases() self.filtersEdited.emit(self.filter_list.get_query()) @pyqtSlot() def pop_phrase(self): self.filter_list.pop_phrase() self.filtersEdited.emit(self.filter_list.get_query()) @pyqtSlot(list) def apply_phrase(self, phrase): fstr = query_to_str([phrase]) try: self.filter_list.append_fstr(fstr) except InvalidQuery as e: display_error_box("Could not add filter:\n\n%s" % e) return self.filtersEdited.emit(self.filter_list.get_query()) @pyqtSlot(int) def _apply_builtin_filter(self, ind): phrase = self.builtin_combo.itemData(ind) if phrase: self.apply_phrase([phrase]) self.builtin_combo.setCurrentIndex(0) def set_is_text(self, is_text): if is_text: self.entry.set_entry(1) else: self.entry.set_entry(0) class ReqListModel(QAbstractTableModel): requestsLoading = pyqtSignal() requestsLoaded = pyqtSignal() HD_ID = 0 HD_VERB = 1 HD_HOST = 2 HD_PATH = 3 HD_SCODE = 4 HD_REQLEN = 5 HD_RSPLEN = 6 HD_TIME = 7 HD_TAGS = 8 HD_MNGL = 9 def __init__(self, client, *args, **kwargs): QAbstractTableModel.__init__(self, *args, **kwargs) self.client = client self.header_order = [ self.HD_ID, self.HD_VERB, self.HD_HOST, self.HD_PATH, self.HD_SCODE, self.HD_REQLEN, self.HD_RSPLEN, self.HD_TIME, self.HD_TAGS, self.HD_MNGL, ] self.table_headers = { self.HD_ID: "ID", self.HD_VERB: "Method", self.HD_HOST: "Host", self.HD_PATH: "Path", self.HD_SCODE: "S-Code", self.HD_REQLEN: "Req Len", self.HD_RSPLEN: "Rsp Len", self.HD_TIME: "Time", self.HD_TAGS: "Tags", self.HD_MNGL: "Mngl", } self.reqs = [] self.sort_enabled = False self.header_count = len(self.header_order) self.reqs_loaded = 0 def headerData(self, section, orientation, role): if role == Qt.DisplayRole and orientation == Qt.Horizontal: hd = self.header_order[section] return self.table_headers[hd] return QVariant() def rowCount(self, parent): return self.reqs_loaded def columnCount(self, parent): return self.header_count def _gen_req_row(self, req): MAX_PATH_LEN = 60 MAX_TAG_LEN = 40 reqid = self.client.get_reqid(req) method = req.method host = hostport(req) path = max_len_str(req.url.path, MAX_PATH_LEN) reqlen = str(req.content_length) tags = max_len_str(', '.join(sorted(req.tags)), MAX_TAG_LEN) if req.response: scode = str(req.response.status_code) + ' ' + req.response.reason rsplen = str(req.response.content_length) else: scode = "--" rsplen = "--" if req.time_start and req.time_end: time_delt = req.time_end - req.time_start reqtime = ("%.2f" % time_delt.total_seconds()) else: reqtime = "--" if req.unmangled and req.response and req.response.unmangled: manglestr = "q/s" elif req.unmangled: manglestr = "q" elif req.response and req.response.unmangled: manglestr = "s" else: manglestr = "N/A" return (req, reqid, method, host, path, scode, reqlen, rsplen, reqtime, tags, manglestr) def data(self, index, role): if role == Qt.BackgroundColorRole: req = self.reqs[index.row()][0] if index.column() == 2: return host_color(hostport(req)) elif index.column() == 4: if req.response: return sc_color(str(req.response.status_code)) elif index.column() == 1: return method_color(req.method) return QVariant() elif role == Qt.DisplayRole: rowdata = self.reqs[index.row()] return rowdata[index.column()+1] return QVariant() def canFetchMore(self, parent): if parent.isValid(): return False return (self.reqs_loaded < len(self.reqs)) def fetchMore(self, parent): if parent.isValid(): return if self.reqs_loaded == len(self.reqs): return n_to_fetch = 50 if self.reqs_loaded + n_to_fetch > len(self.reqs): n_to_fetch = len(self.reqs) - self.reqs_loaded self.beginInsertRows(QModelIndex(), self.reqs_loaded, self.reqs_loaded + n_to_fetch) self.reqs_loaded += n_to_fetch self.endInsertRows() def _sort_reqs(self): def skey(rowdata): return dt_sort_key(rowdata[0]) if self.sort_enabled: self.reqs = sorted(self.reqs, key=skey, reverse=True) def _req_ind(self, req=None, reqid=None): if not reqid: reqid = self.client.get_reqid(req) for ind, rowdata in zip(count(), self.reqs): req = rowdata[0] if self.client.get_reqid(req) == reqid: return ind return -1 def _emit_all_data(self): self.dataChanged.emit(self.createIndex(0, 0), self.createIndex(self.rowCount(None), self.columnCount(None))) def _set_requests(self, reqs): self.reqs = [self._gen_req_row(req) for req in reqs] self.reqs_loaded = 0 def set_requests(self, reqs): self.beginResetModel() self._set_requests(reqs) self._sort_reqs() self._emit_all_data() self.endResetModel() def clear(self): self.beginResetModel() self.reqs = [] self.reqs_loaded = 0 self._emit_all_data() self.endResetModel() def add_request_head(self, req): self.beginInsertRows(QModelIndex(), 0, 0) self.reqs = [self._gen_req_row(req)] + self.reqs self.reqs_loaded += 1 self.endInsertRows() def add_request(self, req): self.beginResetModel() self.reqs.append(self._gen_req_row(req)) self.reqs_loaded = 0 self._sort_reqs() self._emit_all_data() self.endResetModel() def add_requests(self, reqs): self.beginResetModel() for req in reqs: self.reqs.append(self._gen_req_row(req)) self.reqs_loaded = 0 self._sort_reqs() self._emit_all_data() self.endResetModel() def update_request(self, req): ind = self._req_ind(req) if ind < 0: return self.reqs[ind] = self._gen_req_row(req) self.dataChanged.emit(self.createIndex(ind, 0), self.createIndex(ind, self.rowCount(None))) def delete_request(self, req=None, reqid=None): ind = self._req_ind(req, reqid) if ind < 0: return self.beginRemoveRows(QModelIndex(), ind, ind) self.reqs_loaded -= 1 self.reqs = self.reqs[:ind] + self.reqs[(ind+1):] self.endRemoveRows() def has_request(self, req=None, reqid=None): if self._req_ind(req, reqid) < 0: return False return True def get_requests(self): return [row[0] for row in self.reqs] def disable_sort(self): self.sort_enabled = False def enable_sort(self): self.sort_enabled = True self._sort_reqs() def req_by_ind(self, ind): return self.reqs[ind][0] class ReqBrowser(QWidget): # Widget containing request viewer, tabs to view list of reqs, filters, and (evevntually) site map # automatically updated with requests as they're saved def __init__(self, client, repeater_widget=None, macro_widget=None, reload_reqs=True, update=False, filter_tab=True, is_client_context=False): QWidget.__init__(self) self.client = client self.filters = [] self.reload_reqs = reload_reqs self.mylayout = QGridLayout() self.mylayout.setSpacing(0) self.mylayout.setContentsMargins(0, 0, 0, 0) # reqtable updater if update: self.updater = ReqListUpdater(self.client) else: self.updater = None # reqtable/search self.listWidg = ReqTableWidget(client, repeater_widget=repeater_widget, macro_widget=macro_widget) if self.updater: self.updater.add_reqlist_widget(self.listWidg) self.listWidg.requestsSelected.connect(self.update_viewer) self.listLayout = QVBoxLayout() self.listLayout.setContentsMargins(0, 0, 0, 0) self.listLayout.setSpacing(0) self.listButtonLayout = QHBoxLayout() self.listButtonLayout.setContentsMargins(0, 0, 0, 0) clearSelectionBut = QPushButton("Clear Selection") clearSelectionBut.clicked.connect(self.listWidg.clear_selection) self.listButtonLayout.addWidget(clearSelectionBut) self.listButtonLayout.addStretch() self.listLayout.addWidget(self.listWidg) self.listLayout.addLayout(self.listButtonLayout) # Filter widget self.filterWidg = FilterEditor(client=self.client) self.filterWidg.filtersEdited.connect(self.listWidg.set_filter) if is_client_context: self.filterWidg.filtersEdited.connect(self.set_client_context) self.filterWidg.reset_to_scope() # Tree widget self.treeWidg = ReqTreeView() # add tabs self.listTabs = QTabWidget() lwidg = QWidget() lwidg.setLayout(self.listLayout) self.listTabs.addTab(lwidg, "List") self.tree_ind = self.listTabs.count() self.listTabs.addTab(self.treeWidg, "Tree") if filter_tab: self.listTabs.addTab(self.filterWidg, "Filters") self.listTabs.currentChanged.connect(self._tab_changed) # reqview self.reqview = ReqViewWidget(info_tab=True, param_tab=True, tag_tab=True) self.reqview.set_tags_read_only(False) self.reqview.tag_widg.tagsUpdated.connect(self._tags_updated) self.listWidg.req_view_widget = self.reqview self.mylayout.addWidget(self.reqview, 0, 0, 3, 1) self.mylayout.addWidget(self.listTabs, 4, 0, 2, 1) self.setLayout(self.mylayout) def show_filters(self): self.listTabs.setCurrentIndex(2) def show_history(self): self.listTabs.setCurrentIndex(0) def show_tree(self): self.listTabs.setCurrentIndex(1) @pyqtSlot(list) def set_client_context(self, query): self.client.context.set_query(query) @pyqtSlot() def reset_to_scope(self): self.filterWidg.reset_to_scope() @pyqtSlot(list) def update_viewer(self, reqs): self.reqview.set_request(None) if len(reqs) > 0: if self.reload_reqs: reqh = reqs[0] req = self.client.req_by_id(reqh.db_id) else: req = reqs[0] self.reqview.set_request(req) @pyqtSlot(list) def update_filters(self, query): self.filters = query @pyqtSlot(HTTPRequest) def add_request_item(self, req): self.listWidg.add_request_item(req) self.treeWidg.add_request_item(req) @pyqtSlot(list) def set_requests(self, reqs): self.listWidg.set_requests(reqs) self.treeWidg.set_requests(reqs) @pyqtSlot(int) def _tab_changed(self, i): if i == self.tree_ind: self.treeWidg.set_requests(self.listWidg.get_requests()) @pyqtSlot(set) def _tags_updated(self, tags): req = self.reqview.req req.tags = tags if req.db_id: reqid = self.client.get_reqid(req) self.client.clear_tag(reqid) for tag in tags: self.client.add_tag(reqid, tag) def set_filter_is_text(self, is_text): self.filterWidg.set_is_text(is_text) class ReqListUpdater(QObject): newRequest = pyqtSignal(HTTPRequest) requestUpdated = pyqtSignal(HTTPRequest) requestDeleted = pyqtSignal(str) def __init__(self, client): QObject.__init__(self) self.mtx = threading.Lock() self.client = client self.reqlist_widgets = [] self.t = ProxyThread(target=self.run_updater) self.t.start() def add_reqlist_widget(self, widget): self.mtx.acquire() try: self.newRequest.connect(widget.add_request) self.requestUpdated.connect(widget.update_request) self.requestDeleted.connect(widget.delete_request) self.reqlist_widgets.append(widget) finally: self.mtx.release() def run_updater(self): conn = self.client.new_conn() try: try: for msg in conn.watch_storage(): self.mtx.acquire() try: if msg["Action"] == "NewRequest": self.newRequest.emit(msg["Request"]) elif msg["Action"] == "RequestUpdated": self.requestUpdated.emit(msg["Request"]) elif msg["Action"] == "RequestDeleted": self.requestDeleted.emit(msg["MessageId"]) finally: self.mtx.release() except SocketClosed: return finally: conn.close() def stop(self): self.conn.close() class ReqTableWidget(QWidget): requestsChanged = pyqtSignal(list) requestsSelected = pyqtSignal(list) def __init__(self, client, repeater_widget=None, macro_widget=None, *args, **kwargs): QWidget.__init__(self, *args, **kwargs) self.allow_save = False self.client = client self.repeater_widget = repeater_widget self.macro_widget = macro_widget self.query = [] self.req_view_widget = None self.setLayout(QStackedLayout()) self.layout().setContentsMargins(0, 0, 0, 0) self.tableModel = ReqListModel(self.client) self.tableView = QTableView() self.tableView.setModel(self.tableModel) self.tableView.verticalHeader().setSectionResizeMode(QHeaderView.ResizeToContents) self.tableView.horizontalHeader().setSectionResizeMode(QHeaderView.ResizeToContents) self.tableView.verticalHeader().hide() self.tableView.setSelectionBehavior(QAbstractItemView.SelectRows) #self.tableView.setSelectionMode(QAbstractItemView.SingleSelection) self.tableView.horizontalHeader().setStretchLastSection(True) self.tableView.selectionModel().selectionChanged.connect(self.on_select_change) self.tableModel.dataChanged.connect(self._paint_view) self.tableModel.rowsInserted.connect(self._on_rows_inserted) self.requestsChanged.connect(self.set_requests) self.requestsSelected.connect(self._updated_selected_request) self.selected_reqs = [] self.layout().addWidget(self.tableView) self.layout().addWidget(QLabel("Loading requests from data file...")) @pyqtSlot(HTTPRequest) def add_request(self, req): with DisableUpdates(self.tableView): if req.db_id != "": reqid = self.client.get_reqid(req) if self.client.check_request(self.query, reqid=reqid): self.tableModel.add_request_head(req) if req.unmangled and req.unmangled.db_id != "" and self.tableModel.has_request(req.unmangled): self.tableModel.delete_request(req.unmangled) else: if self.client.check_request(self.query, req=req): self.tableModel.add_request_head(req) @pyqtSlot() def clear(self): self.tableModel.clear() def get_requests(self): return self.tableModel.get_requests() @pyqtSlot(list) def set_requests(self, reqs, check_filter=False): to_add = [] if not check_filter: to_add = reqs else: for req in reqs: if req.db_id != "": reqid = self.client.get_reqid(req) if self.client.check_request(self.query, reqid=reqid): to_add.append(req) else: if self.client.check_request(self.query, req=req): to_add.append(req) with DisableUpdates(self.tableView): self.clear() self.tableModel.disable_sort() self.tableModel.add_requests(to_add) self.tableModel.enable_sort() self.set_is_not_loading() @pyqtSlot(HTTPRequest) def update_request(self, req): with DisableUpdates(self.tableView): self.tableModel.update_request(req) if req.db_id != "": if req.unmangled and req.unmangled.db_id != "": self.tableModel.delete_request(reqid=self.client.get_reqid(req.unmangled)) @pyqtSlot(str) def delete_request(self, reqid): with DisableUpdates(self.tableView): self.tableModel.delete_request(reqid=reqid) @pyqtSlot(list) def set_filter(self, query): self.query = query self.set_is_loading() self.client.query_storage_async(self.requestsChanged, self.query, headers_only=True) @pyqtSlot(list) def _updated_selected_request(self, reqs): if len(reqs) > 0: self.selected_reqs = reqs else: self.selected_reqs = [] @pyqtSlot(QModelIndex, int, int) def _on_rows_inserted(self, parent, first, last): rows = self.tableView.selectionModel().selectedRows() if len(rows) > 0: row = rows[0].row() idx = self.tableModel.index(row, 0, QModelIndex()) self.tableView.scrollTo(idx) @pyqtSlot(QItemSelection, QItemSelection) def on_select_change(self, newSelection, oldSelection): reqs = [] added = set() for rowidx in self.tableView.selectionModel().selectedRows(): row = rowidx.row() if row not in added: reqs.append(self.tableModel.req_by_ind(row)) added.add(row) self.requestsSelected.emit(reqs) @pyqtSlot() def clear_selection(self): self.tableView.clearSelection() def get_selected_request(self): # load the full request if len(self.selected_reqs) > 0: return self.client.load_by_reqheaders(self.selected_reqs[0]) else: return None def get_selected_requests(self): ret = [] for hreq in self.selected_reqs: ret.append(self.client.load_by_reqheaders(hreq)) return ret def get_all_requests(self): return [self.client.req_by_id(self.client.get_reqid(req)) for req in self.tableModel.get_requests()] def contextMenuEvent(self, event): if len(self.selected_reqs) > 1: reqs = self.get_selected_requests() display_multi_req_context(self, self.client, reqs, event, macro_widget=self.macro_widget, save_option=self.allow_save) elif len(self.selected_reqs) == 1: req = self.get_selected_request() display_req_context(self, self.client, req, event, repeater_widget=self.repeater_widget, req_view_widget=self.req_view_widget, macro_widget=self.macro_widget, save_option=self.allow_save) def set_is_loading(self): self.set_loading(True) def set_is_not_loading(self): self.set_loading(False) def set_loading(self, is_loading): with DisableUpdates(self.tableView): if is_loading: self.layout().setCurrentIndex(1) else: self.layout().setCurrentIndex(0) @pyqtSlot(QModelIndex, QModelIndex) def _paint_view(self, indA, indB): self.tableView.repaint() @pyqtSlot() def delete_selected(self): with DisableUpdates(self.tableView): for req in self.selected_reqs: self.tableModel.delete_request(req=req) ================================================ FILE: guppyproxy/reqtree.py ================================================ from guppyproxy.proxy import HTTPRequest from PyQt5.QtWidgets import QWidget, QTreeView, QVBoxLayout from PyQt5.QtGui import QStandardItem, QStandardItemModel from PyQt5.QtCore import pyqtSlot, Qt def _include_req(req): if not req.response: return False if req.response.status_code == 404: return False return True class PathNodeItem(QStandardItem): def __init__(self, text, *args, **kwargs): QStandardItem.__init__(self, *args, **kwargs) self.text = text self.children = {} def add_child(self, text): if text not in self.children: newitem = PathNodeItem(text, text) newitem.setFlags(newitem.flags() ^ Qt.ItemIsEditable) self.children[text] = newitem self.appendRow(newitem) def get_child(self, text): return self.children[text] def add_child_path(self, texts): if not texts: return childtext = texts[0] self.add_child(childtext) child = self.get_child(childtext) child.add_child_path(texts[1:]) class ReqTreeView(QWidget): def __init__(self): QWidget.__init__(self) self.setLayout(QVBoxLayout()) self.layout().setSpacing(0) self.layout().setContentsMargins(0, 0, 0, 0) self.nodes = {} self.tree_view = QTreeView() self.tree_view.header().close() self.root = QStandardItemModel() self.tree_view.setModel(self.root) self.layout().addWidget(self.tree_view) @pyqtSlot(HTTPRequest) def add_request_item(self, req): path_parts = req.url.geturl(False).split("/") path_parts = path_parts[1:] path_parts = ["/" + p for p in path_parts] path_parts = [req.dest_host] + path_parts if path_parts[0] not in self.nodes: item = PathNodeItem(path_parts[0], path_parts[0]) item.setFlags(item.flags() ^ Qt.ItemIsEditable) self.nodes[path_parts[0]] = item self.root.appendRow(item) else: item = self.nodes[path_parts[0]] item.add_child_path(path_parts[1:]) @pyqtSlot(list) def set_requests(self, reqs): self.clear() for req in reqs: if _include_req(req): self.add_request_item(req) self.tree_view.expandAll() def clear(self): self.nodes = {} self.root = QStandardItemModel() self.tree_view.setModel(self.root) ================================================ FILE: guppyproxy/reqview.py ================================================ import re from guppyproxy.util import datetime_string, DisableUpdates from guppyproxy.proxy import HTTPRequest, get_full_url, parse_request from guppyproxy.hexteditor import ComboEditor from PyQt5.QtWidgets import QWidget, QTableWidget, QTableWidgetItem, QGridLayout, QHeaderView, QAbstractItemView, QLineEdit, QTabWidget, QVBoxLayout, QToolButton, QHBoxLayout, QStackedLayout from PyQt5.QtCore import pyqtSlot, pyqtSignal, Qt from pygments.lexer import Lexer from pygments.lexers import get_lexer_for_mimetype, TextLexer from pygments.lexers.textfmts import HttpLexer from pygments.util import ClassNotFound from pygments.token import Token class HybridHttpLexer(Lexer): tl = TextLexer() hl = HttpLexer() def __init__(self, max_len=50000, *args, **kwargs): self.max_len = max_len Lexer.__init__(self, *args, **kwargs) def get_tokens_unprocessed(self, text): try: split = re.split(r"(?:\r\n|\n)(?:\r\n|\n)", text, 1) if len(split) == 2: h = split[0] body = split[1] else: h = split[0] body = '' except Exception as e: for v in self.tl.get_tokens_unprocessed(text): yield v raise e for token in self.hl.get_tokens_unprocessed(h): yield token if len(body) > 0: if len(body) <= self.max_len or self.max_len < 0: second_parser = None if "Content-Type" in h: try: ct = re.search("Content-Type: (.*)", h) if ct is not None: hval = ct.groups()[0] mime = hval.split(";")[0] second_parser = get_lexer_for_mimetype(mime) except ClassNotFound: pass if second_parser is None: yield (len(h), Token.Text, text[len(h):]) else: for index, tokentype, value in second_parser.get_tokens_unprocessed(text[len(h):]): yield (index + len(h), tokentype, value) else: yield (len(h), Token.Text, text[len(h):]) class InfoWidget(QWidget): def __init__(self, *args, **kwargs): QWidget.__init__(self, *args, **kwargs) self.request = None self.setLayout(QVBoxLayout()) self.layout().setSpacing(0) self.layout().setContentsMargins(0, 0, 0, 0) self.infotable = QTableWidget() self.infotable.setColumnCount(2) self.infotable.verticalHeader().hide() self.infotable.horizontalHeader().hide() self.infotable.horizontalHeader().setSectionResizeMode(QHeaderView.ResizeToContents) self.infotable.verticalHeader().setSectionResizeMode(QHeaderView.ResizeToContents) self.infotable.horizontalHeader().setStretchLastSection(True) self.layout().addWidget(self.infotable) def _add_info(self, k, v): row = self.infotable.rowCount() self.infotable.insertRow(row) item1 = QTableWidgetItem(k) item1.setFlags(item1.flags() ^ Qt.ItemIsEditable) self.infotable.setItem(row, 0, item1) self.infotable.setItem(row, 1, QTableWidgetItem(v)) def set_request(self, req): with DisableUpdates(self.infotable): self.request = req self.infotable.setRowCount(0) if self.request is None: return reqlen = len(self.request.body) reqlen = '%d bytes' % reqlen rsplen = 'No response' mangle_str = 'Nothing mangled' if self.request.unmangled: mangle_str = 'Request' if self.request.response: response_code = str(self.request.response.status_code) + \ ' ' + self.request.response.reason rsplen = self.request.response.content_length rsplen = '%d bytes' % rsplen if self.request.response.unmangled: if mangle_str == 'Nothing mangled': mangle_str = 'Response' else: mangle_str += ' and Response' else: response_code = '' time_str = '--' if self.request.time_end is not None and self.request.time_start is not None: time_delt = self.request.time_end - self.request.time_start time_str = "%.2f sec" % time_delt.total_seconds() if self.request.use_tls: is_ssl = 'YES' else: is_ssl = 'NO' if self.request.time_start: time_made_str = datetime_string(self.request.time_start) else: time_made_str = '--' verb = self.request.method host = self.request.dest_host self._add_info('Made on', time_made_str) self._add_info('URL', get_full_url(self.request)) self._add_info('Host', host) self._add_info('Path', self.request.url.path) self._add_info('Verb', verb) self._add_info('Status Code', response_code) self._add_info('Request Length', reqlen) self._add_info('Response Length', rsplen) if self.request.response and self.request.response.unmangled: self._add_info('Unmangled Response Length', self.request.response.unmangled.content_length) self._add_info('Time', time_str) self._add_info('Port', str(self.request.dest_port)) self._add_info('SSL', is_ssl) self._add_info('Mangled', mangle_str) self._add_info('Tags', ', '.join(self.request.tags)) class ParamWidget(QWidget): def __init__(self, *args, **kwargs): QWidget.__init__(self, *args, **kwargs) self.request = None self.setLayout(QVBoxLayout()) self.tab_widget = QTabWidget() self.urltable = QTableWidget() self.urltable.setColumnCount(2) self.posttable = QTableWidget() self.posttable.setColumnCount(2) self.cookietable = QTableWidget() self.cookietable.setColumnCount(2) self.tab_widget.addTab(self.urltable, "URL") self.tab_widget.addTab(self.posttable, "POST") self.tab_widget.addTab(self.cookietable, "Cookies") self.format_table(self.urltable) self.format_table(self.posttable) self.format_table(self.cookietable) self.layout().addWidget(self.tab_widget) def _add_info(self, table, k, v): row = table.rowCount() table.insertRow(row) item1 = QTableWidgetItem(k) item1.setFlags(item1.flags() ^ Qt.ItemIsEditable) table.setItem(row, 0, item1) table.setItem(row, 1, QTableWidgetItem(v)) def format_table(self, table): table.verticalHeader().hide() table.horizontalHeader().hide() table.horizontalHeader().setSectionResizeMode(QHeaderView.ResizeToContents) table.verticalHeader().setSectionResizeMode(QHeaderView.ResizeToContents) table.horizontalHeader().setStretchLastSection(True) def clear_tables(self): self.urltable.setRowCount(0) self.posttable.setRowCount(0) self.cookietable.setRowCount(0) def set_request(self, req): with DisableUpdates(self.urltable, self.posttable, self.cookietable): self.clear_tables() if req is None: return post_params = req.parameters() url_params = req.url.parameters() cookies = [(k, v) for k, v in req.cookie_iter()] if url_params: for k, vv in url_params.items(): for v in vv: self._add_info(self.urltable, k, v) if post_params: for k, vv in post_params.items(): for v in vv: self._add_info(self.posttable, k, v) if cookies: for k, v in cookies: self._add_info(self.cookietable, k, v) class TagList(QTableWidget): tagsUpdated = pyqtSignal(set) # list part of the tag tab def __init__(self, *args, **kwargs): QTableWidget.__init__(self, *args, **kwargs) self.tags = set() # Set up table self.setColumnCount(1) self.horizontalHeader().hide() self.horizontalHeader().setSectionResizeMode(QHeaderView.Stretch) self.verticalHeader().hide() self.verticalHeader().setSectionResizeMode(QHeaderView.ResizeToContents) self.setEditTriggers(QAbstractItemView.NoEditTriggers) def add_tag(self, tag): self.tags.add(tag) self.redraw_table() self.tagsUpdated.emit(set(self.tags)) def set_tags(self, tags, emit=True): self.tags = set(tags) self.redraw_table() if emit: self.tagsUpdated.emit(set(self.tags)) def clear_tags(self): self.tags = set() self.redraw_table() self.tagsUpdated.emit(set(self.tags)) def _append_str_row(self, fstr): row = self.rowCount() self.insertRow(row) self.setItem(row, 0, QTableWidgetItem(fstr)) def redraw_table(self): self.setRowCount(0) for tag in sorted(self.tags): self._append_str_row(tag) @pyqtSlot() def delete_selected(self): rows = self.selectionModel().selectedRows() if len(rows) == 0: return for idx in rows: tag = self.item(idx.row(), 0).text() self.tags.remove(tag) self.redraw_table() self.tagsUpdated.emit(set(self.tags)) def get_tags(self): return set(self.tags) class TagWidget(QWidget): tagsUpdated = pyqtSignal(set) def __init__(self, *args, **kwargs): QWidget.__init__(self, *args, **kwargs) self.setLayout(QVBoxLayout()) self.taglist = TagList() self.taglist.tagsUpdated.connect(self.tagsUpdated) self.layout().addWidget(self.taglist) self.taginput = QLineEdit() self.taginput.returnPressed.connect(self.add_tag) self.addbutton = QToolButton() self.addbutton.setText("+") self.removebutton = QToolButton() self.removebutton.setText("-") editbar = QHBoxLayout() editbar.addWidget(self.addbutton) editbar.addWidget(self.removebutton) editbar.addWidget(self.taginput) self.removebutton.clicked.connect(self.taglist.delete_selected) self.addbutton.clicked.connect(self.add_tag) self.layout().addLayout(editbar) @pyqtSlot() def add_tag(self): if self.readonly: return tag = self.taginput.text() if tag == "": return self.taglist.add_tag(tag) self.taginput.setText("") def set_read_only(self, readonly): self.readonly = readonly self.addbutton.setEnabled(not readonly) self.removebutton.setEnabled(not readonly) class ReqViewWidget(QWidget): requestEdited = pyqtSignal(HTTPRequest) def __init__(self, info_tab=False, param_tab=False, tag_tab=False, *args, **kwargs): QWidget.__init__(self, *args, **kwargs) self.request = None self.setLayout(QVBoxLayout()) self.layout().setSpacing(0) self.layout().setContentsMargins(0, 0, 0, 0) view_layout = QGridLayout() view_layout.setSpacing(3) view_layout.setContentsMargins(0, 0, 0, 0) self.req_edit = ComboEditor() self.rsp_edit = ComboEditor() self.req_edit.setReadOnly(True) self.rsp_edit.setReadOnly(True) view_layout.addWidget(self.req_edit, 0, 0) view_layout.addWidget(self.rsp_edit, 0, 1) view_widg = QWidget() view_widg.setLayout(view_layout) use_tab = False if info_tab or tag_tab: # or or use_tab = True self.tab_widget = QTabWidget() self.tab_widget.addTab(view_widg, "Message") self.info_tab = False self.info_widg = None if info_tab: self.info_tab = True self.info_widg = InfoWidget() self.tab_widget.addTab(self.info_widg, "Info") self.param_tab = False self.param_widg = None if param_tab: self.param_tab = True self.param_widg = ParamWidget() self.tab_widget.addTab(self.param_widg, "Params") self.tag_tab = False self.tag_widg = None if tag_tab: self.tag_tab = True self.tag_widg = TagWidget() self.tab_widget.addTab(self.tag_widg, "Tags") if use_tab: self.layout().addWidget(self.tab_widget) else: self.layout().addWidget(view_widg) def set_read_only(self, ro): self.req_edit.setReadOnly(ro) def set_tags_read_only(self, ro): if self.tag_tab: self.tag_widg.set_read_only(ro) def get_request(self): try: req = parse_request(self.req_edit.get_bytes()) req.dest_host = self.dest_host req.dest_port = self.dest_port req.use_tls = self.use_tls if self.tag_widg: req.tags = self.tag_widg.taglist.get_tags() return req except Exception as e: raise e return None @pyqtSlot(HTTPRequest) def set_request(self, req): self.req = req self.dest_host = "" self.dest_port = -1 self.use_tls = False if req: self.dest_host = req.dest_host self.dest_port = req.dest_port self.use_tls = req.use_tls self.update_editors() if self.info_tab: self.info_widg.set_request(req) if self.tag_tab: if req: self.tag_widg.taglist.set_tags(req.tags, emit=False) if self.param_tab: self.param_widg.set_request(req) def update_editors(self): self.req_edit.set_bytes(b"") self.rsp_edit.set_bytes(b"") lex = HybridHttpLexer() if self.req is not None: self.req_edit.set_bytes_highlighted(self.req.full_message(), lexer=lex) if self.req.response is not None: self.rsp_edit.set_bytes_highlighted(self.req.response.full_message(), lexer=lex) def show_message(self): self.tab_widget.setCurrentIndex(0) ================================================ FILE: guppyproxy/settings.py ================================================ from guppyproxy.util import list_remove, display_error_box, set_default_dialog_dir, default_dialog_dir, save_dialog, open_dialog from guppyproxy.proxy import MessageError from guppyproxy.config import ProxyConfig from PyQt5.QtWidgets import QWidget, QTableWidget, QTableWidgetItem, QHeaderView, QAbstractItemView, QFormLayout, QVBoxLayout, QHBoxLayout, QPushButton, QLineEdit, QSizePolicy, QToolButton, QCheckBox, QLabel from PyQt5.QtCore import pyqtSlot, pyqtSignal import os import copy class ListenerList(QTableWidget): listenersUpdated = pyqtSignal(list) # list part of the listener tab def __init__(self, *args, **kwargs): QTableWidget.__init__(self, *args, **kwargs) self.listeners = [] # Set up table self.setColumnCount(1) self.horizontalHeader().hide() self.horizontalHeader().setSectionResizeMode(QHeaderView.Stretch) self.verticalHeader().hide() self.verticalHeader().setSectionResizeMode(QHeaderView.ResizeToContents) self.setEditTriggers(QAbstractItemView.NoEditTriggers) def _add_listener(self, interface, port): self.listeners.append((interface, port)) def add_listener(self, interface, port): self.listeners.append((interface, port)) self.redraw_table() self.listenersUpdated.emit(self.listeners[:]) def set_listeners(self, listeners): self.listeners = [] for interface, port in listeners: self._add_listener(interface, port) self.redraw_table() self.listenersUpdated.emit(copy.deepcopy(self.listeners)) def _append_row(self, interface, port): row = self.rowCount() self.insertRow(row) self.setItem(row, 0, QTableWidgetItem("%s:%s" % (interface, port))) def redraw_table(self): self.setRowCount(0) for interface, port in self.listeners: self._append_row(interface, port) @pyqtSlot() def delete_selected(self): rows = self.selectionModel().selectedRows() if len(rows) == 0: return rownums = [idx.row() for idx in rows] self.listeners = list_remove(self.listeners, rownums) self.redraw_table() self.listenersUpdated.emit(self.listeners[:]) def clear(self): self.listeners = [] self.redraw_table() self.listenersUpdated.emit(self.listeners[:]) def get_listeners(self): return self.listeners[:] class ListenerWidget(QWidget): listenersUpdated = pyqtSignal(list) def __init__(self, *args, **kwargs): QWidget.__init__(self, *args, **kwargs) self.setLayout(QVBoxLayout()) self.layout().setContentsMargins(0, 0, 0, 0) self.listenerlist = ListenerList() self.listenerlist.listenersUpdated.connect(self.listenersUpdated) self.layout().addWidget(self.listenerlist) self.hostinput = QLineEdit() self.hostinput.setText("127.0.0.1") self.hostinput.returnPressed.connect(self.add_listener) self.portinput = QLineEdit() self.portinput.setMaxLength(5) self.portinput.setSizePolicy(QSizePolicy.Minimum, QSizePolicy.Preferred) self.portinput.returnPressed.connect(self.add_listener) self.addbutton = QToolButton() self.addbutton.setText("+") self.removebutton = QToolButton() self.removebutton.setText("-") editbar = QHBoxLayout() editbar.addWidget(self.addbutton) editbar.addWidget(self.removebutton) editbar.addWidget(QLabel("Interface:")) editbar.addWidget(self.hostinput) editbar.addWidget(QLabel("Port:")) editbar.addWidget(self.portinput) self.removebutton.clicked.connect(self.listenerlist.delete_selected) self.addbutton.clicked.connect(self.add_listener) self.layout().addLayout(editbar) @pyqtSlot() def add_listener(self): host = self.hostinput.text() port = self.portinput.text() if host == "": return if port == "": return try: port = int(port) except Exception: return self.listenerlist.add_listener(host, port) self.hostinput.setText("127.0.0.1") self.portinput.setText("") def set_listeners(self, listeners): self.listenerlist.set_listeners(listeners) class DatafileWidget(QWidget): datafileLoaded = pyqtSignal(str) def __init__(self, *args, **kwargs): QWidget.__init__(self, *args, **kwargs) self.setLayout(QHBoxLayout()) self.layout().setContentsMargins(0, 0, 0, 0) self.datapath = QLineEdit() newbutton = QPushButton("New") newbutton.clicked.connect(self.new_datafile) browsebutton = QPushButton("Open") browsebutton.clicked.connect(self.open_datafile) confirmbutton = QPushButton("Go!") confirmbutton.clicked.connect(self._load_datafile) self.layout().addWidget(self.datapath) self.layout().addWidget(newbutton) self.layout().addWidget(browsebutton) self.layout().addWidget(confirmbutton) @pyqtSlot() def _load_datafile(self): path = self.datapath.text() self.datafileLoaded.emit(path) @pyqtSlot() def new_datafile(self): fname = save_dialog(self, filter_string="Database File (*.gpy)") if not fname: return if len(fname) < 4 and fname[:-4] != ".gpy": fname += ".gpy" set_default_dialog_dir(fname) self.datapath.setText(fname) self._load_datafile() @pyqtSlot() def open_datafile(self): fname = open_dialog(self) if not fname: return self.datapath.setText(fname) set_default_dialog_dir(fname) self._load_datafile() class ProxyInfoWidget(QWidget): proxyInfoUpdated = pyqtSignal(dict) def __init__(self, *args, **kwargs): QWidget.__init__(self, *args, **kwargs) self.setLayout(QFormLayout()) self.enablebox = QCheckBox() self.enablebox.stateChanged.connect(self._enable_cb_statechange) self.hostinput = QLineEdit() self.portinput = QLineEdit() self.credsbox = QCheckBox() self.credsbox.stateChanged.connect(self._login_cb_statechange) self.credsbox.setCheckState(0) self.usernameinput = QLineEdit() self.passwordinput = QLineEdit() self.passwordinput.setEchoMode(QLineEdit.Password) self.socksbox = QCheckBox() self.confirmbutton = QPushButton("Confirm") self.confirmbutton.clicked.connect(self._confirm_entry) self.layout().addRow(QLabel("Use Proxy"), self.enablebox) self.layout().addRow(QLabel("Host"), self.hostinput) self.layout().addRow(QLabel("Port"), self.portinput) self.layout().addRow(QLabel("Use Login"), self.credsbox) self.layout().addRow(QLabel("Username"), self.usernameinput) self.layout().addRow(QLabel("Password"), self.passwordinput) self.layout().addRow(QLabel("Use SOCKS"), self.socksbox) self.layout().addRow(QLabel(""), self.confirmbutton) self._set_enabled(False) self._set_login_enabled(False) @pyqtSlot(int) def _login_cb_statechange(self, state): if state == 0: self._set_login_enabled(False) else: self._set_login_enabled(True) @pyqtSlot(int) def _enable_cb_statechange(self, state): if state == 0: self._set_enabled(False) else: self._set_enabled(True) def _set_enabled(self, enabled): self.all_enabled = enabled self.hostinput.setEnabled(enabled) self.portinput.setEnabled(enabled) self.credsbox.setEnabled(enabled) self.socksbox.setEnabled(enabled) if enabled: self._set_login_enabled(self.loginenabled) else: self._set_login_enabled(False) def _set_login_enabled(self, enabled): self.loginenabled = enabled self.usernameinput.setEnabled(enabled) self.passwordinput.setEnabled(enabled) def _fill_form(self, enabled, host, port, need_creds, username, password, use_socks): if enabled: self.enablebox.setCheckState(2) else: self.enablebox.setCheckState(0) self.hostinput.setText(host) if port == 0: self.portinput.setText("") else: self.portinput.setText(str(port)) if need_creds: self.credsbox.setCheckState(2) else: self.credsbox.setCheckState(0) self.usernameinput.setText(username) self.passwordinput.setText(password) if use_socks: self.socksbox.setCheckState(2) else: self.socksbox.setCheckState(0) def _confirm_entry(self): use_proxy = not (self.enablebox.checkState() == 0) if use_proxy: host = self.hostinput.text() port = self.portinput.text() try: port = int(port) except Exception: return is_socks = not (self.socksbox.checkState() == 0) if self.credsbox.checkState() == 0: username = "" password = "" else: username = self.usernameinput.text() password = self.passwordinput.text() entry = {"use_proxy": use_proxy, "host": host, "port": port, "is_socks": is_socks, "username": username, "password": password} else: entry = {"use_proxy": False, "host": "", "port": 0, "is_socks": False, "username": "", "password": ""} self.proxyInfoUpdated.emit(entry) class SettingsWidget(QWidget): datafileLoaded = pyqtSignal() def __init__(self, client, *args, **kwargs): QWidget.__init__(self, *args, **kwargs) self.client = client self.setLayout(QFormLayout()) # Datafile self.datafilewidg = DatafileWidget() self.datafilewidg.datafileLoaded.connect(self._load_datafile) self.layout().addRow(QLabel("Datafile"), self.datafilewidg) # Listeners self.listenerwidg = ListenerWidget() self.listenerwidg.listenersUpdated.connect(self._listeners_updated) self.layout().addRow(QLabel("Listeners"), self.listenerwidg) # Proxy settings self.proxywidg = ProxyInfoWidget() self.proxywidg.proxyInfoUpdated.connect(self._set_proxy_settings) self.layout().addRow(QLabel("Proxy Settings"), self.proxywidg) self.load_config() def load_config(self): # Load config self.config = ProxyConfig() try: configs = self.client.get_plugin_value(ProxyConfig.PLUGIN_KEY) except MessageError: configs = self.config.dumps() self.client.set_plugin_value(ProxyConfig.PLUGIN_KEY, configs) self.config.loads(configs) new_listeners = [(vals[0], vals[1]) for vals in self.config.listeners] self.listenerwidg.set_listeners(new_listeners) # fill proxy self.proxywidg._fill_form(self.config.use_proxy, self.config.proxy_host, self.config.proxy_port, not (self.config.proxy_username == "" and self.config.proxy_password == ""), self.config.proxy_username, self.config.proxy_password, self.config.is_socks_proxy) self.reload_listeners() @pyqtSlot(str) def _load_datafile(self, path): old_storage = self.client.proxy_storage try: storage = self.client.add_sqlite_storage(path, "tmpprefix") except MessageError as e: display_error_box("Could not load datafile:\n%s" % e) return self.client.close_storage(old_storage) self.client.set_storage_prefix(storage.storage_id, "") self.client.set_proxy_storage(storage.storage_id) self.client.disk_storage = storage self.load_config() self.datafileLoaded.emit() @pyqtSlot(list) def _listeners_updated(self, new_listeners): old_listensers = self.client.get_listeners() parsedold = {} for lid, addr in old_listensers: iface, port = addr.rsplit(':', 1) port = int(port) parsedold[(iface, port)] = lid oldset = set(parsedold.keys()) newset = set(new_listeners) hosts_to_remove = oldset.difference(new_listeners) ids_to_remove = [parsedold[i] for i in hosts_to_remove] hosts_to_add = newset.difference(oldset) failed_listeners = [] for i in ids_to_remove: self.client.remove_listener(i) for iface, port in hosts_to_add: try: self.client.add_listener(iface, port) except MessageError as e: err = "%s:%s: %s" % (iface, port, e) failed_listeners.append(err) if failed_listeners: errmsg = "Failed to create listener(s):\n\n%s" % ('\n'.join(failed_listeners)) display_error_box(errmsg) self.config.set_listeners([(host, port, None) for host, port in new_listeners]) # ignore transparent self.save_config() @pyqtSlot(dict) def _set_proxy_settings(self, proxy_data): self.config.proxy = proxy_data use_creds = (self.config.proxy_username != "" or self.config.proxy_password != "") self.client.set_proxy(self.config.use_proxy, self.config.proxy_host, self.config.proxy_port, use_creds, self.config.proxy_username, self.config.proxy_password, self.config.is_socks_proxy) self.save_config() def reload_listeners(self): hosts = self.client.get_listeners() pairs = [] for lid, iface in hosts: host, port = iface.rsplit(":", 1) pairs.append((host, port)) self.listenerwidg.blockSignals(True) self.listenerwidg.set_listeners(pairs) self.listenerwidg.blockSignals(False) def save_config(self): self.client.set_plugin_value(ProxyConfig.PLUGIN_KEY, self.config.dumps()) ================================================ FILE: guppyproxy/shortcuts.py ================================================ from guppyproxy.util import display_info_box, paste_clipboard from PyQt5.QtCore import pyqtSlot, QObject, Qt from PyQt5.QtWidgets import QShortcut from PyQt5.QtGui import QKeySequence class GuppyShortcuts(QObject): ACT_NAV_FILTER_TEXT = 0 ACT_NAV_FILTER_DROPDOWN = 1 ACT_NAV_HISTORY = 2 ACT_NAV_TREE = 3 ACT_NAV_REPEATER = 4 ACT_NAV_INTERCEPTOR = 5 ACT_NAV_DECODER = 6 ACT_NAV_DECODER_PASTE = 7 ACT_NAV_FILTER_POP = 8 ACT_OPEN = 9 ACT_NEW = 10 ACT_NAV_MACRO_ACTIVE = 11 ACT_NAV_MACRO_INT = 12 def __init__(self, guppy_window): QObject.__init__(self) self.guppy_window = guppy_window self.combos = {} self.add_shortcut(self.ACT_NAV_FILTER_TEXT, "Navigate to filter text input", self.nav_to_filter_text, QKeySequence(Qt.CTRL+Qt.Key_U)) self.add_shortcut(self.ACT_NAV_FILTER_DROPDOWN, "Navigate to filter dropdown input", self.nav_to_filter_dropdown, QKeySequence(Qt.CTRL+Qt.Key_I)) self.add_shortcut(self.ACT_NAV_FILTER_POP, "Navigate to filters and pop most recent filter", self.nav_to_filter_pop, QKeySequence(Qt.CTRL+Qt.Key_P)) self.add_shortcut(self.ACT_NAV_HISTORY, "Navigate to request list", self.nav_to_history, QKeySequence(Qt.CTRL+Qt.Key_J)) self.add_shortcut(self.ACT_NAV_TREE, "Navigate to tree view", self.nav_to_tree, QKeySequence(Qt.CTRL+Qt.Key_T)) self.add_shortcut(self.ACT_NAV_REPEATER, "Navigate to repeater", self.nav_to_repeater, QKeySequence(Qt.CTRL+Qt.Key_R)) self.add_shortcut(self.ACT_NAV_INTERCEPTOR, "Navigate to interceptor", self.nav_to_interceptor, QKeySequence(Qt.CTRL+Qt.Key_E)) self.add_shortcut(self.ACT_NAV_DECODER, "Navigate to decoder", self.nav_to_decoder, QKeySequence(Qt.CTRL+Qt.Key_D)) self.add_shortcut(self.ACT_NAV_DECODER_PASTE, "Navigate to decoder and fill with clipboard", self.nav_to_decoder_and_paste, QKeySequence(Qt.CTRL+Qt.SHIFT+Qt.Key_D)) self.add_shortcut(self.ACT_OPEN, "Open datafile", self.open_datafile, QKeySequence(Qt.CTRL+Qt.SHIFT+Qt.Key_O)) self.add_shortcut(self.ACT_NEW, "New datafile", self.new_datafile, QKeySequence(Qt.CTRL+Qt.SHIFT+Qt.Key_N)) self.add_shortcut(self.ACT_NAV_MACRO_ACTIVE, "Navigate to active macros", self.nav_to_active_macros, QKeySequence(Qt.CTRL+Qt.Key_M)) self.add_shortcut(self.ACT_NAV_MACRO_INT, "Navigate to intercepting macros", self.nav_to_int_macros, QKeySequence(Qt.CTRL+Qt.Key_N)) def add_shortcut(self, action, desc, func, key=None): sc = QShortcut(self.guppy_window) self.combos[action] = (sc, desc) sc.activated.connect(func) if key: sc.setKey(key) def set_key(self, action, key): sc = self.combos[action][0] sc.setKey(key) def get_desc(self, action): return self.combos[action][1] @pyqtSlot() def nav_to_filter_text(self): self.guppy_window.show_hist_tab() self.guppy_window.historyWidget.show_filters() self.guppy_window.historyWidget.set_filter_is_text(True) self.guppy_window.historyWidget.filterWidg.entry.text_entry.textEntry.setFocus() @pyqtSlot() def nav_to_filter_dropdown(self): self.guppy_window.show_hist_tab() self.guppy_window.historyWidget.show_filters() self.guppy_window.historyWidget.set_filter_is_text(False) @pyqtSlot() def nav_to_filter_pop(self): self.guppy_window.show_hist_tab() self.guppy_window.historyWidget.show_filters() self.guppy_window.historyWidget.filterWidg.pop_phrase() @pyqtSlot() def nav_to_history(self): self.guppy_window.show_hist_tab() self.guppy_window.historyWidget.show_history() self.guppy_window.historyWidget.reqview.show_message() @pyqtSlot() def nav_to_tree(self): self.guppy_window.show_hist_tab() self.guppy_window.historyWidget.show_tree() @pyqtSlot() def nav_to_repeater(self): self.guppy_window.show_repeater_tab() @pyqtSlot() def nav_to_interceptor(self): self.guppy_window.show_interceptor_tab() @pyqtSlot() def nav_to_decoder(self): self.guppy_window.show_decoder_tab() @pyqtSlot() def nav_to_decoder_and_paste(self): self.guppy_window.show_decoder_tab() text = paste_clipboard() self.guppy_window.decoderWidget.decoder_input.editor.set_bytes(text.encode()) @pyqtSlot() def open_datafile(self): self.guppy_window.settingsWidget.datafilewidg.open_datafile() @pyqtSlot() def new_datafile(self): self.guppy_window.settingsWidget.datafilewidg.new_datafile() @pyqtSlot() def nav_to_active_macros(self): self.guppy_window.show_active_macro_tab() @pyqtSlot() def nav_to_int_macros(self): self.guppy_window.show_int_macro_tab() ================================================ FILE: guppyproxy/util.py ================================================ import os import string import time import datetime import random from guppyproxy.proxy import get_full_url, Headers from pygments.formatters import HtmlFormatter from pygments.styles import get_style_by_name from PyQt5.QtWidgets import QMessageBox, QMenu, QApplication, QFileDialog from PyQt5.QtGui import QColor str_colorcache = {} _last_file_dialog_dir = "" _is_app = False class DisableUpdates: def __init__(self, *args): self.prevs = [(obj, obj.updatesEnabled()) for obj in args] self.undoredo = [] self.readonly = [] for obj, _ in self.prevs: obj.setUpdatesEnabled(False) if hasattr(obj, 'setReadOnly'): self.undoredo.append((obj, obj.isUndoRedoEnabled())) obj.setUndoRedoEnabled(False) if hasattr(obj, 'setReadOnly'): self.readonly.append((obj, obj.isReadOnly())) obj.setReadOnly(True) def __enter__(self): return self def __exit__(self, exc_type, exc_value, traceback): for obj, prev in self.prevs: obj.setUpdatesEnabled(prev) for obj, prev in self.undoredo: obj.setUndoRedoEnabled(prev) for obj, prev in self.readonly: obj.setReadOnly(prev) def str_hash_code(s): h = 0 n = len(s) - 1 for c in s.encode(): h += c * 31 ** n n -= 1 return h qtprintable = string.digits + string.ascii_letters + string.punctuation + ' ' + '\t' + '\n' def dbgline(): from inspect import currentframe, getframeinfo cf = currentframe() print(getframeinfo(cf.f_back).filename, cf.f_back.f_lineno) def is_printable(s): for c in s: if c not in qtprintable: return False return True def printable_data(data, include_newline=True): chars = [] printable = string.printable if not include_newline: printable = [c for c in printable if c != '\n'] for c in data: if chr(c) in printable: chars.append(chr(c)) else: chars.append('.') return ''.join(chars) def max_len_str(s, ln): if ln <= 3: return "..." if len(s) <= ln: return s return s[:(ln - 3)] + "..." def display_error_box(msg, title="Error"): msgbox = QMessageBox() msgbox.setIcon(QMessageBox.Warning) msgbox.setText(msg) msgbox.setWindowTitle(title) msgbox.setStandardButtons(QMessageBox.Ok) return msgbox.exec_() def display_info_box(msg, title="Message"): msgbox = QMessageBox() msgbox.setIcon(QMessageBox.Information) msgbox.setText(msg) msgbox.setWindowTitle(title) msgbox.setStandardButtons(QMessageBox.Ok) return msgbox.exec_() def copy_to_clipboard(s): QApplication.clipboard().setText(s) def paste_clipboard(): return QApplication.clipboard().text() def running_as_app(): global _is_app return _is_app def set_running_as_app(is_app): global _is_app _is_app = is_app def default_dialog_dir(): global _last_file_dialog_dir if _last_file_dialog_dir != "": return _last_file_dialog_dir if running_as_app(): return os.path.expanduser('~') else: return os.getcwd() def set_default_dialog_dir(s): global _last_file_dialog_dir _last_file_dialog_dir = os.path.dirname(s) def save_dialog(parent, filter_string="Any File (*)", caption="Save File", default_dir=None, default_name=None): default_dir = default_dir or default_dialog_dir() fname, _ = QFileDialog.getSaveFileName(parent, caption, default_dir, filter_string) if not fname: return None set_default_dialog_dir(os.path.abspath(fname)) return fname def open_dialog(parent, filter_string="Any File (*)", default_dir=None): fname, _ = QFileDialog.getOpenFileName(parent, "Save File", default_dialog_dir(), filter_string) if not fname: return None set_default_dialog_dir(os.path.abspath(fname)) return fname def display_req_context(parent, client, req, event, repeater_widget=None, req_view_widget=None, macro_widget=None, save_option=False): from guppyproxy.macros import create_macro_template menu = QMenu(parent) repeaterAction = None displayUnmangledReq = None displayUnmangledRsp = None viewInBrowser = None macroAction = None saveToHistAction = None if save_option: saveToHistAction = menu.addAction("Save request to history") if repeater_widget is not None: repeaterAction = menu.addAction("Send to repeater") if req.unmangled and req_view_widget: displayUnmangledReq = menu.addAction("View unmangled request") if req.response and req.response.unmangled and req_view_widget: displayUnmangledRsp = menu.addAction("View unmangled response") if req.db_id != "": viewInBrowser = menu.addAction("View response in browser") curlAction = menu.addAction("Copy as cURL command") saveAction = menu.addAction("Save response to file") saveFullActionReq = menu.addAction("Save request to file (full message)") saveFullActionRsp = menu.addAction("Save response to file (full message)") saveMacroAction = menu.addAction("Create active macro with selected requests") if macro_widget is not None: macroAction = menu.addAction("Add to active macro input") action = menu.exec_(parent.mapToGlobal(event.pos())) if save_option and action == saveToHistAction: client.save_new(req) if repeaterAction and action == repeaterAction: repeater_widget.set_request(req) if displayUnmangledReq and action == displayUnmangledReq: req_view_widget.set_request(req.unmangled) if displayUnmangledRsp and action == displayUnmangledRsp: new_req = req.copy() new_req.response = req.response.unmangled req_view_widget.set_request(new_req) if viewInBrowser and action == viewInBrowser: url = "http://puppy/rsp/%s" % req.db_id copy_to_clipboard(url) display_info_box("URL copied to clipboard.\n\nPaste the URL into the browser being proxied") if action == curlAction: curl = curl_command(req) if curl is None: display_error_box("Request could not be converted to cURL command") try: copy_to_clipboard(curl) except Exception: display_error_box("Error copying command to clipboard") if action == saveAction: if not req.response: display_error_box("No response associated with request") return fname = req.url.path.rsplit('/', 1)[-1] saveloc = save_dialog(parent, default_name=fname) if not saveloc: return with open(saveloc, 'wb') as f: f.write(req.response.body) if action == saveFullActionRsp: if not req.response: display_error_box("No response associated with request") return fname = req.url.path.rsplit('/', 1)[-1] + ".response" saveloc = save_dialog(parent, default_name=fname) if not saveloc: return with open(saveloc, 'wb') as f: f.write(req.response.full_message()) if action == saveFullActionReq: fname = req.url.path.rsplit('/', 1)[-1] + ".request" saveloc = save_dialog(parent, default_name=fname) if not saveloc: return with open(saveloc, 'wb') as f: f.write(req.full_message()) if macroAction and action == macroAction: macro_widget.add_requests([req]) if action == saveMacroAction: saveloc = save_dialog(parent, default_name="macro.py") if saveloc == None: return with open(saveloc, 'w') as f: f.write(create_macro_template([req])) def display_multi_req_context(parent, client, reqs, event, macro_widget=None, save_option=False): from guppyproxy.macros import create_macro_template menu = QMenu(parent) if macro_widget: macroAction = menu.addAction("Add to active macro input") if save_option: saveAction = menu.addAction("Save requests to history") saveMacroAction = menu.addAction("Create active macro with selected requests") action = menu.exec_(parent.mapToGlobal(event.pos())) if macro_widget and action == macroAction: if macro_widget: macro_widget.add_requests(reqs) if save_option and action == saveAction: for req in reqs: client.save_new(req) if action == saveMacroAction: saveloc = save_dialog(parent, default_name="macro.py") if saveloc == None: return with open(saveloc, 'w') as f: f.write(create_macro_template(reqs)) def method_color(method): if method.lower() == 'get': return QColor(240, 240, 255) if method.lower() == 'post': return QColor(255, 255, 230) if method.lower() == 'put': return QColor(255, 240, 240) return QColor(255, 255, 255) def sc_color(sc): if sc[0] == '2': return QColor(240, 255, 240) if sc[0] == '3': return QColor(255, 240, 255) if sc[0] == '4': return QColor(255, 240, 240) if sc[0] == '5': return QColor(255, 255, 230) return QColor(255, 255, 255) def host_color(hostport): return str_color(hostport, lighten=150, seed=1) def str_color(s, lighten=0, seed=0): global str_colorcache if s in str_colorcache: return str_colorcache[s] hashval = str_hash_code(s)+seed gen = random.Random() gen.seed(hashval) r = gen.randint(lighten, 255) g = gen.randint(lighten, 255) b = gen.randint(lighten, 255) col = QColor(r, g, b) str_colorcache[s] = col return col def hostport(req): # returns host:port if to a port besides 80 or 443 host = req.dest_host if req.use_tls and req.dest_port == 443: return host if (not req.use_tls) and req.dest_port == 80: return host return "%s:%d" % (host, req.dest_port) def _sh_esc(s): sesc = s.replace("\\", "\\\\") sesc = sesc.replace("\"", "\\\"") return sesc def curl_command(req): # Creates a curl command that submits a given request command = "curl" if req.method != "GET": command += " -X %s" % req.method for k, v in req.headers.pairs(): if k.lower == "content-length": continue kesc = _sh_esc(k) vesc = _sh_esc(v) command += ' --header "%s: %s"' % (kesc, vesc) if req.body: if not is_printable(req.body): return None besc = _sh_esc(req.body) command += ' -d "%s"' % besc command += ' "%s"' % _sh_esc(get_full_url(req)) return command def list_remove(lst, inds): return [i for j, i in enumerate(lst) if j not in inds] def hexdump(src, length=16): FILTER = ''.join([(len(repr(chr(x))) == 3) and chr(x) or '.' for x in range(256)]) lines = [] for c in range(0, len(src), length): chars = src[c:c + length] hex = ' '.join(["%02x" % x for x in chars]) printable = ''.join(["%s" % ((x <= 127 and FILTER[x]) or '.') for x in chars]) lines.append("%04x %-*s %s\n" % (c, length * 3, hex, printable)) return ''.join(lines) def confirm(message, default='n'): """ A helper function to get confirmation from the user. It prints ``message`` then asks the user to answer yes or no. Returns True if the user answers yes, otherwise returns False. """ if 'n' in default.lower(): default = False else: default = True print(message) if default: answer = input('(Y/n) ') else: answer = input('(y/N) ') if not answer: return default if answer[0].lower() == 'y': return True else: return False # Taken from http://stackoverflow.com/questions/4770297/python-convert-utc-datetime-string-to-local-datetime def utc2local(utc): epoch = time.mktime(utc.timetuple()) offset = datetime.datetime.fromtimestamp(epoch) - datetime.datetime.utcfromtimestamp(epoch) return utc + offset def datetime_string(dt): dtobj = utc2local(dt) time_made_str = dtobj.strftime('%a, %b %d, %Y, %I:%M:%S.%f %p') return time_made_str def query_to_str(query): retstr = "" for p in query: fstrs = [] for f in p: fstrs.append(' '.join(f)) retstr += (' OR '.join(fstrs)) return retstr def textedit_highlight(text, lexer): from pygments import highlight wrapper_head = """
"""
    wrapper_foot = "
" highlighted = highlight(text, lexer, HtmlFormatter(noclasses=True, style=get_style_by_name("colorful"), nowrap=True)) highlighted = wrapper_head + highlighted + wrapper_foot return highlighted ================================================ FILE: install.sh ================================================ #!/bin/bash prompt_yn() { read -p "$1 (yN) " yn; case $yn in [Yy]* ) return 0;; * ) return 1;; esac } require() { if ! $@; then echo "Error running $@, exiting..."; exit 1; fi } GO="$(which go)" BUILDFLAGS="" PUPPYREPO="https://github.com/roglew/puppy.git" PUPPYVERSION="tags/0.2.6" INSTALLDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" TMPGOPATH="$INSTALLDIR/gopath" DATADIR="$HOME/.guppy" VIRTUALENVNAME="guppyenv" while getopts "g:f:r:dph" opt; do case $opt in g) GO="$OPTARG" ;; f) BUILDFLAGS="${OPTARG}" ;; r) PUPPYREPO="${OPTARG}" DEV="yes" ;; d) DEV="yes" ;; p) DOPUPPY="yes" ;; h) echo -e "Build script flags:" echo -e "-p\tCompile puppy from source rather than using pre-built binaries" echo -e "-g [path to go]\tUse specific go binary to compile puppy" echo -e "-f [arguments]\tArguments to pass to \"go build\". ie -f \"-ldflags -s\"" echo -e "-r [git repository link]\t download puppy from an alternate repository" echo -e "-d\tinstall puppy in development mode by using \"pip install -e\" to install puppy" echo -e "-h\tprint this help message" echo -e "" exit 0; ;; \?) echo "Invalid option: -$OPTARG" >&2 exit 1; ;; esac done if ! type "python3" > /dev/null; then echo "python3 not installed. Please install python3 and try again" exit 1; fi if ! type "pip" > /dev/null; then if ! type "easy_install" > /dev/null; then echo "pip not available. Please install pip then try again." exit 1; fi if prompt_yn "Installation requires pip. Install pip using \"sudo easy_install pup\"?"; then require sudo easy_install pip; else echo "Please install pip and try the installation again" exit 1; fi fi cd "$INSTALLDIR" mkdir -p $DATADIR if [ $DOPUPPY ]; then # Compile puppy from source if [ ! $GO ]; then if ! type "go" > /dev/null; then echo "go not installed. Please install go and try again" exit 1; fi fi # Set up fake gopath export GOPATH="$TMPGOPATH"; require mkdir -p "$GOPATH/src" # Clone the repo REPODIR="$GOPATH/src/puppy"; if [ ! -d "$REPODIR" ]; then # Clone the repo if it doesn't exist require mkdir -p "$REPODIR"; echo git clone "$PUPPYREPO" "$REPODIR"; require git clone "$PUPPYREPO" "$REPODIR"; fi # Check out the correct version cd "$REPODIR"; if [ $DEV ] || [ $REPODIR ]; then # If it's development, get the most recent version of puppy require git pull; else # if it's not development, get the specified version require git checkout "$PUPPYVERSION"; fi cd "$INSTALLDIR" # Get dependencies cd "$REPODIR"; echo "Getting puppy dependencies..." require "$GO" get ./...; # Build puppy into the data dir echo "Building puppy into $DATADIR/puppy..."; require mkdir -p "$DATADIR"; require "$GO" build -o "$DATADIR"/puppy $BUILDFLAGS "puppy/cmd/main"; else # copy the pre-compiled binary UNAME="$(uname -s)" PUPPYFILE="" if [ "$UNAME" = "Darwin" ]; then echo "copying mac version of pre-built puppy to $DATADIR/puppy" PUPPYFILE="puppy.osx" elif [ "$UNAME" = "Linux" ]; then if [ "$(uname -m)" = "x86_64" ]; then echo "copying 64-bit linux version of pre-built puppy to $DATADIR/puppy" PUPPYFILE="puppy.linux64" else echo "copying 32-bit linux version of pre-built puppy to $DATADIR/puppy" PUPPYFILE="puppy.linux32" fi else echo "could not detect system type. Please use -p to compile puppy from source (requires go installation)" exit 1; fi cp "$INSTALLDIR/puppyrsc/$PUPPYFILE" "$DATADIR/puppy" fi # Clear out old .pyc files require find "$INSTALLDIR/guppyproxy" -iname "*.pyc" -exec rm -f {} \; # Set up the virtual environment if ! type "virtualenv" > /dev/null; then if prompt_yn "\"virtualenv\" not installed. Install using pip?"; then require sudo pip install virtualenv else exit 1; fi fi VENVDIR="$DATADIR/venv"; require mkdir -p "$VENVDIR"; require virtualenv -p "$(which python3)" "$VENVDIR"; cd "$VENVDIR"; require source bin/activate; cd "$INSTALLDIR"; if [ -z $DEV ]; then require pip install -e . else require pip install . fi echo -e "#!/bin/bash\nsource \"$VENVDIR/bin/activate\";\nguppy \$@ || killall puppy;\n" > start chmod +x start; echo "" echo "Guppy installed. Run guppy by executing the generated \"start\" script." ================================================ FILE: puppyrsc/NOTE.md ================================================ These binaries are pre-built versions of puppy () which is used to proxy HTTP requests as they pass through the proxy. By default, installation will use these pre-built binaries. If you would like to compile the binary yourself, ensure you have a compatible go installation and use the `-p` flag with the install script. ================================================ FILE: puppyrsc/puppy.linux64 ================================================ [File too large to display: 15.1 MB] ================================================ FILE: puppyrsc/puppy.osx ================================================ [File too large to display: 14.9 MB] ================================================ FILE: setup.py ================================================ #!/usr/bin/env python import pkgutil from setuptools import setup, find_packages try: import py2app except ImportError: pass VERSION = "0.1.0" setup(name='GuppyProxy', version=VERSION, description='The Guppy Intercepting Proxy', author='Rob Glew', author_email='rglew56@gmail.com', packages=['guppyproxy'], app=['guppyproxy/gup.py'], include_package_data = True, license='MIT', options={'py2app': { 'packages': ['lxml','pygments','PyQt5'], 'iconfile': 'img/shark.icns', 'resources': ['puppyrsc'], } }, entry_points = { 'console_scripts':['guppy = guppyproxy.gup:start'], }, long_description="The Guppy Proxy", keywords='http proxy hacking 1337hax pwnurmum', install_requires=[ 'lxml>=4.1.1', 'Pygments>=2.0.2', 'PyQt5>=5.9', ], classifiers=[ 'Intended Audience :: Developers', 'Intended Audience :: Information Technology', 'Operating System :: MacOS', 'Operating System :: POSIX :: Linux', 'Development Status :: 2 - Pre-Alpha', 'Programming Language :: Python :: 3.6', 'License :: OSI Approved :: MIT License', 'Topic :: Security', ] )