[ { "path": ".gitignore", "content": "*.pyc\ndata.db\n.coverage\n.cache\ntests/.cache\n.DS_Store\nTAGS\nconfig.json\nbuild/*\n*.egg-info/*\n.#*\n*notes*\n*.org\nstart\n" }, { "path": "LICENSE", "content": "MIT License\n\nCopyright (c) 2018 Rob Glew\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n" }, { "path": "README.md", "content": "# Guppy Proxy\n\nThe Guppy Proxy is an intercepting proxy for performing web application security testing. Its features are often similar to, or straight up rippoffs from [Burp Suite](https://portswigger.net/burp/). However, Burp Suite has its own issues (search, licensing) which led to the creation of Guppy.\n\n![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_main.png)\n\n# Installation\n\n## Dependencies\n\nMake sure the following commands are available:\n\n* `python3`\n* `pip`\n* `virtualenv` (can be installed with pip)\n\n## Installing\n\n### Mac\n\n1. Download the .app of version of guppy [available here](https://guppydist.s3-us-west-2.amazonaws.com/GuppyProxy-0.0.15.zip)\n1. Start the application\n1. Add the CA cert in `~/.guppy/certs` to your browser as a CA\n1. Configure your browser to use `localhost:8080` as a proxy\n1. Navigate to a site and look at the history in the main window\n\n### Linux / Alternative for Mac\n\n1. Clone this repo somewhere it won't get deleted: `git clone https://github.com/roglew/guppy-proxy.git`\n1. `cd /path/to/guppy-proxy`\n1. `./install.sh` to use pre-built binary or `./install.sh -p` to compile the go component from source (requires a [go installation](https://golang.org/doc/install))\n1. Test that the application starts up and generate certs: `./start` (keep the window open and continue to test it works)\n1. Copy/symlink the generated `start` script somewhere in your PATH (i.e. `~/bin` if you have that included) and rename it to `guppy` if you want\n1. Add the CA cert in `~/.guppy/certs` to your browser as a CA\n1. Configure your browser to use `localhost:8080` as a proxy\n1. Navigate to a site and look at the history in the main window\n\n## Updating\n\n1. Navigate to the guppy-proxy folder with this repo in it\n1. `git pull` to pull the latest version\n1. run `./install.sh` again\n\nThe same start script as before should still work\n\n## Uninstalling\n\n1. Delete the guppy-proxy directory you made during installation\n1. Delete `~/.guppy`\n1. Remove the start script from wherever you put it\n\n# How to Use Guppy\n\n## History View\n\n![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_main.png)\n![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_pretty_view.png)\n![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_tree.png)\n\nThe first thing you see when you open Guppy is the history view. As requests pass through the proxy they are displayed in the lower half of the window. You can click a request to view the full request/response in the windows on the upper half or right click them for more options. The tabs on the upper half will let you view additional information about the selected request:\n\n* Messages - The full request/response\n* Info - A list of values associated with the message\n* Tags - Lets you view/edit the tags currently associated with the request\n\nThe bottom half has tabs which relate to all of the requests that have been recorded by the proxy:\n\n* List - A list of all of the requests that have been recorded by the proxy\n* Tree - A site map of all of the endpoints visited\n* Filters - An advanced search interface which is described below in the Filters section\n\n## Filters and Search\n\n![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_search.png)\n\nGuppy's main selling point over other similar proxies is its search. You can search for a wide variety of fields within a request or response and apply more than one search condition at the same time. This allows you to perform complex searches over your history so that you can always find the request that you want. You would be surprised what you can find when searching for paths, headers, and body contents. For example you can find potential CSRF targets by finding requests which are not GET requests and also do not have a header with \"CSRF\" in it.\n\nHow to apply a filter to your search:\n\n1. Select the field you want to search by\n1. Select how you want to search it (whether it contains a value, matches a regexp, is an exact value, etc)\n1. Enter the value to search by in the text box\n1. Click \"Ok\" or press enter in the text box\n\nOnce you apply a filter, the \"list\" and \"tree\" tabs will only include requests which match ALL of the active filters.\n\nIn addition, you can apply different filters for the key and value of key/value fields (such as headers or request parameters). This can be done by:\n\n1. Select a key/value field such as \"Rsp. Header\" or \"URL Param\"\n1. Click the \"+\" button on the right\n1. Enter the filter for the key on the left and the filter for the value on the right\n1. Click \"Ok\" or press enter in one of the text boxes\n\n![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_search_kv.png)\n\nAnd that's it! The filter tab has the following additional controls:\n\n1. Clear - Delete all active filters\n1. Pop - Delete the most recent filter\n1. Scope - Set the active search to your project's scope (see below)\n1. Save Scope - Set your project's scope to the currently active filters (see below)\n1. Apply a built-in filter dropdown - Guppy has a list of commonly useful filters. Select one from this list to apply it\n\n### Text Filter Entry\n\nAlong with the provided dropdowns you can manually type in a filter by clicking the `>` button. In some cases it may be faster to type your filter out rather than clicking on dropdowns. In addition it allows you to create filter statements that contain an `OR` and will pass a request that matches any one of the given filters. In fact, all the dropdown input does is generate these strings for you.\n\nMost filter strings have the following format:\n\n```\n \n```\n\nWhere `` is some part of the request/response, `` is some comparison to ``. For example, if you wanted a filter that only matches requests to `target.org`, you could use the following filter string:\n\n```\nhost is target.org\n\nfield = \"host\"\ncomparer = \"is\"\nvalue = \"target.org\"\n```\n\nFor fields that are a list of key/value pairs (headers, get params, post params, and cookies) you can use the following format:\n\n```\n [ ]\n```\n\nThis is a little more complicated. If you don't give comparer2/value2, the filter will pass any pair where the key or the value matches comparer1 and value1. If you do give comparer2/value2, the key must match comparer1/value1 and the value must match comparer2/value2 For example:\n\n```\nFilter A:\n cookie contains Session\n\nFilter B:\n cookie contains Session contains 456\n\nFilter C:\n inv cookie contains Ultra\n\nCookie: SuperSession=abc123\nMatches A and C but not B\n\nCookie: UltraSession=abc123456\nMatches both A and B but not C\n```\n\n#### List of fields\n\n| Field Name | Aliases | Description | Format |\n|:--------|:------------|:-----|:------|\n| all | all | Anywhere in the request, response, or a websocket message | String |\n| reqbody | reqbody, reqbd, qbd, qdata, qdt | The body of the request | String |\n| rspbody | rspbody, rspbd, sbd, sdata, sdt | The body of the response | String |\n| body | body, bd, data, dt | The body in either the request or the response | String |\n| wsmessage | wsmessage, wsm | In a websocket message | String |\n| method | method, verb, vb | The request method (GET, POST, etc) | String |\n| host | host, domain, hs, dm | The host that the request was sent to | String |\n| path | path, pt | The path of the request | String |\n| url | url | The full URL of the request | String |\n| statuscode | statuscode, sc | The status code of the response (200, 404, etc) | String |\n| tag | tag | Any of the tags of the request | String |\n| reqheader | reqheader, reqhd, qhd | A header in the request | Key/Value |\n| rspheader | rspheader, rsphd, shd | A header in the response | Key/Value |\n| header | header, hd | A header in the request or the response | Key/Value |\n| param | param, pm | Either a URL or a POST parameter | Key/Value |\n| urlparam | urlparam, uparam | A URL parameter of the request | Key/Value |\n| postparam | postparam, pparam | A post parameter of the request | Key/Value |\n| rspcookie | rspcookie, rspck, sck | A cookie set by the response | Key/Value |\n| reqcookie | reqcookie, reqck, qck | A cookie submitted by the request | Key/Value |\n| cookie | cookie, ck | A cookie sent by the request or a cookie set by the response | Key/Value |\n\n#### List of comparers\n\n| Field Name | Aliases | Description |\n|:--------|:------------|:-----|\n| is | is | Exact string match | \n| contains | contains, ct | A contain B is true if B is a substring of A |\n| containsr | containsr, ctr | A containr B is true if A matches regexp B |\n| leneq | leneq | A Leq B if A's length equals B (B must be a number) |\n| lengt | lengt | A Lgt B if A's length is greater than B (B must be a number ) |\n| lenlt | lenlt | A Llt B if A's length is less than B (B must be a number) |\n\n#### Special form filters\n\nA few filters don't conform to the field, comparer, value format. You can still negate these.\n\n| Format | Aliases | Description |\n|:--|:--|:--|\n| invert | invert, inv | Inverts a filter string. Anything that matches the filter string will not pass the filter. |\n\nExamples:\n\n```\nShow state-changing requests\n inv method is GET\n\nShow requests without a csrf parameter\n inv param ct csrf\n```\n\n#### Using OR\n\nIf you want to create a filter that will pass a request if it matches any of one of a few filters you can create `OR` statements. This is done by entering in each filter on the same line and separating them with an `OR` (It's case sensitive!).\n\nExamples:\n\n```\nShow requests to target.org or example.com:\n host is target.org OR host is example.com\n\nShow requests that either are to /foobar or have foobar in the response or is a 404\n path is /foobar OR sbd ct foobar OR sc is 404\n```\n\n### Scope\n\nThe scope of your project describes which requests should be recorded as they pass through the proxy. Guppy allows you to define a set of filters which describe which requests are in scope. For example, if your scope is just `host ctr example.com$` only requests to example.com will be recorded in history.\n\nTo set the scope of your project:\n\n1. Enter the filters you want to be your scope\n1. Press the \"Save Scope\" button\n\nAnd you're done! Requests that do not match this set of filters will no longer be saved. You can also set your current search to your scope by clicking the \"Scope\" button. The scope can be deleted by pressing the \"Clear\" button to delete all active filters and then clicking \"Save Scope\".\n\n# Repeater\n\n![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_repeater.png)\n\nThe repeater lets you repeatedly tweak and submit a request. You can use a request in the repeater by:\n\n1. Find the request you which to resubmit in the history list view\n1. Right click the request and click \"Send to Repeater\"\n1. Navigate to the repeater tab\n1. Edit the request on the left\n1. Click the submit button\n\nWhen you click submit:\n\n* The request will be submitted\n* The request and response will be saved in history\n* Any tags under the \"tag\" tab will be applied to the request\n\n# Interceptor\n\n![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_interceptor.png)\n\nThe interceptor lets you edit requests and responses as they pass through the proxy. To use this:\n\n1. Navigate to the interceptor tab\n1. Click the \"Int. Requests\" and/or the \"Int. Responses\" buttons\n1. Wait for a request to pass through the proxy\n1. Edit the message in the text box then click \"Forward\" to forward the edited message or click \"Cancel\" to just drop the message altogether\n\n# Decoder\n\n![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_decoder.png)\n\nThe decoder allows you to perform common encoding/decoding actions. You use the decoder by:\n\n1. Paste the data that you want to encode/decode\n1. Select how you wish to encode/decode it\n1. Press \"Go!\"\n\nThe text will be processed and it will appear in the same text box. Easy!\n\n# Macros\n\nGuppy includes support for loading and executing Python scripts in order to allow for more complex attacks than can be performed by hand with the repeater. It is worth noting that **this feature is not user friendly. Use it at your own risk.** No attempt is made to make this feature user-friendly, stable, or good. The main reason it exists is to make it easier to write python scripts which integrate with Guppy history and to provide some way to extend Guppy's features without a pull request. If you haven't been scared away yet, read on.\n\nThere are two types of macros that you can write:\n\n* Active macros: Take requests as an input, make more requests, edit the input requests, etc, then output a new set of requests for review\n* Intercepting macros: Modify requests and responses as they pass through the proxy\n\nMost features that you want will fall into one of those categories. Both macros are created by creating a `.py` file and defining specific functions which will be run when the macro is executed. For example an active macro must define `run_macro` and an intercepting macro must define `mangle_request` and/or `mangle_response`. See their respective sections below for more details.\n\n## The API\n\nUnfortunately since this feature was pretty much just thrown together for my own use, the documentation is looking at the source. Hopefully it will become more stable once Guppy development slows down, but for now you'll have to look at the relevant classes to figure out how to do stuff on your own. The following classes are the most important when writing a macro:\n\n### MacroClient\n\n`MacroClient` is defined in `(guppyproxy/macro.py)` and is the interface that macros use to submit requests, save requests to history, and produce output. At the time of writing the class provides:\n\n```\nMacroClient.submit(req, save=False): Submits a request to the server and sets req.response to the response. req is the HTTPRequest to submit. req.dest_host, req.dest_port, and req.use_tls will be used to determine the location to submit the request to\nMacroClient.save(req): Permenantly saves an HTTPRequest to history\nMacroClient.output(s): Prints a string to the output tab in the macros interface\nMacroClient.output_req(req): Adds a request to the output request table in the macros interface\nMacroClient.new_request(method=\"GET\", path=\"/\", proto_major=1, proto_minor=1,\n headers=None, body=bytes(), dest_host=\"\", dest_port=80,\n use_tls=False, tags=None): Creates a new HTTPRequest from scratch that can be submitted with the client\n```\n\n### HTTPRequest and HTTPResponse\n\n`HTTPRequest` and `HTTPResponse` are defined in `guppyproxy/proxy.py`. These classes represent HTTP messages. `HTTPRequest` contains both the contents of the message and information about its intended destination (host, port, whether to use TLS). Below are a few examples on how to use these classes, however for more deails you will need to consult `proxy.py`:\n\n```python\nreq = HTTPRequest()\nrsp = HTTPResponse()\n\nreq2 = req.copy() # Copy a request\nrsp2 = rsp.copy() # Copy a response\n\n# Refer to the messages associated with a messages\nrsp3 = req.response # Response to a request, will be `None` if there was no response\nunm = req.unmangled # Unmangled version of a request, is `None` if none exist\nunm2 = rsp.unmangled # Unmangled version of a response, is `None` if none exist\n\n# Get the full message of an object (is a bytes())\nfull_req = req.full_message()\nfull_rsp = rsp.full_message()\n\n# Get timing info for a request\ntstart = req.time_start # datetime.datetime when the request was made\ntend = req.time_end # datetime.datetime when the request's response was received\n\n# Get destination info from a request\ndest_host = req.dest_host\ndest_port = req.dest_port\nuse_tls = req.use_tls\n\n# Get/set the method of a request\nm = req.method # Get the method of the request\nreq.method = \"POST\" # Set the method of the request\n\n# Get/set url info of a request\nrequrl = req.full_url() # get the full URL of a request\npath = req.url.path # get the path of a request\nreq.url.path = \"/foo/bar/baz\" # set the path of a request\nv = req.url.get_param(\"foo\") # get the value of the \"foo\" URL parameter\nreq.url.set_param(\"foo\", \"bar\") # set the value of the \"foo\" URL parameter to \"bar\"\nreq.url.add_param(\"foo\", \"bar2\") # add a URL parameter allowing duplicates\nreq.url.del_param(\"foo\") # delete a url parameter\n[(k, v) for k, v in req.url.param_iter] # iterate over all the key/value pairs in the URL parameters\nfrag = req.url.fragment # get the fragment of the url (the bit after the #)\nreq.url.fragment = \"frag\" # set the url fragment of the request\n\n# Manage headers in a message\nreq.headers.set(\"Foo\", \"Bar\") # set a header, repalcing existing value\nhd = req.headers.get(\"Foo\") # get the value of a header (for duplicates, returns first value)\nreq.headers.add(\"Foo\", \"Bar2\") # add a header without replacing an existing one\npairs = req.headers.pairs() # returns all the key/value pairs of the headers in the message\nreq.headers.delete(\"Foo\") # delete a header\nreq.headers.dict() # Returns a dict of the headers in the form of {\"key1\": [\"val1\", \"val2\"], \"key2\": [\"val3\", \"val4\"]}\n# Same for responses\nrsp.headers.set(\"Foo\", \"Bar\")\nhd = rsp.headers.get(\"Foo\")\nrsp.headers.add(\"Foo\", \"Bar2\")\npairs = rsp.headers.pairs()\nrsp.headers.delete(\"Foo\")\nrsp.headers.dict()\n\n# Manage body of a message\nreq.body = \"foo=bar\" # set the body of the message to a string\nreq.body = b\"\\x01\\x02\\x03\" # set the body to bytes\nbd = req.body # Get the value of the body (always is bytes())\n# Same for responses\nrsp.body = \"foo=bar\"\nrsp.body = b\"\\x01\\x02\\x03\"\nbd = rsp.body\n\n# Manage POST parameters of a request\nparams = req.parameters() # Returns a dict of the POST parameters in the form of {\"key1\": [\"val1\", \"val2\"], \"key2\": [\"val3\", \"val4\"]}\n[(k, v) for k, v in req.param_iter()] # Iterate through all the key/value pairs of the request parameters\nreq.set_param(\"Foo\", \"Bar\") # Set the \"Foo\" parameter to \"Bar\"\nreq.add_param(\"Foo\", \"Bar2\") # Add a POST parameter to the request allowing duplicates\nreq.del_param(\"Foo\") # Delete a parameter from the request\n# NOTE: Setting a POST parameter will not change the request method to POST\n\n# Managing the cookies of a message\ncookie = req.cookies() # Returns an http.cookies.BaseCookie representing the request's cookies\nreq.set_cookie(\"foo\", \"bar\") # set a cookie in the request\nreq.del_cookie(\"foo\") # delete a cookie from the request\n[(k, v) for k, v in req.cookie_iter()] # Iterate over the key/value pairs of the cookies in a request\nreq.set_cookies({\"cookie1\": \"val1\", \"cookie2\": \"val2\"}) # Set the cookies in the request\nreq.set_cookies(req2) # Set the requests on req to the cookies in req2\nreq.add_cookies({\"cookie1\": \"val1\", \"cookie2\": \"val2\"}) # Add cookies to the request replacing existing values\nreq.add_cookies(req2) # Add cookies from req2 to the request replacing existing values\n# Same for responses\ncookie = rsp.cookies()\nrsp.set_cookie(\"foo\", \"bar\")\nrsp.del_cookie(\"foo\")\n[(k, v) for k, v in rsp.cookie_iter()]\nrsp.set_cookies({\"cookie1\": \"val1\", \"cookie2\": \"val2\"})\nrsp.set_cookies(rsp2)\nrsp.add_cookies({\"cookie1\": \"val1\", \"cookie2\": \"val2\"})\nrsp.add_cookies(rsp2)\n\n# Manage tags of a request\nhastag = (\"tagname\" in req.tags) # check if a request has a tag\nreq.tags.add(\"tagname\") # add a tag to a request\nreq.tags.remove(\"tagname\") # remove a tag from the request\n# NOTE: req.tags is a regular set() and you can do whatever you want to it\n```\n\n## Macro Arguments\n\n![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_macro_args.png)\n\nBoth active and intercepting macros can optionally have Guppy prompt for a set of arguments before running. These arguments will be passed as a dict in the `args` variable when calling the relevant function. A macro can request arguments by defining a `get_args` function and returning a list of strings. For example if a macro defines the following `get_args` function:\n\n```python\ndef get_args():\n return [\"foo\", \"bar\"]\n```\n\nthe proxy will prompt for values for foo and bar. If the user enters \"FOOARG\" and \"BARARG\" for the values `args` will have a value of:\n\n```python\n{\"foo\": \"FOOARG\", \"bar\": \"BARARG\"}\n```\n\nSee below for examples on how to use arguments in macros. If `get_args` is not defined, `None` will be passed in for `args`.\n\n## Active Macros\n\n![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_macro_active_in.png)\n\nActive macros are a Python script that define a `run_macro` function that takes in two arguments. A `MacroClient` (as defined in `guppyproxy/macros.py`) and a list of requests (`HTTPRequest` and `HTTPResponse` are defined in `guppyproxy/proxy.py`). The following is an example of a macro that resubmits all of the input requests but adds a new header:\n\n```python\n# addheader.py\n\ndef get_args():\n return [\"header_key\", \"header_val\"]\n\ndef run_macro(client, args, reqs):\n for req in reqs:\n client.output(\"Submitting request to %s...\" % req.full_url())\n req.headers.set(args[\"header_key\"], args[\"header_val\"])\n client.submit(req)\n client.output_req(req)\n```\n\nMacros such as this can be used for things such as testing auth controls or brute forcing paths/filenames.\n\n## Intercepting Macros\n\n![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_macro_int.png)\n\nIntercepting macros are used to look at/modify requests as they pass through the proxy. This is done by defining `mangle_request` and/or `mangle_response`:\n\n```\nmangle_request(client, req): Takes in a client and an HTTPRequest and returns an HTTPRequest. The returned HTTPRequest will be sent to the server instead of the original.\nmangle_response(client, req, rsp): Takes in a client, HTTPRequest, and HTTResponse and returns an HTTPResponse. The returned HTTPResponse will be sent to the browser instead of the original.\n```\n\nAs an example, the following macro will ask for a find/replace value. When run, it will set the `session` cookie in the request to `bar` before submitting it to the server and then perform the given find and replace on the body of the response.\n\n```python\n# intexample.py\n\ndef get_args():\n return [\"find\", \"replace\"]\n\ndef mangle_request(client, args, req):\n req.set_cookie(\"session\", \"bar\")\n return req\n\ndef mangle_response(client, args, req, rsp):\n rsp.body = rsp.body.replace(args['find'].encode(), args['replace'].encode())\n return rsp\n```\n\n# Settings\n\n![screenshot](https://github.com/roglew/guppy-static/blob/master/ss_settings.png)\n\nThis tab allows you to edit your proxy settings. It lets you select a file to store your history/settings in and configure what ports the proxy listens on. It also allows you to configure an upstream proxy to use. You can add a listener by entering the interface and port into the text boxes and clicking the \"+\" button. They can be deleted by selecting them from the list and clicking the \"-\" button.\n\nYou can also specify settings for an upstream proxy by checking the \"Use Proxy\" box, filling out the appropriate info, and clicking \"confirm\".\n\n## Data Files\n\nYour entire request history and your settings can be stored in a data file on disk. This allows you to save your work for later and even send your work to someone else. You can start a new project with a new data file by clicking the \"New\" button in the settings tab. Once you do this, your settings, scope, and all the messages that pass through the proxy will be saved to the specified file. You can also load an existing project by using the \"Open\" button. Finally, you can specify a data file by typing the path into the text box and clicking \"Go!\"\n\n# Keybindings\n\nGuppy has the following keybindings:\n\n| Key | Action |\n|:--------|:------------|\n| `Ctrl+J` | Navigate to request list |\n| `Ctrl+T` | Navigate to tree view |\n| `Ctrl+R` | Navigate to repeater |\n| `Ctrl+N` | Navigate to interceptor |\n| `Ctrl+D` | Navigate to decoder |\n| `Ctrl+U` | Navigate to filter text input |\n| `Ctrl+I` | Navigate to filter dropdown input |\n| `Ctrl+P` | Navigate to filters and pop most recent filter |\n| `Ctrl+Shift+D` | Navigate to decoder and fill with clipboard |\n| `Ctrl+Shift+N` | Create new datafile |\n| `Ctrl+Shift+O` | Open existing datafile |\n" }, { "path": "guppyproxy/__init__.py", "content": "" }, { "path": "guppyproxy/config.py", "content": "import copy\nimport json\n\ndefault_config = \"\"\"{\n \"listeners\": [\n {\"iface\": \"127.0.0.1\", \"port\": 8080}\n ],\n \"proxy\": {\"use_proxy\": false, \"host\": \"\", \"port\": 0, \"is_socks\": false}\n}\"\"\"\n\n\nclass ProxyConfig:\n PLUGIN_KEY = \"guppy_config\"\n\n def __init__(self):\n self._listeners = [('127.0.0.1', 8080, None)]\n self._proxy = {'use_proxy': False, 'host': '', 'port': 0, 'is_socks': False}\n\n def loads(self, js):\n config_info = json.loads(js)\n self._set_config(config_info)\n\n def dumps(self):\n listeners = []\n for l in self._listeners:\n listener = {\"host\": l[0], \"port\": l[1]}\n listeners.append(listener)\n _config_info = {\"listeners\": listeners,\n \"proxy\": self._proxy}\n return json.dumps(_config_info)\n\n def load(self, fname):\n try:\n with open(fname, 'r') as f:\n config_info = json.loads(f.read())\n except IOError:\n config_info = json.loads(default_config)\n with open(fname, 'w') as f:\n f.write(default_config)\n self._set_config(config_info)\n\n def _set_config(self, config_info):\n # Listeners\n if 'listeners' in config_info:\n self._parse_listeners(config_info['listeners'])\n\n if 'proxy' in config_info:\n self._proxy = config_info['proxy']\n\n def _parse_listeners(self, listeners):\n self._listeners = []\n for info in listeners:\n if 'port' in info:\n port = info['port']\n else:\n port = 8080\n\n if 'interface' in info:\n iface = info['interface']\n elif 'iface' in info:\n iface = info['iface']\n else:\n iface = '127.0.0.1'\n\n if \"transparent\" in info:\n trans_info = info['transparent']\n transparent_dest = (trans_info.get('host', \"\"),\n trans_info.get('port', 0),\n trans_info.get('use_tls', False))\n else:\n transparent_dest = None\n\n self._listeners.append((iface, port, transparent_dest))\n\n def set_listeners(self, listeners):\n self._listeners = listeners\n\n @property\n def listeners(self):\n return copy.deepcopy(self._listeners)\n\n @listeners.setter\n def listeners(self, val):\n self._parse_listeners(val)\n\n @property\n def proxy(self):\n # don't use this, use the getters to get the parsed values\n return self._proxy\n\n @proxy.setter\n def proxy(self, val):\n self._proxy = val\n\n @property\n def use_proxy(self):\n if self._proxy is None:\n return False\n if 'use_proxy' in self._proxy:\n if self._proxy['use_proxy']:\n return True\n return False\n\n @property\n def proxy_host(self):\n if self._proxy is None:\n return ''\n if 'host' in self._proxy:\n return self._proxy['host']\n return ''\n\n @property\n def proxy_port(self):\n if self._proxy is None:\n return ''\n if 'port' in self._proxy:\n return self._proxy['port']\n return ''\n\n @property\n def proxy_username(self):\n if self._proxy is None:\n return ''\n if 'username' in self._proxy:\n return self._proxy['username']\n return ''\n\n @property\n def proxy_password(self):\n if self._proxy is None:\n return ''\n if 'password' in self._proxy:\n return self._proxy['password']\n return ''\n\n @property\n def use_proxy_creds(self):\n return ('username' in self._proxy or 'password' in self._proxy)\n\n @property\n def is_socks_proxy(self):\n if self._proxy is None:\n return False\n if 'is_socks' in self._proxy:\n if self._proxy['is_socks']:\n return True\n return False\n" }, { "path": "guppyproxy/decoder.py", "content": "import html\nimport base64\nimport urllib\nimport json\n\nfrom guppyproxy.util import display_error_box\nfrom guppyproxy.hexteditor import ComboEditor\nfrom PyQt5.QtWidgets import QWidget, QHBoxLayout, QVBoxLayout, QComboBox, QPlainTextEdit, QPushButton\nfrom PyQt5.QtCore import pyqtSlot, pyqtSignal\nfrom datetime import datetime\n\nclass DecodeError(Exception):\n pass\n\ndef asciihex_encode_helper(s):\n return ''.join('{0:x}'.format(c) for c in s).encode()\n\n\ndef asciihex_decode_helper(s):\n ret = []\n try:\n for a, b in zip(s[0::2], s[1::2]):\n c = chr(a) + chr(b)\n ret.append(chr(int(c, 16)))\n return ''.join(ret).encode()\n except Exception as e:\n raise DecodeError(\"Unable to decode asciihex\")\n\n\ndef base64_decode_helper(s):\n s = s.decode()\n for i in range(0, 8):\n try:\n s_padded = base64.b64decode(s + '=' * i)\n return s_padded\n except Exception as e2:\n pass\n raise DecodeError(\"Unable to base64 decode string: %s\" % s)\n\n\ndef url_decode_helper(s):\n bs = s.decode()\n return urllib.parse.unquote(bs).encode()\n\n\ndef url_encode_helper(s):\n bs = s.decode()\n return urllib.parse.quote_plus(bs).encode()\n\n\ndef html_encode_helper(s):\n return ''.join(['&#x{0:x};'.format(c) for c in s]).encode()\n\n\ndef html_decode_helper(s):\n return html.unescape(s.decode()).encode()\n\n\ndef pp_json(s):\n d = json.loads(s.strip())\n return json.dumps(d, indent=2, sort_keys=True).encode()\n\ndef decode_jwt(s):\n # in case they paste the whole auth header or the token with \"bearer\"\n s = s.strip()\n fields = s.split(b' ')\n s = fields[-1].strip()\n parts = s.split(b'.')\n ret = b''\n for part in parts:\n try:\n ret += base64_decode_helper(part.decode()) + b'\\n\\n'\n except:\n ret += b\"[error decoding]\\n\\n\"\n return ret\n\ndef decode_unixtime(s):\n ts = int(s)\n dfmt = '%b %d, %Y %I:%M:%S %p'\n try:\n return datetime.utcfromtimestamp(ts).strftime(dfmt).encode()\n except ValueError:\n ts = ts/1000\n return datetime.utcfromtimestamp(ts).strftime(dfmt).encode()\n\nclass DecoderWidget(QWidget):\n\n def __init__(self):\n QWidget.__init__(self)\n layout = QVBoxLayout()\n\n self.decoder_input = DecoderInput()\n layout.addWidget(self.decoder_input)\n\n self.setLayout(layout)\n self.layout().setContentsMargins(0, 0, 0, 0)\n\n\nclass DecoderInput(QWidget):\n\n decodeRun = pyqtSignal(bytes)\n\n decoders = {\n \"encode_b64\": (\"Encode Base64\", base64.b64encode),\n \"decode_b64\": (\"Decode Base64\", base64_decode_helper),\n \"encode_ah\": (\"Encode Asciihex\", asciihex_encode_helper),\n \"decode_ah\": (\"Decode Asciihex\", asciihex_decode_helper),\n \"encode_url\": (\"URL Encode\", url_encode_helper),\n \"decode_url\": (\"URL Decode\", url_decode_helper),\n \"encode_html\": (\"HTML Encode\", html_encode_helper),\n \"decode_html\": (\"HTML Decode\", html_decode_helper),\n \"decode_unixtime\": (\"Format Unix Timestamp\", decode_unixtime),\n \"pp_json\": (\"Pretty-Print JSON\", pp_json),\n \"decode_jwt\": (\"Decode JWT Token\", decode_jwt),\n }\n\n def __init__(self, *args, **kwargs):\n QWidget.__init__(self)\n layout = QVBoxLayout()\n tool_layout = QHBoxLayout()\n\n self.editor = ComboEditor(pretty_tab=False, enable_pretty=False)\n self.encode_entry = QComboBox()\n encode_button = QPushButton(\"Go!\")\n\n encode_button.clicked.connect(self.encode)\n\n for k, v in self.decoders.items():\n self.encode_entry.addItem(v[0], k)\n\n layout.addWidget(self.editor)\n tool_layout.addWidget(self.encode_entry)\n tool_layout.addWidget(encode_button)\n tool_layout.addStretch()\n layout.addLayout(tool_layout)\n\n self.setLayout(layout)\n self.layout().setContentsMargins(0, 0, 0, 0)\n\n @pyqtSlot()\n def encode(self):\n text = self.editor.get_bytes()\n encode_type = self.encode_entry.itemData(self.encode_entry.currentIndex())\n encode_func = DecoderInput.decoders[encode_type][1]\n try:\n encoded = encode_func(text)\n except Exception as e:\n display_error_box(\"Error processing string:\\n\" + str(e))\n return\n self.editor.set_bytes(encoded)\n" }, { "path": "guppyproxy/gui.py", "content": "import random\n\nfrom guppyproxy.reqlist import ReqBrowser, ReqListModel\nfrom guppyproxy.repeater import RepeaterWidget\nfrom guppyproxy.interceptor import InterceptorWidget\nfrom guppyproxy.decoder import DecoderWidget\nfrom guppyproxy.settings import SettingsWidget\nfrom guppyproxy.shortcuts import GuppyShortcuts\nfrom guppyproxy.macros import MacroWidget\nfrom PyQt5.QtWidgets import QWidget, QTabWidget, QVBoxLayout, QTableView\nfrom PyQt5.QtCore import Qt, QTimer, QObject, pyqtSlot\n\n\nclass GuppyWindow(QWidget):\n titles = (\n \"Guppy Proxy\",\n )\n\n def __init__(self, client):\n QWidget.__init__(self)\n self.client = client\n \n self.delayTimeout = 100\n self._resizeTimer = QTimer(self)\n self._resizeTimer.timeout.connect(self._delayedUpdate)\n \n self.setFocusPolicy(Qt.StrongFocus)\n self.shortcuts = GuppyShortcuts(self)\n self.tabWidget = QTabWidget()\n self.repeaterWidget = RepeaterWidget(self.client)\n self.interceptorWidget = InterceptorWidget(self.client)\n self.macroWidget = MacroWidget(self.client)\n self.historyWidget = ReqBrowser(self.client,\n repeater_widget=self.repeaterWidget,\n macro_widget=self.macroWidget,\n is_client_context=True,\n update=True)\n self.decoderWidget = DecoderWidget()\n self.settingsWidget = SettingsWidget(self.client)\n\n self.settingsWidget.datafileLoaded.connect(self.historyWidget.reset_to_scope)\n \n self.history_ind = self.tabWidget.count()\n self.tabWidget.addTab(self.historyWidget, \"History\")\n self.repeater_ind = self.tabWidget.count()\n self.tabWidget.addTab(self.repeaterWidget, \"Repeater\")\n self.interceptor_ind = self.tabWidget.count()\n self.tabWidget.addTab(self.interceptorWidget, \"Interceptor\")\n self.decoder_ind = self.tabWidget.count()\n self.tabWidget.addTab(self.decoderWidget, \"Decoder\")\n self.macro_ind = self.tabWidget.count()\n self.tabWidget.addTab(self.macroWidget, \"Macros\")\n self.settings_ind = self.tabWidget.count()\n self.tabWidget.addTab(self.settingsWidget, \"Settings\")\n\n self.mainLayout = QVBoxLayout()\n self.mainLayout.addWidget(self.tabWidget)\n self.mainWidget = QWidget()\n self.mainWidget.setLayout(self.mainLayout)\n \n self.wrapperLayout = QVBoxLayout()\n self.wrapperLayout.addWidget(self.mainWidget)\n self.wrapperLayout.setContentsMargins(0, 0, 0, 0)\n self.setLayout(self.wrapperLayout)\n\n self.setWindowTitle(random.choice(GuppyWindow.titles))\n self.show()\n \n def show_hist_tab(self):\n self.tabWidget.setCurrentIndex(self.history_ind)\n\n def show_repeater_tab(self):\n self.tabWidget.setCurrentIndex(self.repeater_ind)\n \n def show_interceptor_tab(self):\n self.tabWidget.setCurrentIndex(self.interceptor_ind)\n\n def show_decoder_tab(self):\n self.tabWidget.setCurrentIndex(self.decoder_ind)\n \n def show_active_macro_tab(self):\n self.tabWidget.setCurrentIndex(self.macro_ind)\n self.macroWidget.show_active()\n\n def show_int_macro_tab(self):\n self.tabWidget.setCurrentIndex(self.macro_ind)\n self.macroWidget.show_int()\n \n def resizeEvent(self, event):\n QWidget.resizeEvent(self, event)\n self._resizeTimer.stop()\n self._resizeTimer.start(self.delayTimeout)\n self.mainWidget.setVisible(False)\n\n @pyqtSlot()\n def _delayedUpdate(self):\n self._resizeTimer.stop()\n self.mainWidget.setVisible(True)\n\n def close(self):\n self.interceptorWidget.close()\n \n" }, { "path": "guppyproxy/gup.py", "content": "import argparse\nimport sys\nimport os\n\nfrom PyQt5.QtWidgets import QApplication\nfrom PyQt5.QtCore import Qt\nfrom guppyproxy.gui import GuppyWindow\nfrom guppyproxy.proxy import ProxyClient, MessageError, ProxyThread\nfrom guppyproxy.util import confirm, set_running_as_app\nfrom guppyproxy.macros import MacroClient\n\n\ndef load_certificates(client, path):\n client.load_certificates(os.path.join(path, \"server.pem\"),\n os.path.join(path, \"server.key\"))\n\n\ndef generate_certificates(client, path):\n try:\n os.makedirs(path, 0o755)\n except os.error as e:\n if not os.path.isdir(path):\n raise e\n pkey_file = os.path.join(path, 'server.key')\n cert_file = os.path.join(path, 'server.pem')\n client.generate_certificates(pkey_file, cert_file)\n\n\ndef main():\n parser = argparse.ArgumentParser(description=\"Guppy debug flags. Don't worry about most of these\")\n parser.add_argument(\"--binary\", nargs=1, help=\"location of the backend binary\")\n parser.add_argument(\"--attach\", nargs=1, help=\"attach to an already running backend\")\n parser.add_argument(\"--dbgattach\", nargs=1, help=\"attach to an already running backend and also perform setup\")\n parser.add_argument('--debug', help='run in debug mode', action='store_true')\n parser.add_argument('--dog', help='dog', action='store_true')\n args = parser.parse_args()\n\n if args.binary is not None and args.attach is not None:\n print(\"Cannot provide both a binary location and an address to connect to\")\n exit(1)\n\n data_dir = os.path.join(os.path.expanduser('~'), '.guppy')\n\n if args.binary is not None:\n binloc = args.binary[0]\n msg_addr = None\n elif args.attach is not None or args.dbgattach:\n binloc = None\n if args.attach is not None:\n msg_addr = args.attach[0]\n if args.dbgattach is not None:\n msg_addr = args.dbgattach[0]\n else:\n msg_addr = None\n binloc = os.path.join(data_dir, \"puppy\")\n if 'RESOURCEPATH' in os.environ:\n rpath = os.environ['RESOURCEPATH']\n checkloc = os.path.join(rpath, 'puppyrsc', 'puppy.osx')\n if os.path.exists(checkloc):\n set_running_as_app(True)\n binloc = checkloc\n if not os.path.exists(binloc):\n print(\"Could not find puppy binary. Please ensure that it has been compiled and placed in ~/.guppy/, or pass in the binary location from the command line\")\n exit(1)\n\n cert_dir = os.path.join(data_dir, \"certs\")\n\n with ProxyClient(binary=binloc, conn_addr=msg_addr, debug=args.debug) as client:\n try:\n load_certificates(client, cert_dir)\n except MessageError as e:\n generate_certificates(client, cert_dir)\n print(\"Certificates generated to {}\".format(cert_dir))\n print(\"Be sure to add {} to your trusted CAs in your browser!\".format(os.path.join(cert_dir, \"server.pem\")))\n load_certificates(client, cert_dir)\n try:\n # Only try and listen/set default storage if we're not attaching\n if args.attach is None:\n storage = client.add_in_memory_storage(\"\")\n client.disk_storage = storage\n client.inmem_storage = client.add_in_memory_storage(\"m\")\n client.set_proxy_storage(storage.storage_id)\n\n app = QApplication(sys.argv)\n window = GuppyWindow(client)\n try:\n app.exec_()\n finally:\n window.close()\n except MessageError as e:\n print(str(e))\n MacroClient._ded = True # pray this kills the threads\n ProxyThread.waitall()\n\n\ndef start():\n main()\n\n\nif __name__ == '__main__':\n main()\n" }, { "path": "guppyproxy/hexteditor.py", "content": "import base64\nfrom guppyproxy.util import printable_data, qtprintable, textedit_highlight, DisableUpdates\nfrom guppyproxy.proxy import _parse_message, Headers\nfrom itertools import count\nfrom PyQt5.QtWidgets import QWidget, QTextEdit, QTableWidget, QVBoxLayout, QTableWidgetItem, QTabWidget, QStackedLayout, QLabel, QComboBox\nfrom PyQt5.QtGui import QTextCursor, QTextCharFormat, QImage, QColor, QTextImageFormat, QTextDocument, QTextDocumentFragment, QTextBlockFormat\nfrom PyQt5.QtCore import Qt, pyqtSlot, QUrl\nfrom pygments import highlight\nfrom pygments.formatters import HtmlFormatter\nfrom pygments.lexers import get_lexer_for_mimetype, TextLexer\nfrom pygments.lexers.data import JsonLexer\nfrom pygments.lexers.html import HtmlLexer\nfrom pygments.styles import get_style_by_name\n\n\nclass PrettyPrintWidget(QWidget):\n VIEW_NONE = 0\n VIEW_HIGHLIGHTED = 1\n VIEW_JSON = 2\n VIEW_HTMLXML = 3\n def __init__(self, *args, **kwargs):\n QWidget.__init__(self, *args, **kwargs)\n self.headers = Headers()\n self.data = b''\n self.view = 0\n self.setLayout(QVBoxLayout())\n self.layout().setContentsMargins(0, 0, 0, 0)\n\n self.stack = QStackedLayout()\n self.stack.setContentsMargins(0, 0, 0, 0)\n self.nopp_widg = QLabel(\"No pretty version available\")\n self.stack.addWidget(self.nopp_widg)\n self.highlighted_widg = QTextEdit()\n self.highlighted_widg.setReadOnly(True)\n self.stack.addWidget(self.highlighted_widg)\n self.json_widg = QTextEdit()\n self.json_widg.setReadOnly(True)\n self.stack.addWidget(self.json_widg)\n self.htmlxml_widg = QTextEdit()\n self.htmlxml_widg.setReadOnly(True)\n self.stack.addWidget(self.htmlxml_widg)\n\n self.selector = QComboBox()\n self.selector.addItem(\"Manually Select Printer\", self.VIEW_NONE)\n self.selector.addItem(\"Highlighted\", self.VIEW_HIGHLIGHTED)\n self.selector.addItem(\"JSON\", self.VIEW_JSON)\n self.selector.addItem(\"HTML/XML\", self.VIEW_HTMLXML)\n self.selector.currentIndexChanged.connect(self._combo_changed)\n \n self.layout().addWidget(self.selector)\n self.layout().addLayout(self.stack)\n \n def guess_format(self):\n if 'Content-Type' in self.headers:\n ct = self.headers.get('Content-Type').lower()\n if 'json' in ct:\n self.set_view(self.VIEW_JSON)\n elif 'html' in ct or 'xml' in ct:\n self.set_view(self.VIEW_HTMLXML)\n else:\n self.set_view(self.VIEW_HIGHLIGHTED)\n else:\n self.set_view(self.VIEW_NONE)\n \n @pyqtSlot()\n def _combo_changed(self):\n field = self.selector.itemData(self.selector.currentIndex())\n old = self.selector.blockSignals(True)\n self.set_view(field)\n self.selector.blockSignals(old)\n\n def set_view(self, view):\n if view == self.VIEW_NONE:\n self.clear_output()\n self.stack.setCurrentIndex(self.VIEW_NONE)\n elif view == self.VIEW_JSON:\n self.clear_output()\n self.fill_json()\n self.stack.setCurrentIndex(self.VIEW_JSON)\n elif view == self.VIEW_HTMLXML:\n self.clear_output()\n self.fill_htmlxml()\n self.stack.setCurrentIndex(self.VIEW_HTMLXML)\n elif view == self.VIEW_HIGHLIGHTED:\n self.clear_output()\n self.fill_highlighted()\n self.stack.setCurrentIndex(self.VIEW_HIGHLIGHTED)\n else:\n return\n self.selector.setCurrentIndex(view)\n self.view = view\n \n def clear_output(self):\n self.json_widg.setPlainText(\"\")\n self.htmlxml_widg.setPlainText(\"\")\n \n def set_bytes(self, bs):\n self.clear_output()\n self.headers = Headers()\n self.data = b''\n if not bs:\n return\n _, h, body = _parse_message(bs, lambda x: None)\n self.headers = h\n self.data = body\n \n def fill_json(self):\n from .decoder import pp_json\n with DisableUpdates(self.json_widg):\n self.json_widg.setPlainText(\"\")\n if not self.data:\n return\n try:\n j = pp_json(self.data.decode())\n except Exception:\n return\n highlighted = textedit_highlight(j, JsonLexer())\n self.json_widg.setHtml(highlighted)\n \n def fill_htmlxml(self):\n from lxml import etree, html\n\n with DisableUpdates(self.htmlxml_widg):\n self.htmlxml_widg.setPlainText(\"\")\n if not self.data:\n return\n try:\n fragments = html.fragments_fromstring(self.data.decode())\n parsed_frags = []\n for f in fragments:\n parsed_frags.append(etree.tostring(f, pretty_print=True))\n pretty = b''.join(parsed_frags)\n except Exception:\n return\n highlighted = textedit_highlight(pretty, HtmlLexer())\n self.htmlxml_widg.setHtml(highlighted)\n \n def fill_highlighted(self):\n with DisableUpdates(self.htmlxml_widg):\n self.highlighted_widg.setPlainText(\"\")\n if not self.data:\n return\n ct = self.headers.get('Content-Type').lower()\n if \";\" in ct:\n ct = ct.split(\";\")[0]\n try:\n lexer = get_lexer_for_mimetype(ct)\n highlighted = textedit_highlight(self.data, lexer)\n except:\n highlighted = printable_data(self.data)\n self.highlighted_widg.setHtml(highlighted)\n\n\nclass HextEditor(QWidget):\n byte_image = QImage()\n byte_image.loadFromData(base64.b64decode(\"iVBORw0KGgoAAAANSUhEUgAAAAgAAAAICAYAAADED76LAAAAQklEQVQYlYWPMQoAMAgD82jpSzL613RpqG2FBly8QwywQlJ1UENSipAilIAS2FKFziFZ8LIVOjg6ocJx/+ELD/zVnJcqe5vHUAJgAAAAAElFTkSuQmCC\"))\n byte_url = \"data://byte.png\"\n byte_property = 0x100000 + 1\n nonce_property = 0x100000 + 2\n byte_nonce = count()\n\n def __init__(self, enable_pretty=True):\n QWidget.__init__(self)\n layout = QVBoxLayout()\n self.enable_pretty = enable_pretty\n self.setLayout(layout)\n self.layout().setSpacing(0)\n self.layout().setContentsMargins(0, 0, 0, 0)\n self.lexer = TextLexer()\n\n self.textedit = QTextEdit()\n self.textedit.setAcceptRichText(False)\n doc = self.textedit.document()\n font = doc.defaultFont()\n font.setFamily(\"Courier New\")\n font.setPointSize(10)\n doc.setDefaultFont(font)\n doc.addResource(QTextDocument.ImageResource,\n QUrl(self.byte_url),\n HextEditor.byte_image)\n self.textedit.focusInEvent = self.focus_in_event\n self.textedit.focusOutEvent = self.focus_left_event\n self.data = b''\n\n self.pretty_mode = False\n self.layout().addWidget(self.textedit)\n\n def focus_in_event(self, e):\n QTextEdit.focusInEvent(self.textedit, e)\n if not self.textedit.isReadOnly():\n self.set_bytes(self.data)\n self.pretty_mode = False\n\n def focus_left_event(self, e):\n QTextEdit.focusOutEvent(self.textedit, e)\n if not self.textedit.isReadOnly():\n self.data = self.get_bytes()\n self.set_bytes_highlighted(self.data)\n self.pretty_mode = True\n\n def setReadOnly(self, ro):\n self.textedit.setReadOnly(ro)\n\n def _insert_byte(self, cursor, b):\n f = QTextImageFormat()\n f2 = QTextCursor().charFormat()\n cursor.document().addResource(QTextDocument.ImageResource,\n QUrl(self.byte_url),\n HextEditor.byte_image)\n f.setName(self.byte_url)\n f.setProperty(HextEditor.byte_property, b + 1)\n f.setProperty(HextEditor.nonce_property, next(self.byte_nonce))\n cursor.insertImage(f)\n cursor.setCharFormat(QTextCursor().charFormat())\n\n def clear(self):\n self.textedit.setPlainText(\"\")\n\n def set_lexer(self, lexer):\n self.lexer = lexer\n\n def set_bytes(self, bs):\n with DisableUpdates(self.textedit):\n self.pretty_mode = False\n self.data = bs\n chunks = HextEditor._split_by_printables(bs)\n self.clear()\n cursor = QTextCursor(self.textedit.document())\n cursor.beginEditBlock()\n try:\n cursor.select(QTextCursor.Document)\n cursor.setCharFormat(QTextCharFormat())\n cursor.clearSelection()\n for chunk in chunks:\n if chr(chunk[0]) in qtprintable:\n cursor.insertText(chunk.decode())\n else:\n for b in chunk:\n self._insert_byte(cursor, b)\n finally:\n cursor.endEditBlock()\n self.repaint() # needed to fix issue with py2app\n\n def set_bytes_highlighted(self, bs, lexer=None):\n if not self.enable_pretty:\n self.set_bytes(bs)\n return\n with DisableUpdates(self.textedit):\n self.pretty_mode = True\n self.clear()\n self.data = bs\n if lexer:\n self.lexer = lexer\n printable = printable_data(bs)\n highlighted = textedit_highlight(printable, self.lexer)\n self.textedit.setHtml(highlighted)\n self.repaint() # needed to fix issue with py2app\n\n def get_bytes(self):\n if not self.pretty_mode:\n self.data = self._get_bytes()\n return self.data\n\n def _get_bytes(self):\n from .util import hexdump\n bs = bytearray()\n block = self.textedit.document().firstBlock()\n newline = False\n while block.length() > 0:\n if newline:\n bs.append(ord('\\n'))\n newline = True\n it = block.begin()\n while not it.atEnd():\n f = it.fragment()\n fmt = f.charFormat()\n byte = fmt.intProperty(HextEditor.byte_property)\n if byte > 0:\n text = f.text().encode()\n if text == b\"\\xef\\xbf\\xbc\":\n bs.append(byte - 1)\n else:\n bs += text\n else:\n text = f.text()\n bs += text.encode()\n it += 1\n block = block.next()\n return bytes(bs)\n\n @classmethod\n def _split_by_printables(cls, bs):\n if len(bs) == 0:\n return []\n\n def is_printable(c):\n return c in qtprintable\n\n chunks = []\n printable = is_printable(chr(bs[0]))\n a = 0\n b = 1\n while b < len(bs):\n if is_printable(chr(bs[b])) != printable:\n chunks.append(bs[a:b])\n a = b\n printable = not printable\n b += 1\n chunks.append(bs[a:b])\n return chunks\n\n\nclass HexEditor(QWidget):\n\n def __init__(self):\n QWidget.__init__(self)\n self.setLayout(QVBoxLayout())\n self.layout().setContentsMargins(0, 0, 0, 0)\n self.layout().setSpacing(0)\n\n self.data = bytearray()\n self.datatable = QTableWidget()\n self.datatable.cellChanged.connect(self._cell_changed)\n self.datatable.horizontalHeader().setStretchLastSection(True)\n self.row_size = 16\n self.read_only = False\n self.redraw_table()\n self.layout().addWidget(self.datatable)\n\n def set_bytes(self, bs):\n self.data = bytearray(bs)\n self.redraw_table()\n\n def get_bytes(self):\n return bytes(self.data)\n\n def setReadOnly(self, ro):\n self.read_only = ro\n self.redraw_table()\n\n def _redraw_strcol(self, row):\n start = self.row_size * row\n end = start + self.row_size\n data = self.data[start:end]\n print_data = printable_data(data, include_newline=False)\n item = QTableWidgetItem(print_data)\n item.setFlags(item.flags() ^ Qt.ItemIsEditable)\n self.datatable.setItem(row, self.str_col, item)\n\n def redraw_table(self, length=None):\n with DisableUpdates(self.datatable):\n oldsig = self.datatable.blockSignals(True)\n self.row_size = length or self.row_size\n self.datatable.setColumnCount(self.row_size + 1)\n self.datatable.setRowCount(0)\n self.str_col = self.row_size\n\n self.datatable.horizontalHeader().hide()\n self.datatable.verticalHeader().hide()\n\n rows = int(len(self.data) / self.row_size)\n if len(self.data) % self.row_size > 0:\n rows += 1\n self.datatable.setRowCount(rows)\n\n for i in range(rows * self.row_size):\n row = i / self.row_size\n col = i % self.row_size\n if i < len(self.data):\n dataval = \"%02x\" % self.data[i]\n item = QTableWidgetItem(dataval)\n if self.read_only:\n item.setFlags(item.flags() ^ Qt.ItemIsEditable)\n else:\n item = QTableWidgetItem(\"\")\n item.setFlags(item.flags() ^ Qt.ItemIsEditable)\n self.datatable.setItem(row, col, item)\n\n for row in range(rows):\n self._redraw_strcol(row)\n self.datatable.blockSignals(oldsig)\n self.datatable.resizeColumnsToContents()\n self.datatable.resizeRowsToContents()\n\n @classmethod\n def _format_hex(cls, n):\n return (\"%02x\" % n).upper()\n\n @pyqtSlot(int, int)\n def _cell_changed(self, row, col):\n oldsig = self.datatable.blockSignals(True)\n if col == self.str_col:\n return\n if len(self.data) == 0:\n return\n\n data_ind = self.row_size * row + col\n if data_ind >= len(self.data):\n return\n\n data_text = self.datatable.item(row, col).text()\n try:\n data_val = int(data_text, 16)\n if data_val < 0x0 or data_val > 0xff:\n raise Exception()\n except Exception as e:\n item = QTableWidgetItem(self._format_hex(self.data[data_ind]))\n self.datatable.setItem(row, col, item)\n self.datatable.blockSignals(oldsig)\n return\n\n if data_text != self._format_hex(data_val):\n self.datatable.setItem(row, col, QTableWidgetItem(self._format_hex(data_val)))\n\n self.data[data_ind] = data_val\n self._redraw_strcol(row)\n self.datatable.blockSignals(oldsig)\n\n\nclass ComboEditor(QWidget):\n def __init__(self, pretty_tab=True, enable_pretty=True):\n QWidget.__init__(self)\n self.setLayout(QVBoxLayout())\n self.layout().setSpacing(0)\n self.layout().setContentsMargins(0, 0, 0, 0)\n\n self.data = b''\n self.enable_pretty = enable_pretty\n\n self.tabWidget = QTabWidget()\n self.hexteditor = HextEditor(enable_pretty=self.enable_pretty)\n self.hexeditor = HexEditor()\n self.ppwidg = PrettyPrintWidget()\n\n self.hexteditor_ind = self.tabWidget.count()\n self.tabWidget.addTab(self.hexteditor, \"Text\")\n self.hexeditor_ind = self.tabWidget.count()\n self.tabWidget.addTab(self.hexeditor, \"Hex\")\n self.pp_ind = -1\n if pretty_tab:\n self.pp_ind = self.tabWidget.count()\n self.tabWidget.addTab(self.ppwidg, \"Pretty\")\n self.tabWidget.currentChanged.connect(self._tab_changed)\n\n self.previous_tab = self.tabWidget.currentIndex()\n\n self.layout().addWidget(self.tabWidget)\n\n\n @pyqtSlot(int)\n def _tab_changed(self, i):\n # commit data from old tab\n if self.previous_tab == self.hexteditor_ind:\n self.data = self.hexteditor.get_bytes()\n if self.previous_tab == self.hexeditor_ind:\n self.data = self.hexeditor.get_bytes()\n\n # set up new tab\n if i == self.hexteditor_ind:\n if self.hexteditor.pretty_mode:\n self.hexteditor.set_bytes_highlighted(self.data)\n else:\n self.hexteditor.set_bytes(self.data)\n if i == self.hexeditor_ind:\n self.hexeditor.set_bytes(self.data)\n if i == self.pp_ind:\n self.ppwidg.set_bytes(self.data)\n self.ppwidg.guess_format()\n\n # update previous tab\n self.previous_tab = self.tabWidget.currentIndex()\n \n\n @pyqtSlot(bytes)\n def set_bytes(self, bs):\n self.data = bs\n self.tabWidget.setCurrentIndex(0)\n if self.tabWidget.currentIndex() == self.hexteditor_ind:\n self.hexteditor.set_bytes(bs)\n elif self.tabWidget.currentIndex() == self.hexeditor_ind:\n self.hexeditor.set_bytes(bs)\n elif self.tabWidget.currentIndex() == self.pp_ind:\n self.ppwidg.set_bytes(bs)\n\n\n @pyqtSlot(bytes)\n def set_bytes_highlighted(self, bs, lexer=None):\n self.data = bs\n self.tabWidget.setCurrentIndex(0)\n if self.enable_pretty:\n self.hexteditor.set_bytes_highlighted(bs, lexer=lexer)\n else:\n self.set_bytes(bs)\n\n def get_bytes(self):\n if self.tabWidget.currentIndex() == self.hexteditor_ind:\n self.data = self.hexteditor.get_bytes()\n elif self.tabWidget.currentIndex() == self.hexeditor_ind:\n self.data = self.hexeditor.get_bytes()\n return self.data\n\n def setReadOnly(self, ro):\n self.hexteditor.setReadOnly(ro)\n self.hexeditor.setReadOnly(ro)\n" }, { "path": "guppyproxy/interceptor.py", "content": "from guppyproxy.util import display_error_box\nfrom guppyproxy.proxy import InterceptMacro, parse_request, parse_response\nfrom guppyproxy.hexteditor import ComboEditor\nfrom PyQt5.QtWidgets import QWidget, QVBoxLayout, QHBoxLayout, QPushButton\nfrom PyQt5.QtCore import pyqtSlot, pyqtSignal, QObject\n\nimport threading\n\nedit_queue = []\n\n\nclass InterceptEvent:\n\n def __init__(self):\n self.e = threading.Event()\n self.canceled = False\n self.message = None\n\n def wait(self):\n self.e.wait()\n return self.message\n\n def set(self, message):\n self.message = message\n self.e.set()\n\n def cancel(self):\n self.canceled = True\n self.set(None)\n\n\nclass InterceptedMessage:\n\n def __init__(self, request=None, response=None, wsmessage=None):\n self.request = request\n self.response = response\n self.wsmessage = wsmessage\n self.event = InterceptEvent()\n\n self.message_type = None\n if self.request:\n self.message_type = \"request\"\n elif self.response:\n self.message_type = \"response\"\n elif self.wsmessage:\n self.message_type = \"wsmessage\"\n\n\nclass InterceptorMacro(InterceptMacro, QObject):\n \"\"\"\n A class representing a macro that modifies requests as they pass through the\n proxy\n \"\"\"\n\n messageReceived = pyqtSignal(InterceptedMessage)\n\n def __init__(self, int_widget):\n InterceptMacro.__init__(self)\n QObject.__init__(self)\n self.int_widget = int_widget\n self.messageReceived.connect(self.int_widget.message_received)\n self.name = \"InterceptorMacro\"\n\n def mangle_request(self, request):\n int_msg = InterceptedMessage(request=request)\n self.messageReceived.emit(int_msg)\n req = int_msg.event.wait()\n if int_msg.event.canceled:\n return request\n req.dest_host = request.dest_host\n req.dest_port = request.dest_port\n req.use_tls = request.use_tls\n return req\n\n def mangle_response(self, request, response):\n int_msg = InterceptedMessage(response=response)\n self.messageReceived.emit(int_msg)\n rsp = int_msg.event.wait()\n if int_msg.event.canceled:\n return response\n return rsp\n\n def mangle_websocket(self, request, response, message):\n # just don't do this right now\n pass\n\n\nclass InterceptorWidget(QWidget):\n def __init__(self, client):\n QWidget.__init__(self)\n self.client = client\n self.int_conn = None\n self.queued_messages = []\n self.editing_message = None\n self.editing = False\n\n self.int_req = False\n self.int_rsp = False\n self.int_ws = False\n\n # layouts\n self.setLayout(QVBoxLayout())\n self.layout().setSpacing(0)\n self.layout().setContentsMargins(0, 0, 0, 0)\n\n buttons = QHBoxLayout()\n buttons.setContentsMargins(0, 0, 0, 0)\n buttons.setSpacing(10)\n\n # widgets\n intReqButton = QPushButton(\"Int. Requests\")\n intRspButton = QPushButton(\"Int. Responses\")\n intWsButton = QPushButton(\"Int. Websocket\")\n forwardButton = QPushButton(\"Forward\")\n cancelButton = QPushButton(\"Cancel\")\n self.editor = ComboEditor()\n\n intReqButton.setCheckable(True)\n intRspButton.setCheckable(True)\n intWsButton.setCheckable(True)\n\n intWsButton.setEnabled(False)\n\n forwardButton.clicked.connect(self.forward_message)\n cancelButton.clicked.connect(self.cancel_edit)\n intReqButton.toggled.connect(self.int_req_toggled)\n intRspButton.toggled.connect(self.int_rsp_toggled)\n intWsButton.toggled.connect(self.int_ws_toggled)\n\n buttons.addWidget(forwardButton)\n buttons.addWidget(cancelButton)\n buttons.addWidget(intReqButton)\n buttons.addWidget(intRspButton)\n buttons.addWidget(intWsButton)\n # checkbox for req/rsp/ws\n self.layout().addLayout(buttons)\n self.layout().addWidget(self.editor)\n\n @pyqtSlot(bool)\n def int_req_toggled(self, state):\n self.int_req = state\n self.restart_intercept()\n\n @pyqtSlot(bool)\n def int_rsp_toggled(self, state):\n self.int_rsp = state\n self.restart_intercept()\n\n @pyqtSlot(bool)\n def int_ws_toggled(self, state):\n self.int_ws = state\n self.restart_intercept()\n\n @pyqtSlot(InterceptedMessage)\n def message_received(self, msg):\n self.queued_messages.append(msg)\n # Update queue list\n self.edit_next_message()\n\n def set_edited_message(self, msg):\n if msg.message_type == \"request\":\n self.editor.set_bytes(msg.request.full_message())\n elif msg.message_type == \"response\":\n self.editor.set_bytes(msg.response.full_message())\n elif msg.message_type == \"wsmessage\":\n # this is not gonna work\n self.editor.set_bytes(msg.wsmessage.message)\n\n def edit_next_message(self):\n if self.editing:\n return\n self.editor.set_bytes(b\"\")\n if not self.queued_messages:\n return\n self.editing_message = self.queued_messages.pop()\n self.set_edited_message(self.editing_message)\n self.editing = True\n\n @pyqtSlot()\n def forward_message(self):\n if not self.editing:\n return\n if self.editing_message.message_type == \"request\":\n try:\n req = parse_request(self.editor.get_bytes())\n except Exception:\n display_error_box(\"Could not parse request\")\n return\n self.editing_message.event.set(req)\n elif self.editing_message.message_type == \"response\":\n try:\n rsp = parse_response(self.editor.get_bytes())\n except Exception:\n display_error_box(\"Could not parse response\")\n return\n self.editing_message.event.set(rsp)\n elif self.editing_message.message_type == \"wsmessage\":\n pass\n self.editing = False\n self.edit_next_message()\n\n @pyqtSlot()\n def cancel_edit(self):\n if self.editing_message:\n self.editing_message.event.cancel()\n self.editing = False\n self.edit_next_message()\n\n def clear_edit_queue(self):\n while self.queued_messages or self.editing_message:\n if self.editing_message:\n self.editing_message.event.cancel()\n self.editing_message = False\n if self.queued_messages:\n self.editing_message = self.queued_messages.pop()\n\n def restart_intercept(self):\n self.close()\n self.editor.set_bytes(\"\")\n self.editing = False\n\n if not (self.int_req or self.int_rsp or self.int_ws):\n return\n\n mangle_macro = InterceptorMacro(self)\n mangle_macro.intercept_requests = self.int_req\n mangle_macro.intercept_responses = self.int_rsp\n mangle_macro.intercept_ws = self.int_ws\n self.int_conn = self.client.new_conn()\n self.int_conn.intercept(mangle_macro)\n\n def close(self):\n if self.int_conn:\n self.int_conn.close()\n self.int_conn = None\n self.clear_edit_queue()\n" }, { "path": "guppyproxy/macros.py", "content": "import glob\nimport imp\nimport os\nimport random\nimport re\nimport stat\nimport sys\nimport traceback\n\nfrom guppyproxy.proxy import InterceptMacro, HTTPRequest, ProxyThread\nfrom guppyproxy.util import display_error_box, qtprintable, set_default_dialog_dir, default_dialog_dir, open_dialog, save_dialog, display_info_box\n\nfrom collections import namedtuple\nfrom itertools import count\nfrom PyQt5.QtWidgets import QWidget, QTableWidget, QTableWidgetItem, QGridLayout, QHeaderView, QAbstractItemView, QVBoxLayout, QHBoxLayout, QComboBox, QTabWidget, QPushButton, QLineEdit, QStackedLayout, QToolButton, QCheckBox, QLabel, QTableView, QPlainTextEdit, QFormLayout, QSizePolicy, QDialog, QPlainTextEdit, QTextEdit\nfrom PyQt5.QtCore import pyqtSlot, pyqtSignal, QObject, QVariant, Qt, QAbstractTableModel, QModelIndex, QItemSelection, QSortFilterProxyModel\n\n\nerrwins = set()\n\n\nclass MacroException(Exception):\n pass\n\nclass MacroClient(QObject):\n # A wrapper around proxy.ProxyClient that provides a simplified interface\n # to a macro to prevent it from accidentally making the proxy unstable.\n # Will add to it as needed/requested\n\n _macroOutput = pyqtSignal(str)\n _requestOutput = pyqtSignal(HTTPRequest)\n _ded = False\n\n def __init__(self, client):\n QObject.__init__(self)\n self._client = client\n\n def check_dead(self):\n \"\"\"\n Raises an exception if the program is trying to close. Use this in your loops so that your macro doesn't keep the program from quitting\n \"\"\"\n if self._ded:\n raise Exception(\"program over=very yes\")\n \n def submit(self, req, save=False):\n \"\"\"\n Submit a request. If save == True, it will be saved to history\n \"\"\"\n self.check_dead()\n self._client.submit(req, save=save)\n\n def save(self, req):\n \"\"\"\n Manually save a request to history. This can be used to perform a request and only save\n requests with interesting responses\n \"\"\"\n self.check_dead()\n self._client.save_new(req)\n\n def output(self, s):\n \"\"\"\n Write text to the \"output\" tab\n \"\"\"\n self.check_dead()\n self._macroOutput.emit(str(s)+\"\\n\")\n\n def output_req(self, req):\n \"\"\"\n Add a request/response to the list of outputted requests\n \"\"\"\n self.check_dead()\n self._requestOutput.emit(req)\n\n def new_request(self, method=\"GET\", path=\"/\", proto_major=1, proto_minor=1,\n headers=None, body=bytes(), dest_host=\"\", dest_port=80,\n use_tls=False, tags=None):\n \"\"\"\n Manually create a request object that can be submitted with client.submit()\n \"\"\"\n self.check_dead()\n return HTTPRequest(method=method, path=path, proto_major=proto_major, proto_minor=proto_minor,\n headers=headers, body=body, dest_host=dest_host, dest_port=dest_port,\n use_tls=use_tls, tags=tags)\n\nclass FileInterceptMacro(InterceptMacro, QObject):\n \"\"\"\n An intercepting macro that loads a macro from a file.\n \"\"\"\n macroError = pyqtSignal(str)\n \n def __init__(self, parent, client, filename):\n InterceptMacro.__init__(self)\n QObject.__init__(self)\n self.fname = filename or None # name from the file\n self.source = None\n self.client = client\n self.parent = parent\n self.mclient = MacroClient(self.client)\n self.cached_args = {}\n self.used_args = {}\n\n if filename:\n self.load(filename)\n\n def __repr__(self):\n s = self.fname or \"(No loaded macro)\"\n return \"\" % s\n\n def load(self, fname):\n if fname:\n self.fname = fname\n # yes there's a race condition here, but it's better than nothing\n st = os.stat(self.fname)\n if (st.st_mode & stat.S_IWOTH):\n raise MacroException(\"Refusing to load world-writable macro: %s\" % self.fname)\n module_name = self.fname\n try:\n if module_name in sys.modules and self.source != None:\n del sys.modules[module_name]\n del self.source\n self.source = imp.load_source(module_name, self.fname)\n except Exception as e:\n self.macroError.emit(make_err_str(self, e))\n else:\n self.fname = None\n self.source = None\n\n # Update what we can do\n if self.source and hasattr(self.source, 'mangle_request'):\n self.intercept_requests = True\n else:\n self.intercept_requests = False\n\n if self.source and hasattr(self.source, 'mangle_response'):\n self.intercept_responses = True\n else:\n self.intercept_responses = False\n\n if self.source and hasattr(self.source, 'mangle_websocket'):\n self.intercept_ws = True\n else:\n self.intercept_ws = False\n\n def prompt_args(self):\n if not hasattr(self.source, \"get_args\"):\n self.used_args = {}\n return True\n try:\n spec = self.source.get_args()\n except Exception as e:\n self.macroError.emit(make_err_str(self, e))\n return False\n args = get_macro_args(self.parent, spec, cached=self.cached_args)\n if args is None:\n return False\n self.cached_args = args\n self.used_args = args\n return True\n\n def init(self, args):\n if hasattr(self.source, 'init'):\n try:\n self.source.init(self.mclient, args)\n except Exception as e:\n self.macroError.emit(make_err_str(self, e))\n return False\n return True\n\n def mangle_request(self, request):\n if hasattr(self.source, 'mangle_request'):\n try:\n return self.source.mangle_request(self.mclient, self.used_args, request)\n except Exception as e:\n self.macroError.emit(make_err_str(self, e))\n return request\n\n def mangle_response(self, request, response):\n if hasattr(self.source, 'mangle_response'):\n try:\n return self.source.mangle_response(self.mclient, self.used_args, request, response)\n except Exception as e:\n self.macroError.emit(make_err_str(self, e))\n return response\n\n def mangle_websocket(self, request, response, message):\n if hasattr(self.source, 'mangle_websocket'):\n try:\n return self.source.mangle_websocket(self.mclient, self.used_args, request, response, message)\n except Exception as e:\n self.macroError.emit(make_err_str(self, e))\n return message\n\nclass FileMacro(QObject):\n macroError = pyqtSignal(str)\n macroComplete = pyqtSignal(str)\n requestOutput = pyqtSignal(HTTPRequest)\n macroOutput = pyqtSignal(str)\n\n def __init__(self, parent, filename='', resultSlot=None):\n QObject.__init__(self)\n self.fname = filename or None # filename we load from\n self.source = None\n self.parent = parent\n self.cached_args = {}\n self.load()\n\n def load(self):\n if self.fname:\n st = os.stat(self.fname)\n if (st.st_mode & stat.S_IWOTH):\n raise MacroException(\"Refusing to load world-writable macro: %s\" % self.fname)\n module_name = self.fname\n try:\n if module_name in sys.modules and self.source != None:\n del sys.modules[module_name]\n del self.source\n self.source = imp.load_source('%s'%module_name, self.fname)\n except Exception as e:\n self.macroError.emit(make_err_str(self, e))\n\n def execute(self, client, reqs):\n self.load()\n # Execute the macro\n if self.source:\n args = None\n if hasattr(self.source, \"get_args\"):\n try:\n spec = self.source.get_args()\n except Exception as e:\n self.macroError.emit(make_err_str(self, e))\n return\n args = get_macro_args(self.parent, spec, cached=self.cached_args)\n if args is None:\n return\n self.cached_args = args\n def perform_macro():\n mclient = MacroClient(client)\n mclient._macroOutput.connect(self.macroOutput)\n mclient._requestOutput.connect(self.requestOutput)\n try:\n self.source.run_macro(mclient, args, reqs)\n _, fname = os.path.split(self.fname)\n self.macroComplete.emit(\"%s has finished running\" % fname)\n except Exception as e:\n self.macroError.emit(make_err_str(self, e))\n ProxyThread(target=perform_macro).start()\n \nclass MacroWidget(QWidget):\n # Tabs containing both int and active macros\n\n def __init__(self, client, *args, **kwargs):\n self.client = client\n QWidget.__init__(self, *args, **kwargs)\n\n self.setLayout(QVBoxLayout())\n self.layout().setContentsMargins(0, 0, 0, 0)\n self.tab_widg = QTabWidget()\n\n self.active_widg = ActiveMacroWidget(client)\n self.active_ind = self.tab_widg.count()\n self.tab_widg.addTab(self.active_widg, \"Active\")\n\n self.int_widg = IntMacroWidget(client)\n self.int_ind = self.tab_widg.count()\n self.tab_widg.addTab(self.int_widg, \"Intercepting\")\n\n self.warning_widg = QLabel(\"

Warning! Macros may cause instability

Macros load and run python files into the Guppy process. If you're not careful when you write them you may cause Guppy to crash. If an active macro ends up in an infinite loop you may need to force kill the application when you quit.

PROCEED WITH CAUTION

\")\n self.warning_widg.setWordWrap(True)\n self.tab_widg.addTab(self.warning_widg, \"Warning\")\n\n self.layout().addWidget(self.tab_widg)\n \n def show_active(self):\n self.tab_widg.setCurrentIndex(self.active_ind)\n\n def show_int(self):\n self.tab_widg.setCurrentIndex(self.int_ind)\n \n def add_requests(self, reqs):\n # Add requests to active macro inputw\n self.active_widg.add_requests(reqs)\n \nclass IntMacroListModel(QAbstractTableModel):\n err_window = None\n \n def __init__(self, parent, client, *args, **kwargs):\n self.client = client\n QAbstractTableModel.__init__(self, *args, **kwargs)\n self.macros = []\n self.int_conns = {}\n self.conn_ids = count()\n self.parent = parent\n self.headers = [\"On\", \"Path\"]\n\n def _emit_all_data(self):\n self.dataChanged.emit(self.createIndex(0, 0), self.createIndex(self.columnCount(None), self.rowCount(None)))\n\n def headerData(self, section, orientation, role):\n if role == Qt.DisplayRole and orientation == Qt.Horizontal:\n return self.headers[section]\n return QVariant()\n\n def rowCount(self, parent):\n return len(self.macros)\n\n def columnCount(self, parent):\n return len(self.headers)\n \n def data(self, index, role):\n if role == Qt.DisplayRole:\n if index.column() == 1:\n rowdata = self.macros[index.row()]\n macro = rowdata[index.column()]\n return macro.fname\n if role == Qt.CheckStateRole:\n if index.column() == 0:\n if self.macros[index.row()][0]:\n return 2\n return 0\n return QVariant()\n \n def flags(self, index):\n f = Qt.ItemIsEnabled | Qt.ItemIsSelectable\n if index.column() == 0:\n f = f | Qt.ItemIsUserCheckable | Qt.ItemIsEditable\n return f\n\n def setData(self, index, value, role):\n if role == Qt.CheckStateRole and index.column() == 0:\n if value:\n self.enable_macro(index.row())\n else:\n self.disable_macro(index.row())\n return True\n return False\n\n # Non model functions\n\n @pyqtSlot(str)\n def add_macro_exception(self, estr):\n if not self.err_window:\n self.err_window = MacroErrWindow()\n self.err_window.add_error(estr)\n\n def add_macro(self, macro_path):\n self.beginResetModel()\n macro = FileInterceptMacro(self.parent, self.client, macro_path)\n macro.macroError.connect(self.add_macro_exception)\n self.macros.append([False, macro, -1])\n self._emit_all_data()\n self.endResetModel()\n \n\n def remove_macro(self, ind):\n self.beginResetModel()\n self.disable_macro(ind)\n self.macros = self.macros[:ind] + self.macros[ind+1:]\n self._emit_all_data()\n self.endResetModel()\n\n\n def enable_macro(self, ind):\n self.beginResetModel()\n macro = self.macros[ind][1]\n if not macro.init(None):\n return\n try:\n macro.load(macro.fname)\n except MacroException as e:\n display_error_box(\"Macro could not be loaded: %s\", e)\n return\n except Exception as e:\n self.add_macro_exception(make_err_str(macro, e))\n return\n if not (macro.intercept_requests or macro.intercept_responses or macro.intercept_ws):\n display_error_box(\"Macro must implement mangle_request or mangle_response\")\n return\n if not macro.prompt_args():\n return\n conn = self.client.new_conn()\n conn_id = next(self.conn_ids)\n self.macros[ind][2] = conn_id\n self.int_conns[conn_id] = conn\n conn.intercept(macro)\n self.macros[ind][0] = True\n self._emit_all_data()\n self.endResetModel()\n\n def disable_macro(self, ind):\n self.beginResetModel()\n conn_id = self.macros[ind][2]\n if conn_id >= 0:\n conn = self.int_conns[conn_id]\n conn.close()\n del self.int_conns[conn_id]\n self.macros[ind][2] = -1\n self.macros[ind][0] = False\n self._emit_all_data()\n self.endResetModel()\n\nclass IntMacroWidget(QWidget):\n # Lets the user enable/disable int. macros\n\n def __init__(self, client, *args, **kwargs):\n self.client = client\n self.macros = []\n QWidget.__init__(self, *args, **kwargs)\n \n self.setLayout(QVBoxLayout())\n self.layout().setContentsMargins(0, 0, 0, 0)\n \n buttonLayout = QHBoxLayout()\n new_button = QPushButton(\"New\")\n add_button = QPushButton(\"Add...\")\n remove_button = QPushButton(\"Remove\")\n new_button.clicked.connect(self.new_macro)\n add_button.clicked.connect(self.browse_macro)\n remove_button.clicked.connect(self.remove_selected)\n \n # Set up table\n self.macroListModel = IntMacroListModel(self, self.client)\n self.macroListView = QTableView()\n self.macroListView.setModel(self.macroListModel)\n\n self.macroListView.verticalHeader().setSectionResizeMode(QHeaderView.ResizeToContents)\n self.macroListView.verticalHeader().hide()\n self.macroListView.horizontalHeader().setSectionResizeMode(QHeaderView.ResizeToContents)\n self.macroListView.horizontalHeader().hide()\n self.macroListView.horizontalHeader().setStretchLastSection(True)\n\n self.macroListView.setSelectionBehavior(QAbstractItemView.SelectRows)\n self.macroListView.setSelectionMode(QAbstractItemView.SingleSelection)\n \n buttonLayout.addWidget(new_button)\n buttonLayout.addWidget(add_button)\n buttonLayout.addWidget(remove_button)\n buttonLayout.addStretch()\n self.layout().addWidget(self.macroListView)\n self.layout().addLayout(buttonLayout)\n \n def add_macro(self, fname):\n self.macroListModel.add_macro(fname)\n \n def reload_macros(self):\n self.macroListModel.reload_macros()\n\n @pyqtSlot()\n def new_macro(self):\n fname = save_dialog(self, filter_string=\"Python File (*.py)\")\n if not fname:\n return\n with open(fname, 'w') as f:\n contents = new_int_macro()\n f.write(contents)\n self.add_macro(fname)\n \n @pyqtSlot()\n def browse_macro(self):\n fname = open_dialog(self, filter_string=\"Python File (*.py)\")\n if not fname:\n return\n self.add_macro(fname)\n\n @pyqtSlot()\n def remove_selected(self):\n rows = self.macroListView.selectionModel().selectedRows()\n if len(rows) == 0:\n return\n for idx in rows:\n row = idx.row()\n self.macroListModel.remove_macro(row)\n return\n \nclass ActiveMacroModel(QAbstractTableModel):\n err_window = None\n requestOutput = pyqtSignal(HTTPRequest)\n macroOutput = pyqtSignal(str)\n\n def __init__(self, parent, client, *args, **kwargs):\n QAbstractTableModel.__init__(self, *args, **kwargs)\n self.client = client\n self.parent = parent\n self.headers = [\"Path\"]\n self.macros = []\n\n def _emit_all_data(self):\n self.dataChanged.emit(self.createIndex(0, 0), self.createIndex(self.columnCount(None), self.rowCount(None)))\n \n def headerData(self, section, orientation, role):\n if role == Qt.DisplayRole and orientation == Qt.Horizontal:\n return self.headers[section]\n return QVariant()\n\n def rowCount(self, parent):\n return len(self.macros)\n\n def columnCount(self, parent):\n return len(self.headers)\n\n def data(self, index, role):\n if role == Qt.DisplayRole:\n return self.macros[index.row()][0]\n return QVariant()\n\n def flags(self, index):\n return Qt.ItemIsEnabled | Qt.ItemIsSelectable\n \n def add_macro(self, path):\n self.beginResetModel()\n self._emit_all_data()\n fileMacro = FileMacro(self.parent, filename=path)\n fileMacro.macroOutput.connect(self.macroOutput)\n fileMacro.macroError.connect(self.add_macro_exception)\n fileMacro.requestOutput.connect(self.requestOutput)\n fileMacro.macroComplete.connect(self.display_macro_complete)\n self.macros.append((path, fileMacro))\n self.endResetModel()\n\n def run_macro(self, ind, reqs=None):\n path, macro = self.macros[ind]\n reqs = reqs or []\n macro.execute(self.client, reqs)\n\n def remove_macro(self, ind):\n self.beginResetModel()\n self._emit_all_data()\n self.macros = self.macros[:ind] + self.macros[ind+1:]\n self.endResetModel()\n\n @pyqtSlot(str)\n def add_macro_exception(self, estr):\n if not self.err_window:\n self.err_window = MacroErrWindow()\n self.err_window.add_error(estr)\n\n @pyqtSlot(str)\n def display_macro_complete(self, msg):\n display_info_box(msg, title=\"Macro complete\")\n \n\nclass ActiveMacroWidget(QWidget):\n # Provides an interface to send a set of requests to python scripts\n \n def __init__(self, client, *args, **kwargs):\n from .reqlist import ReqTableWidget, ReqBrowser\n\n QWidget.__init__(self, *args, **kwargs)\n self.client = client\n self.setLayout(QVBoxLayout())\n tab_widg = QTabWidget()\n\n # Input\n inputLayout = QVBoxLayout()\n inputLayout.setContentsMargins(0, 0, 0, 0)\n inputLayout.addWidget(QLabel(\"Input\"))\n inputLayout.setSpacing(8)\n self.reqlist = ReqTableWidget(self.client)\n butlayout = QHBoxLayout()\n delButton = QPushButton(\"Remove\")\n clearButton = QPushButton(\"Clear\")\n importAllButton = QPushButton(\"Import Currently Filtered Requests\")\n delButton.clicked.connect(self.reqlist.delete_selected)\n clearButton.clicked.connect(self.reqlist.clear)\n importAllButton.clicked.connect(self.import_all_reqs)\n butlayout.addWidget(delButton)\n butlayout.addWidget(clearButton)\n butlayout.addWidget(importAllButton)\n butlayout.addStretch()\n inputLayout.addWidget(self.reqlist)\n inputLayout.addLayout(butlayout)\n\n # Macro selection\n listLayout = QVBoxLayout()\n listLayout.addWidget(QLabel(\"Macros\"))\n listLayout.setContentsMargins(0, 0, 0, 0)\n listLayout.setSpacing(8)\n self.tableModel = ActiveMacroModel(self, self.client)\n self.tableModel.macroOutput.connect(self.add_macro_output)\n self.tableView = QTableView()\n self.tableModel.requestOutput.connect(self.add_request_output)\n self.tableView.setModel(self.tableModel)\n self.tableView.verticalHeader().setSectionResizeMode(QHeaderView.ResizeToContents)\n self.tableView.verticalHeader().hide()\n self.tableView.horizontalHeader().setSectionResizeMode(QHeaderView.ResizeToContents)\n self.tableView.horizontalHeader().setStretchLastSection(True)\n self.tableView.horizontalHeader().hide()\n self.tableView.setSelectionMode(QAbstractItemView.SingleSelection)\n butlayout2 = QHBoxLayout()\n newButton = QPushButton(\"New\")\n newButton.clicked.connect(self.new_macro)\n addButton2 = QPushButton(\"Add...\")\n addButton2.clicked.connect(self.browse_macro)\n delButton2 = QPushButton(\"Remove\")\n delButton2.clicked.connect(self.remove_selected)\n runButton2 = QPushButton(\"Run\")\n runButton2.clicked.connect(self.run_selected_macro)\n butlayout2.addWidget(newButton)\n butlayout2.addWidget(addButton2)\n butlayout2.addWidget(delButton2)\n butlayout2.addWidget(runButton2)\n butlayout2.addStretch()\n listLayout.addWidget(self.tableView)\n listLayout.addLayout(butlayout2)\n\n # Output\n outputLayout = QVBoxLayout()\n outputLayout.setContentsMargins(0, 0, 0, 0)\n outputLayout.setSpacing(8)\n self.outreqlist = ReqBrowser(self.client, reload_reqs=False, filter_tab=False)\n self.outreqlist.listWidg.allow_save = True\n outbutlayout = QHBoxLayout()\n delButton = QPushButton(\"Clear\")\n delButton.clicked.connect(self.clear_output)\n outbutlayout.addWidget(delButton)\n outbutlayout.addStretch()\n outputLayout.addWidget(self.outreqlist)\n outputLayout.addLayout(outbutlayout)\n\n text_out_layout = QVBoxLayout()\n text_out_layout.setContentsMargins(0, 0, 0, 0)\n self.macro_text_out = QPlainTextEdit()\n text_out_layout.addWidget(self.macro_text_out)\n text_out_butlayout = QHBoxLayout()\n clearBut = QPushButton(\"Clear\")\n clearBut.clicked.connect(self.clear_text_output)\n text_out_butlayout.addWidget(clearBut)\n text_out_butlayout.addStretch()\n text_out_layout.addLayout(text_out_butlayout)\n\n # Tabs\n intab = QWidget()\n intabLayout = QVBoxLayout()\n intabLayout.setContentsMargins(0, 0, 0, 0)\n intabLayout.addLayout(listLayout)\n intabLayout.addLayout(inputLayout)\n intab.setLayout(intabLayout)\n tab_widg.addTab(intab, \"Input\")\n\n reqOutputWidg = QWidget()\n reqOutputWidg.setLayout(outputLayout)\n tab_widg.addTab(reqOutputWidg, \"Req. Output\")\n\n textOutputWidg = QWidget()\n textOutputWidg.setLayout(text_out_layout)\n tab_widg.addTab(textOutputWidg, \"Text Output\")\n\n self.layout().addWidget(tab_widg)\n\n @pyqtSlot(list)\n def add_requests(self, reqs):\n # Add requests to active macro input\n for req in reqs:\n self.reqlist.add_request(req)\n \n @pyqtSlot()\n def new_macro(self):\n fname = save_dialog(self, filter_string=\"Python File (*.py)\")\n if not fname:\n return\n with open(fname, 'w') as f:\n contents = new_active_macro()\n f.write(contents)\n self.tableModel.add_macro(fname)\n\n @pyqtSlot()\n def browse_macro(self):\n fname = open_dialog(self, filter_string=\"Python File (*.py)\")\n if not fname:\n return\n self.tableModel.add_macro(fname)\n \n @pyqtSlot()\n def remove_selected(self):\n rows = self.tableView.selectionModel().selectedRows()\n if len(rows) == 0:\n return\n for idx in rows:\n row = idx.row()\n self.tableModel.remove_macro(row)\n return\n\n @pyqtSlot()\n def run_selected_macro(self):\n rows = self.tableView.selectionModel().selectedRows()\n if len(rows) == 0:\n return\n for idx in rows:\n row = idx.row()\n reqs = self.reqlist.get_all_requests()\n self.tableModel.run_macro(row, reqs)\n return\n\n @pyqtSlot(HTTPRequest)\n def add_request_output(self, req):\n self.outreqlist.listWidg.add_request(req)\n\n @pyqtSlot()\n def clear_output(self):\n self.outreqlist.set_requests([])\n\n @pyqtSlot()\n def clear_text_output(self):\n self.macro_text_out.setPlainText(\"\")\n\n @pyqtSlot(str)\n def add_macro_output(self, s):\n t = self.macro_text_out.toPlainText()\n t += s\n self.macro_text_out.setPlainText(t)\n \n @pyqtSlot()\n def import_all_reqs(self):\n reqs = self.client.in_context_requests(headers_only=True)\n self.add_requests(reqs)\n \n\nclass MacroErrWindow(QWidget):\n\n def __init__(self, *args, **kwargs):\n QObject.__init__(self, *args, **kwargs)\n self.msg = \"\"\n self.setLayout(QVBoxLayout())\n self.msgwidg = QPlainTextEdit()\n self.layout().addWidget(self.msgwidg)\n \n def add_error(self, msg):\n self.msg += msg + \"\\n\\n\"\n self.msgwidg.setPlainText(self.msg)\n self.show()\n \n def closeEvent(self, event):\n self.msgwidg.setPlainText(\"\")\n IntMacroListModel.err_window = None\n ActiveMacroModel.err_window = None\n \ndef make_err_str(macro, e):\n estr = \"Exception in macro %s:\\n\" % macro.fname\n estr += str(e) + '\\n'\n estr += str(traceback.format_exc())\n return estr\n\nclass ArgWindow(QDialog):\n def __init__(self, parent, argspec, cached=None):\n QDialog.__init__(self, parent)\n winLayout = QVBoxLayout()\n formLayout = QFormLayout()\n self.shownargs = []\n self.canceled = False\n argnames = set()\n for spec in argspec:\n name = None\n argtype = None\n argval = None\n if isinstance(spec, str):\n name = spec\n argtype = \"str\"\n else:\n if len(spec) > 0:\n name = spec[0]\n if len(spec) > 1:\n argtype = spec[1]\n if len(spec) > 2:\n argval = spec[2]\n if not name:\n continue\n if not argtype:\n continue\n\n if name in argnames:\n continue\n\n widg = None\n if argtype.lower() in (\"str\", \"string\"):\n argtype = \"str\"\n widg = QLineEdit()\n if name in cached:\n widg.setText(cached[name])\n else:\n return\n formLayout.addRow(QLabel(name), widg)\n self.shownargs.append(((name, argtype, argval), widg))\n argnames.add(name)\n butlayout = QHBoxLayout()\n okbut = QPushButton(\"Ok\")\n okbut.clicked.connect(self.accept)\n cancelbut = QPushButton(\"Cancel\")\n cancelbut.clicked.connect(self.reject)\n self.rejected.connect(self._set_canceled)\n butlayout.addWidget(okbut)\n butlayout.addWidget(cancelbut)\n butlayout.addStretch()\n winLayout.addLayout(formLayout)\n winLayout.addLayout(butlayout)\n\n self.setLayout(winLayout)\n\n @pyqtSlot()\n def _set_canceled(self):\n self.canceled = True\n\n def get_args(self):\n if self.canceled:\n return None\n retargs = {}\n for shownarg in self.shownargs:\n spec, widg = shownarg\n name, argtype, typeargs = spec\n if argtype == \"str\":\n retargs[name] = widg.text()\n return retargs\n\ndef get_macro_args(parent, argspec, cached=None):\n if not isinstance(argspec, list):\n return\n\n argwin = ArgWindow(parent, argspec, cached=cached)\n argwin.exec_()\n return argwin.get_args()\n\ndef req_python_def(varname, req):\n method = req.method\n path = req.url.geturl()\n pmajor = req.proto_major\n pminor = req.proto_minor\n headers = req.headers.dict().items()\n dest_host = req.dest_host\n dest_port = req.dest_port\n if req.use_tls:\n use_tls = \"True\"\n else:\n use_tls = \"False\"\n body = \"\"\n if len(req.body) > 0:\n s = '\"'\n if b'\\n' in req.body:\n s = '\"\"\"'\n for c in req.body:\n if chr(c) in qtprintable:\n body += chr(c)\n else:\n body += \"\\\\x%02x\" % c\n body = \"%s%s%s\" % (s, body, s)\n\n ret = ''\n ret += '%s = HTTPRequest(' % varname\n ret += 'proto_major=%d, proto_minor=%d,\\n' % (pmajor, pminor)\n ret += ' use_tls=%s, dest_host=\"%s\", dest_port=%d,\\n' % (use_tls, dest_host, dest_port)\n ret += ' method=\"%s\", path=\"%s\", headers={\\n' % (method, path)\n for k, vs in headers:\n qvs = []\n for v in vs:\n qvs.append('\"%s\"' % v)\n vstr = \"[\" + \", \".join(qvs) + \"]\"\n ret += ' \"%s\": %s,\\n' % (k, vstr)\n ret += ' },\\n'\n if len(body) > 0:\n ret += ' body=%s\\n' % body\n ret += \")\"\n return ret\n\ndef create_macro_template(reqs):\n ret = \"from guppyproxy.proxy import HTTPRequest\\n\\n\"\n i = 0\n for req in reqs:\n ret += req_python_def(\"req%d\"%i, req)\n ret += \"\\n\\n\"\n i += 1\n ret += \"def run_macro(client, args, reqs):\\n\"\n if i == 0:\n ret += \" pass\\n\"\n for ii in range(i):\n ret += \" client.submit(req%d)\\n\" % ii\n ret += \" client.output_req(req%d)\\n\\n\" % ii\n return ret\n\ndef new_active_macro():\n return \"def run_macro(client, args, reqs):\\n # Macro code goes here\\n pass\"\n\ndef new_int_macro():\n return \"\"\"def mangle_request(client, args, req):\n # modify request here\n return req\n\ndef mangle_response(client, args, req, rsp):\n # modify response here\n return rsp\n\"\"\"\n" }, { "path": "guppyproxy/proxy.py", "content": "#!/usr/bin/env python3\n\nimport base64\nimport copy\nimport datetime\nimport json\nimport math\nimport re\nimport socket\nimport threading\n\nfrom collections import namedtuple\nfrom itertools import count\nfrom urllib.parse import urlparse, ParseResult, parse_qs, urlencode\nfrom subprocess import Popen, PIPE\nfrom http import cookies as hcookies\nfrom PyQt5.QtCore import QThread, QObject, pyqtSlot\n\n\nclass MessageError(Exception):\n pass\n\n\nclass ProxyException(Exception):\n pass\n\n\nclass InvalidQuery(Exception):\n pass\n\n\nclass SocketClosed(Exception):\n pass\n\n\nclass SockBuffer:\n # I can't believe I have to implement this\n\n def __init__(self, sock):\n self.buf = [] # a list of chunks of strings\n self.s = sock\n self.closed = False\n\n def close(self):\n try:\n self.s.shutdown(socket.SHUT_RDWR)\n self.s.close()\n except OSError:\n # already closed\n pass\n finally:\n self.closed = True\n\n def _check_newline(self):\n for chunk in self.buf:\n if '\\n' in chunk:\n return True\n return False\n\n def readline(self):\n # Receive until we get a newline, raise SocketClosed if socket is closed\n while True:\n try:\n data = self.s.recv(256)\n except OSError:\n raise SocketClosed()\n if not data:\n raise SocketClosed()\n self.buf.append(data)\n if b'\\n' in data:\n break\n\n # Combine chunks\n allbytes = b''.join(self.buf)\n head, tail = allbytes.split(b'\\n', 1)\n self.buf = [tail]\n return head.decode()\n\n def send(self, data):\n try:\n self.s.send(data)\n except OSError:\n raise SocketClosed()\n\n\nclass ProxyThread(QThread):\n threads = {}\n tiditer = count()\n\n def __init__(self, target=None, args=tuple()):\n global mainWidg\n QThread.__init__(self)\n self.f = target\n self.args = args\n self.tid = next(ProxyThread.tiditer)\n ProxyThread.threads[self.tid] = self\n self.finished.connect(clean_thread(self.tid))\n\n def run(self):\n self.f(*self.args)\n\n def wait(self):\n QThread.wait(self)\n\n @classmethod\n def waitall(cls):\n ts = [(tid, thread) for tid, thread in cls.threads.items()]\n for tid, thread in ts:\n thread.wait()\n\ndef clean_thread(tid):\n @pyqtSlot()\n def clean():\n del ProxyThread.threads[tid]\n return clean\n\nclass Headers:\n def __init__(self, headers=None):\n self.headers = {}\n if headers is not None:\n if isinstance(headers, Headers):\n for _, pairs in headers.headers.items():\n for k, v in pairs:\n self.add(k, v)\n else:\n for k, vs in headers.items():\n for v in vs:\n self.add(k, v)\n\n def __contains__(self, hd):\n for k, _ in self.headers.items():\n if k.lower() == hd.lower():\n return True\n return False\n\n def add(self, k, v):\n try:\n lst = self.headers[k.lower()]\n lst.append((k, v))\n except KeyError:\n self.headers[k.lower()] = [(k, v)]\n\n def set(self, k, v):\n self.headers[k.lower()] = [(k, v)]\n\n def get(self, k):\n return self.headers[k.lower()][0][1]\n\n def delete(self, k):\n try:\n del self.headers[k.lower()]\n except KeyError:\n pass\n\n def pairs(self, key=None):\n for _, kvs in self.headers.items():\n for k, v in kvs:\n if key is None or k.lower() == key.lower():\n yield (k, v)\n\n def dict(self):\n retdict = {}\n for _, kvs in self.headers.items():\n for k, v in kvs:\n if k in retdict:\n retdict[k].append(v)\n else:\n retdict[k] = [v]\n return retdict\n\n\nclass RequestContext:\n def __init__(self, client, query=None):\n self._current_query = []\n self.client = client\n if query is not None:\n self._current_query = query\n\n def _validate(self, query):\n self.client.validate_query(query)\n\n def set_query(self, query):\n self._validate(query)\n self._current_query = query\n\n def apply_phrase(self, phrase):\n self._validate([phrase])\n self._current_query.append(phrase)\n\n def pop_phrase(self):\n if len(self._current_query) > 0:\n self._current_query.pop()\n\n def apply_filter(self, filt):\n self._validate([[filt]])\n self._current_query.append([filt])\n\n @property\n def query(self):\n return copy.deepcopy(self._current_query)\n\n\nclass URL:\n def __init__(self, url):\n parsed = urlparse(url)\n if url is not None:\n parsed = urlparse(url)\n self.scheme = parsed.scheme\n self.netloc = parsed.netloc\n self.path = parsed.path\n self.params = parsed.params\n self.query = parsed.query\n self.fragment = parsed.fragment\n else:\n self.scheme = \"\"\n self.netloc = \"\"\n self.path = \"/\"\n self.params = \"\"\n self.query = \"\"\n self.fragment = \"\"\n\n def geturl(self, include_params=True):\n params = self.params\n query = self.query\n fragment = self.fragment\n\n if not include_params:\n params = \"\"\n query = \"\"\n fragment = \"\"\n\n r = ParseResult(scheme=self.scheme,\n netloc=self.netloc,\n path=self.path,\n params=params,\n query=query,\n fragment=fragment)\n return r.geturl()\n\n def parameters(self):\n try:\n return parse_qs(self.query, keep_blank_values=True)\n except Exception:\n return {}\n\n def param_iter(self):\n for k, vs in self.parameters().items():\n for v in vs:\n yield k, v\n\n def set_param(self, key, val):\n params = self.parameters()\n params[key] = [val]\n self.query = urlencode(params)\n\n def add_param(self, key, val):\n params = self.parameters()\n if key in params:\n params[key].append(val)\n else:\n params[key] = [val]\n self.query = urlencode(params)\n\n def del_param(self, key):\n params = self.parameters()\n del params[key]\n self.query = urlencode(params)\n\n def set_params(self, params):\n self.query = urlencode(params)\n\n\nclass InterceptMacro:\n \"\"\"\n A class representing a macro that modifies requests as they pass through the\n proxy\n \"\"\"\n\n def __init__(self):\n self.name = ''\n self.intercept_requests = False\n self.intercept_responses = False\n self.intercept_ws = False\n\n def __repr__(self):\n return \"\" % self.name\n\n def mangle_request(self, request):\n return request\n\n def mangle_response(self, request, response):\n return response\n\n def mangle_websocket(self, request, response, message):\n return message\n\n\nclass HTTPRequest:\n def __init__(self, method=\"GET\", path=\"/\", proto_major=1, proto_minor=1,\n headers=None, body=bytes(), dest_host=\"\", dest_port=80,\n use_tls=False, time_start=None, time_end=None, db_id=\"\",\n tags=None, headers_only=False, storage_id=0):\n # http info\n self.method = method\n self.url = URL(path)\n self.proto_major = proto_major\n self.proto_minor = proto_minor\n\n self.headers = Headers(headers)\n\n self.headers_only = headers_only\n self._body = bytes()\n if not headers_only:\n self.body = body\n\n # metadata\n self.dest_host = dest_host\n self.dest_port = dest_port\n self.use_tls = use_tls\n self.time_start = time_start\n self.time_end = time_end\n\n self.response = None\n self.unmangled = None\n self.ws_messages = []\n\n self.db_id = db_id\n self.storage_id = storage_id\n if tags is not None:\n self.tags = set(tags)\n else:\n self.tags = set()\n\n @property\n def body(self):\n return self._body\n\n @body.setter\n def body(self, bs):\n self.headers_only = False\n if type(bs) is str:\n self._body = bs.encode()\n elif type(bs) is bytes:\n self._body = bs\n else:\n raise Exception(\"invalid body type: {}\".format(type(bs)))\n self.headers.set(\"Content-Length\", str(len(self._body)))\n\n @property\n def content_length(self):\n if 'content-length' in self.headers:\n return int(self.headers.get('content-length'))\n return len(self.body)\n\n def status_line(self):\n sline = \"{method} {path} HTTP/{proto_major}.{proto_minor}\".format(\n method=self.method, path=self.url.geturl(), proto_major=self.proto_major,\n proto_minor=self.proto_minor).encode()\n return sline\n\n def headers_section(self):\n message = self.status_line() + b\"\\r\\n\"\n for k, v in self.headers.pairs():\n message += \"{}: {}\\r\\n\".format(k, v).encode()\n return message\n\n def full_message(self):\n message = self.headers_section()\n message += b\"\\r\\n\"\n message += self.body\n return message\n\n def parameters(self):\n try:\n return parse_qs(self.body.decode(), keep_blank_values=True)\n except Exception:\n return {}\n\n def param_iter(self, ignore_content_type=False):\n if not ignore_content_type:\n if \"content-type\" not in self.headers:\n return\n if \"www-form-urlencoded\" not in self.headers.get(\"content-type\").lower():\n return\n for k, vs in self.parameters().items():\n for v in vs:\n yield k, v\n\n def set_param(self, key, val):\n params = self.parameters()\n params[key] = [val]\n self.body = urlencode(params)\n\n def add_param(self, key, val):\n params = self.parameters()\n if key in params:\n params[key].append(val)\n else:\n params[key] = [val]\n self.body = urlencode(params)\n\n def del_param(self, key):\n params = self.parameters()\n del params[key]\n self.body = urlencode(params)\n\n def set_params(self, params):\n self.body = urlencode(params)\n\n def cookies(self):\n try:\n cookie = hcookies.BaseCookie()\n cookie.load(self.headers.get(\"cookie\"))\n return cookie\n except Exception as e:\n return hcookies.BaseCookie()\n\n def cookie_iter(self):\n c = self.cookies()\n for k in c:\n yield k, c[k].value\n\n def set_cookie(self, key, val):\n c = self.cookies()\n c[key] = val\n self.set_cookies(c)\n\n def del_cookie(self, key):\n c = self.cookies()\n del c[key]\n self.set_cookies(c)\n\n def set_cookies(self, c):\n if isinstance(c, hcookies.BaseCookie):\n # it's a basecookie\n cookie_pairs = []\n for k in c:\n cookie_pairs.append('{}={}'.format(k, c[k].value))\n header_str = '; '.join(cookie_pairs)\n elif isinstance(c, HTTPRequest):\n # it's a request we should copy cookies from\n try:\n header_str = c.headers.get(\"Cookie\")\n except KeyError:\n header_str = \"\"\n else:\n # it's a dictionary\n cookie_pairs = []\n for k, v in c.items():\n cookie_pairs.append('{}={}'.format(k, v))\n header_str = '; '.join(cookie_pairs)\n\n if header_str == '':\n try:\n self.headers.delete(\"Cookie\")\n except KeyError:\n pass\n else:\n self.headers.set(\"Cookie\", header_str)\n\n def add_cookies(self, c):\n new_cookies = self.cookies()\n if isinstance(c, hcookies.BaseCookie):\n for k in c:\n new_cookies[k] = c[k].value\n elif isinstance(c, HTTPRequest):\n for k, v in c.cookie_iter():\n new_cookies[k] = v\n elif isinstance(c, HTTPResponse):\n for k, v in c.cookie_iter():\n new_cookies[k] = v\n else:\n for k, v in c.items():\n new_cookies[k] = v\n self.set_cookies(new_cookies)\n\n def full_url(self):\n return get_full_url(self)\n\n def copy(self):\n return HTTPRequest(\n method=self.method,\n path=self.url.geturl(),\n proto_major=self.proto_major,\n proto_minor=self.proto_minor,\n headers=self.headers,\n body=self.body,\n dest_host=self.dest_host,\n dest_port=self.dest_port,\n use_tls=self.use_tls,\n tags=copy.deepcopy(self.tags),\n headers_only=self.headers_only,\n )\n\n\nclass HTTPResponse:\n def __init__(self, status_code=200, reason=\"OK\", proto_major=1, proto_minor=1,\n headers=None, body=bytes(), db_id=\"\", headers_only=False, storage_id=0):\n self.status_code = status_code\n self.reason = reason\n self.proto_major = proto_major\n self.proto_minor = proto_minor\n\n self.headers = Headers()\n if headers is not None:\n for k, vs in headers.items():\n for v in vs:\n self.headers.add(k, v)\n\n self.headers_only = headers_only\n self._body = bytes()\n if not headers_only:\n self.body = body\n\n self.unmangled = None\n self.db_id = db_id\n self.storage = storage_id\n\n @property\n def body(self):\n return self._body\n\n @body.setter\n def body(self, bs):\n self.headers_only = False\n if type(bs) is str:\n self._body = bs.encode()\n elif type(bs) is bytes:\n self._body = bs\n else:\n raise Exception(\"invalid body type: {}\".format(type(bs)))\n self.headers.set(\"Content-Length\", str(len(self._body)))\n\n @property\n def content_length(self):\n if 'content-length' in self.headers:\n return int(self.headers.get('content-length'))\n return len(self.body)\n\n def status_line(self):\n sline = \"HTTP/{proto_major}.{proto_minor} {status_code} {reason}\".format(\n proto_major=self.proto_major, proto_minor=self.proto_minor,\n status_code=self.status_code, reason=self.reason).encode()\n return sline\n\n def headers_section(self):\n message = self.status_line() + b\"\\r\\n\"\n for k, v in self.headers.pairs():\n message += \"{}: {}\\r\\n\".format(k, v).encode()\n return message\n\n def full_message(self):\n message = self.headers_section()\n message += b\"\\r\\n\"\n message += self.body\n return message\n\n def cookies(self):\n try:\n cookie = hcookies.BaseCookie()\n for _, v in self.headers.pairs('set-cookie'):\n cookie.load(v)\n return cookie\n except Exception as e:\n return hcookies.BaseCookie()\n\n def cookie_iter(self):\n c = self.cookies()\n for k in c:\n yield k, c[k].value\n\n def set_cookie(self, key, val):\n c = self.cookies()\n c[key] = val\n self.set_cookies(c)\n\n def del_cookie(self, key):\n c = self.cookies()\n del c[key]\n self.set_cookies(c)\n\n def set_cookies(self, c):\n self.headers.delete(\"set-cookie\")\n if isinstance(c, hcookies.BaseCookie):\n cookies = c\n else:\n cookies = hcookies.BaseCookie()\n for k, v in c.items():\n cookies[k] = v\n for _, m in c.items():\n self.headers.add(\"Set-Cookie\", m.OutputString())\n\n def copy(self):\n return HTTPResponse(\n status_code=self.status_code,\n reason=self.reason,\n proto_major=self.proto_major,\n proto_minor=self.proto_minor,\n headers=self.headers.headers,\n body=self.body,\n headers_only=self.headers_only,\n )\n\n\nclass WSMessage:\n def __init__(self, is_binary=True, message=bytes(), to_server=True,\n timestamp=None, db_id=\"\", storage_id=0):\n self.is_binary = is_binary\n self.message = message\n self.to_server = to_server\n self.timestamp = timestamp or datetime.datetime(1970, 1, 1)\n\n self.unmangled = None\n self.db_id = db_id\n self.storage = storage_id\n\n def copy(self):\n return WSMessage(\n is_binary=self.is_binary,\n message=self.message,\n to_server=self.to_server,\n )\n\n\nScopeResult = namedtuple(\"ScopeResult\", [\"is_custom\", \"filter\"])\nListenerResult = namedtuple(\"ListenerResult\", [\"lid\", \"addr\"])\nGenPemCertsResult = namedtuple(\"GenPemCertsResult\", [\"key_pem\", \"cert_pem\"])\nSavedQuery = namedtuple(\"SavedQuery\", [\"name\", \"query\"])\nSavedStorage = namedtuple(\"SavedStorage\", [\"storage_id\", \"description\"])\n\n\ndef messagingFunction(func):\n def f(self, *args, **kwargs):\n if self.is_interactive:\n raise MessageError(\"cannot be called while other message is interactive\")\n if self.closed:\n raise MessageError(\"connection is closed\")\n with self.message_lock:\n return func(self, *args, **kwargs)\n return f\n\n\nclass ProxyConnection:\n next_id = 1\n\n def __init__(self, kind=\"\", addr=\"\"):\n self.connid = ProxyConnection.next_id\n ProxyConnection.next_id += 1\n self.sbuf = None\n self.buf = bytes()\n self.parent_client = None\n self.debug = False\n self.is_interactive = False\n self.closed = True\n self.message_lock = threading.Lock()\n self.kind = None\n self.addr = None\n self.int_thread = None\n\n if kind.lower() == \"tcp\":\n tcpaddr, port = addr.rsplit(\":\", 1)\n self.connect_tcp(tcpaddr, int(port))\n elif kind.lower() == \"unix\":\n self.connect_unix(addr)\n\n def __enter__(self):\n return self\n\n def __exit__(self, exc_type, exc_value, traceback):\n self.close()\n\n def connect_tcp(self, addr, port):\n s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)\n s.connect((addr, port))\n self.sbuf = SockBuffer(s)\n self.closed = False\n self.kind = \"tcp\"\n self.addr = \"{}:{}\".format(addr, port)\n\n def connect_unix(self, addr):\n s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)\n s.connect(addr)\n self.sbuf = SockBuffer(s)\n self.closed = False\n self.kind = \"unix\"\n self.addr = addr\n\n @property\n def maddr(self):\n if self.kind is not None:\n return \"{}:{}\".format(self.kind, self.addr)\n else:\n return None\n\n def close(self):\n self.sbuf.close()\n if self.parent_client is not None:\n try:\n self.parent_client.conns.remove(self)\n except KeyError:\n pass\n self.closed = True\n\n def read_message(self):\n ln = self.sbuf.readline()\n if self.debug:\n print(\"<({}) {}\".format(self.connid, ln))\n j = json.loads(ln)\n if (\"Success\" in j) and (j[\"Success\"] is False):\n if \"Reason\" in j:\n raise MessageError(j[\"Reason\"])\n raise MessageError(\"unknown error\")\n return j\n\n def submit_command(self, cmd):\n ln = json.dumps(cmd).encode() + b\"\\n\"\n if self.debug:\n print(\">({}) {}\".format(self.connid, ln.decode()[:-1]))\n self.sbuf.send(ln)\n\n def reqrsp_cmd(self, cmd):\n self.submit_command(cmd)\n ret = self.read_message()\n if ret is None:\n raise Exception()\n return ret\n\n ###########\n # Commands\n\n @messagingFunction\n def ping(self):\n cmd = {\"Command\": \"Ping\"}\n result = self.reqrsp_cmd(cmd)\n return result[\"Ping\"]\n\n @messagingFunction\n def submit(self, req, storage=0):\n cmd = {\n \"Command\": \"Submit\",\n \"Request\": encode_req(req),\n \"Storage\": 0,\n }\n if storage is not None:\n cmd[\"Storage\"] = storage\n result = self.reqrsp_cmd(cmd)\n if \"SubmittedRequest\" not in result:\n raise MessageError(\"no request returned\")\n newreq = decode_req(result[\"SubmittedRequest\"], storage=storage)\n req.response = newreq.response\n req.unmangled = newreq.unmangled\n req.time_start = newreq.time_start\n req.time_end = newreq.time_end\n req.db_id = newreq.db_id\n\n req.storage_id = storage\n\n @messagingFunction\n def save_new(self, req, storage):\n cmd = {\n \"Command\": \"SaveNew\",\n \"Request\": encode_req(req),\n \"Storage\": storage,\n }\n result = self.reqrsp_cmd(cmd)\n req.db_id = result[\"DbId\"]\n req.storage_id = storage\n return result[\"DbId\"]\n\n def _query_storage(self, q, storage, headers_only=False, max_results=0):\n cmd = {\n \"Command\": \"StorageQuery\",\n \"Query\": q,\n \"HeadersOnly\": headers_only,\n \"MaxResults\": max_results,\n \"Storage\": storage,\n }\n result = self.reqrsp_cmd(cmd)\n reqs = []\n unmangled = set()\n for reqd in result[\"Results\"]:\n req = decode_req(reqd, headers_only=headers_only, storage=storage)\n req.storage_id = storage\n reqs.append(req)\n if req.unmangled is not None:\n unmangled.add(req.unmangled.db_id)\n return [r for r in reqs if r.db_id not in unmangled]\n\n @messagingFunction\n def query_storage(self, q, storage, max_results=0, headers_only=False):\n return self._query_storage(q, storage, headers_only=headers_only, max_results=max_results)\n\n @messagingFunction\n def req_by_id(self, reqid, storage, headers_only=False):\n results = self._query_storage([[[\"dbid\", \"is\", reqid]]], storage,\n headers_only=headers_only, max_results=1)\n if len(results) == 0:\n raise MessageError(\"request with id {} does not exist\".format(reqid))\n return results[0]\n\n @messagingFunction\n def set_scope(self, filt):\n cmd = {\n \"Command\": \"SetScope\",\n \"Query\": filt,\n }\n self.reqrsp_cmd(cmd)\n\n @messagingFunction\n def get_scope(self):\n cmd = {\n \"Command\": \"ViewScope\",\n }\n result = self.reqrsp_cmd(cmd)\n ret = ScopeResult(result[\"IsCustom\"], result[\"Query\"])\n return ret\n\n @messagingFunction\n def add_tag(self, reqid, tag, storage):\n cmd = {\n \"Command\": \"AddTag\",\n \"ReqId\": reqid,\n \"Tag\": tag,\n \"Storage\": storage,\n }\n self.reqrsp_cmd(cmd)\n\n @messagingFunction\n def remove_tag(self, reqid, tag, storage):\n cmd = {\n \"Command\": \"RemoveTag\",\n \"ReqId\": reqid,\n \"Tag\": tag,\n \"Storage\": storage,\n }\n self.reqrsp_cmd(cmd)\n\n @messagingFunction\n def clear_tag(self, reqid, storage):\n cmd = {\n \"Command\": \"ClearTag\",\n \"ReqId\": reqid,\n \"Storage\": storage,\n }\n self.reqrsp_cmd(cmd)\n\n @messagingFunction\n def all_saved_queries(self, storage):\n cmd = {\n \"Command\": \"AllSavedQueries\",\n \"Storage\": storage,\n }\n results = self.reqrsp_cmd(cmd)\n queries = []\n for result in results[\"Queries\"]:\n queries.append(SavedQuery(name=result[\"Name\"], query=result[\"Query\"]))\n return queries\n\n @messagingFunction\n def save_query(self, name, filt, storage):\n cmd = {\n \"Command\": \"SaveQuery\",\n \"Name\": name,\n \"Query\": filt,\n \"Storage\": storage,\n }\n self.reqrsp_cmd(cmd)\n\n @messagingFunction\n def load_query(self, name, storage):\n cmd = {\n \"Command\": \"LoadQuery\",\n \"Name\": name,\n \"Storage\": storage,\n }\n result = self.reqrsp_cmd(cmd)\n return result[\"Query\"]\n\n @messagingFunction\n def delete_query(self, name, storage):\n cmd = {\n \"Command\": \"DeleteQuery\",\n \"Name\": name,\n \"Storage\": storage,\n }\n self.reqrsp_cmd(cmd)\n\n @messagingFunction\n def add_listener(self, addr, port, transparent=False, destHost=\"\",\n destPort=0, destUseTLS=False):\n laddr = \"{}:{}\".format(addr, port)\n cmd = {\n \"Command\": \"AddListener\",\n \"Type\": \"tcp\",\n \"Addr\": laddr,\n\n \"TransparentMode\": transparent,\n \"DestHost\": destHost,\n \"DestPort\": destPort,\n \"DestUseTLS\": destUseTLS,\n }\n result = self.reqrsp_cmd(cmd)\n lid = result[\"Id\"]\n return lid\n\n @messagingFunction\n def remove_listener(self, lid):\n cmd = {\n \"Command\": \"RemoveListener\",\n \"Id\": lid,\n }\n self.reqrsp_cmd(cmd)\n\n @messagingFunction\n def get_listeners(self):\n cmd = {\n \"Command\": \"GetListeners\",\n }\n result = self.reqrsp_cmd(cmd)\n results = []\n for r in result[\"Results\"]:\n results.append((r[\"Id\"], r[\"Addr\"]))\n return results\n\n @messagingFunction\n def load_certificates(self, cert_file, pkey_file):\n cmd = {\n \"Command\": \"LoadCerts\",\n \"KeyFile\": pkey_file,\n \"CertificateFile\": cert_file,\n }\n self.reqrsp_cmd(cmd)\n\n @messagingFunction\n def set_certificates(self, pkey_pem, cert_pem):\n cmd = {\n \"Command\": \"SetCerts\",\n \"KeyPEMData\": pkey_pem,\n \"CertificatePEMData\": cert_pem,\n }\n self.reqrsp_cmd(cmd)\n\n @messagingFunction\n def clear_certificates(self):\n cmd = {\n \"Command\": \"ClearCerts\",\n }\n self.reqrsp_cmd(cmd)\n\n @messagingFunction\n def generate_certificates(self, pkey_file, cert_file):\n cmd = {\n \"Command\": \"GenCerts\",\n \"KeyFile\": pkey_file,\n \"CertFile\": cert_file,\n }\n self.reqrsp_cmd(cmd)\n\n @messagingFunction\n def generate_pem_certificates(self):\n cmd = {\n \"Command\": \"GenPEMCerts\",\n }\n result = self.reqrsp_cmd(cmd)\n ret = GenPemCertsResult(result[\"KeyPEMData\"], result[\"CertificatePEMData\"])\n return ret\n\n @messagingFunction\n def validate_query(self, query):\n cmd = {\n \"Command\": \"ValidateQuery\",\n \"Query\": query,\n }\n try:\n self.reqrsp_cmd(cmd)\n except MessageError as e:\n raise InvalidQuery(str(e))\n\n @messagingFunction\n def check_request(self, query, req=None, storage_id=-1, db_id=\"\"):\n cmd = {\n \"Command\": \"checkrequest\",\n \"Query\": query,\n }\n if req:\n cmd[\"Request\"] = encode_req(req)\n if db_id != \"\":\n cmd[\"DbId\"] = db_id\n cmd[\"StorageId\"] = storage_id\n result = self.reqrsp_cmd(cmd)\n return result[\"Result\"]\n\n @messagingFunction\n def add_sqlite_storage(self, path, desc):\n cmd = {\n \"Command\": \"AddSQLiteStorage\",\n \"Path\": path,\n \"Description\": desc\n }\n result = self.reqrsp_cmd(cmd)\n return result[\"StorageId\"]\n\n @messagingFunction\n def add_in_memory_storage(self, desc):\n cmd = {\n \"Command\": \"AddInMemoryStorage\",\n \"Description\": desc\n }\n result = self.reqrsp_cmd(cmd)\n return result[\"StorageId\"]\n\n @messagingFunction\n def close_storage(self, storage_id):\n cmd = {\n \"Command\": \"CloseStorage\",\n \"StorageId\": storage_id,\n }\n self.reqrsp_cmd(cmd)\n\n @messagingFunction\n def set_proxy_storage(self, storage_id):\n cmd = {\n \"Command\": \"SetProxyStorage\",\n \"StorageId\": storage_id,\n }\n self.reqrsp_cmd(cmd)\n\n @messagingFunction\n def list_storage(self):\n cmd = {\n \"Command\": \"ListStorage\",\n }\n result = self.reqrsp_cmd(cmd)\n ret = []\n for ss in result[\"Storages\"]:\n ret.append(SavedStorage(ss[\"Id\"], ss[\"Description\"]))\n return ret\n\n @messagingFunction\n def set_proxy(self, use_proxy=False, proxy_host=\"\", proxy_port=0, use_creds=False,\n username=\"\", password=\"\", is_socks=False):\n cmd = {\n \"Command\": \"SetProxy\",\n \"UseProxy\": use_proxy,\n \"ProxyHost\": proxy_host,\n \"ProxyPort\": proxy_port,\n \"ProxyIsSOCKS\": is_socks,\n \"UseCredentials\": use_creds,\n \"Username\": username,\n \"Password\": password,\n }\n self.reqrsp_cmd(cmd)\n\n @messagingFunction\n def intercept(self, macro):\n # Run an intercepting macro until closed\n\n # Start intercepting\n self.is_interactive = True\n cmd = {\n \"Command\": \"Intercept\",\n \"InterceptRequests\": macro.intercept_requests,\n \"InterceptResponses\": macro.intercept_responses,\n \"InterceptWS\": macro.intercept_ws,\n }\n try:\n self.reqrsp_cmd(cmd)\n except Exception as e:\n self.is_interactive = False\n raise e\n\n def run_macro():\n iditer = count()\n threads = {}\n while True:\n try:\n msg = self.read_message()\n except MessageError as e:\n return\n except SocketClosed:\n return\n\n def mangle_and_respond(msg):\n retCmd = None\n if msg[\"Type\"] == \"httprequest\":\n req = decode_req(msg[\"Request\"])\n newReq = macro.mangle_request(req)\n\n if newReq is None:\n retCmd = {\n \"Id\": msg[\"Id\"],\n \"Dropped\": True,\n }\n else:\n newReq.unmangled = None\n newReq.response = None\n newReq.ws_messages = []\n\n retCmd = {\n \"Id\": msg[\"Id\"],\n \"Dropped\": False,\n \"Request\": encode_req(newReq),\n }\n elif msg[\"Type\"] == \"httpresponse\":\n req = decode_req(msg[\"Request\"])\n rsp = decode_rsp(msg[\"Response\"])\n newRsp = macro.mangle_response(req, rsp)\n\n if newRsp is None:\n retCmd = {\n \"Id\": msg[\"Id\"],\n \"Dropped\": True,\n }\n else:\n newRsp.unmangled = None\n\n retCmd = {\n \"Id\": msg[\"Id\"],\n \"Dropped\": False,\n \"Response\": encode_rsp(newRsp),\n }\n elif msg[\"Type\"] == \"wstoserver\" or msg[\"Type\"] == \"wstoclient\":\n req = decode_req(msg[\"Request\"])\n rsp = decode_rsp(msg[\"Response\"])\n wsm = decode_ws(msg[\"WSMessage\"])\n newWsm = macro.mangle_websocket(req, rsp, wsm)\n\n if newWsm is None:\n retCmd = {\n \"Id\": msg[\"Id\"],\n \"Dropped\": True,\n }\n else:\n newWsm.unmangled = None\n\n retCmd = {\n \"Id\": msg[\"Id\"],\n \"Dropped\": False,\n \"WSMessage\": encode_ws(newWsm),\n }\n else:\n raise Exception(\"Unknown message type: \" + msg[\"Type\"])\n if retCmd is not None:\n try:\n self.submit_command(retCmd)\n except SocketClosed:\n return\n\n tid = next(iditer)\n mangle_thread = ProxyThread(target=mangle_and_respond,\n args=(msg,))\n threads[tid] = mangle_thread\n mangle_thread.start()\n\n self.int_thread = ProxyThread(target=run_macro)\n self.int_thread.start()\n\n @messagingFunction\n def watch_storage(self, storage_id=-1, headers_only=True):\n # Generator that generates request, response, and wsmessages as they\n # are stored by the proxy\n cmd = {\n \"Command\": \"WatchStorage\",\n \"StorageId\": storage_id,\n \"HeadersOnly\": headers_only,\n }\n try:\n self.reqrsp_cmd(cmd)\n except Exception as e:\n self.is_interactive = False\n raise e\n\n while True:\n msg = self.read_message()\n if msg[\"Request\"]:\n msg[\"Request\"] = decode_req(msg[\"Request\"],\n storage=msg[\"StorageId\"],\n headers_only=headers_only)\n if msg[\"Response\"]:\n msg[\"Response\"] = decode_rsp(msg[\"Response\"],\n storage=msg[\"StorageId\"],\n headers_only=headers_only)\n if msg[\"WSMessage\"]:\n msg[\"WSMessage\"] = decode_ws(msg[\"WSMessage\"],\n storage=msg[\"StorageId\"],\n headers_only=headers_only)\n yield msg\n\n @messagingFunction\n def set_plugin_value(self, key, value, storage_id):\n cmd = {\n \"Command\": \"SetPluginValue\",\n \"Storage\": storage_id,\n \"Key\": key,\n \"Value\": value,\n }\n self.reqrsp_cmd(cmd)\n\n @messagingFunction\n def get_plugin_value(self, key, storage_id):\n cmd = {\n \"Command\": \"GetPluginValue\",\n \"Storage\": storage_id,\n \"Key\": key,\n }\n result = self.reqrsp_cmd(cmd)\n return result[\"Value\"]\n\n\nActiveStorage = namedtuple(\"ActiveStorage\", [\"type\", \"storage_id\", \"prefix\"])\n\n\ndef _serialize_storage(stype, prefix):\n return \"{}|{}\".format(stype, prefix)\n\n\nclass ProxyClient:\n def __init__(self, binary=None, debug=False, conn_addr=None):\n self.binloc = binary\n self.proxy_proc = None\n self.ltype = None\n self.laddr = None\n self.debug = debug\n self.conn_addr = conn_addr\n\n self.conns = set()\n self.msg_conn = None # conn for single req/rsp messages\n\n self.context = RequestContext(self)\n\n self.storage_by_id = {}\n self.storage_by_prefix = {}\n self.proxy_storage = None\n self.inmem_storage = None\n\n self.reqrsp_methods = {\n \"submit_command\",\n # \"reqrsp_cmd\",\n \"ping\",\n # \"submit\",\n # \"save_new\",\n # \"query_storage\",\n # \"req_by_id\",\n \"set_scope\",\n \"get_scope\",\n # \"add_tag\",\n # \"remove_tag\",\n # \"clear_tag\",\n \"all_saved_queries\",\n \"save_query\",\n \"load_query\",\n \"delete_query\",\n \"add_listener\",\n \"remove_listener\",\n \"get_listeners\",\n \"load_certificates\",\n \"set_certificates\",\n \"clear_certificates\",\n \"generate_certificates\",\n \"generate_pem_certificates\",\n \"validate_query\",\n # \"check_request\",\n \"list_storage\",\n # \"add_sqlite_storage\",\n # \"add_in_memory_storage\",\n # \"close_storage\",\n # \"set_proxy_storage\",\n \"set_proxy\",\n # \"set_plugin_value\",\n # \"get_plugin_value\",\n }\n\n def __enter__(self):\n if self.conn_addr is not None:\n self.msg_connect(self.conn_addr)\n else:\n self.execute_binary(binary=self.binloc, debug=self.debug)\n return self\n\n def __exit__(self, exc_type, exc_value, traceback):\n self.close()\n\n def __getattr__(self, name):\n if name in self.reqrsp_methods:\n return getattr(self.msg_conn, name)\n raise NotImplementedError(name)\n\n @property\n def maddr(self):\n if self.ltype is not None:\n return \"{}:{}\".format(self.ltype, self.laddr)\n else:\n return None\n\n def execute_binary(self, binary=None, debug=False, listen_addr=None):\n self.binloc = binary\n args = [self.binloc]\n if listen_addr is not None:\n args += [\"--msglisten\", listen_addr]\n else:\n args += [\"--msgauto\"]\n\n if debug:\n args += [\"--dbg\"]\n self.proxy_proc = Popen(args, stdout=PIPE, stderr=PIPE)\n\n # Wait for it to start and make connection\n listenstr = self.proxy_proc.stdout.readline().rstrip()\n self.msg_connect(listenstr.decode())\n\n def msg_connect(self, addr):\n self.ltype, self.laddr = addr.split(\":\", 1)\n self.msg_conn = self.new_conn()\n self._get_storage()\n\n def close(self):\n conns = list(self.conns)\n for conn in conns:\n conn.close()\n if self.proxy_proc is not None:\n self.proxy_proc.terminate()\n\n def new_conn(self):\n conn = ProxyConnection(kind=self.ltype, addr=self.laddr)\n conn.parent_client = self\n conn.debug = self.debug\n self.conns.add(conn)\n return conn\n\n # functions involving storage\n\n def _add_storage(self, storage, prefix):\n self.storage_by_prefix[prefix] = storage\n self.storage_by_id[storage.storage_id] = storage\n\n def _clear_storage(self):\n self.storage_by_prefix = {}\n self.storage_by_id = {}\n\n def _get_storage(self):\n self._clear_storage()\n storages = self.list_storage()\n for s in storages:\n stype, prefix = s.description.split(\"|\")\n storage = ActiveStorage(stype, s.storage_id, prefix)\n self._add_storage(storage, prefix)\n\n def parse_reqid(self, reqid):\n if reqid[0].isalpha():\n prefix = reqid[0]\n realid = reqid[1:]\n else:\n prefix = \"\"\n realid = reqid\n # `u`, `s` are special cases for the unmangled version of req and rsp\n if prefix == 'u':\n req = self.req_by_id(realid)\n if req.unmangled is None:\n raise MessageError(\"request %s was not mangled\" % reqid)\n ureq = req.unmangled\n return self.storage_by_id[ureq.storage_id], ureq.db_id\n elif prefix == 's':\n req = self.req_by_id(realid)\n if req.response is None:\n raise MessageError(\"response %s was not mangled\" % reqid)\n if req.response.unmangled is None:\n raise MessageError(\"response %s was not mangled\" % reqid)\n return self.storage_by_id[req.storage_id], req.db_id\n else:\n storage = self.storage_by_prefix[prefix]\n return storage, realid\n\n def storage_iter(self):\n for _, s in self.storage_by_id.items():\n yield s\n\n def _stg_or_def(self, storage):\n if storage is None:\n return self.proxy_storage\n return storage\n\n def is_in_context(self, req):\n return self.check_request(self.context.query, req)\n\n def in_context_requests(self, headers_only=False, max_results=0):\n return self.query_storage(self.context.query,\n headers_only=headers_only,\n max_results=max_results)\n\n def in_context_requests_async(self, slot, headers_only=False, max_results=0, *args, **kwargs):\n return self.query_storage(slot,\n self.context.query,\n headers_only=headers_only,\n max_results=max_results)\n\n def in_context_requests_iter(self, headers_only=False, max_results=0):\n results = self.query_storage(self.context.query,\n headers_only=headers_only,\n max_results=max_results)\n ret = results\n if max_results > 0 and len(results) > max_results:\n ret = results[:max_results]\n for reqh in ret:\n req = self.req_by_id(reqh.db_id, storage_id=reqh.storage_id)\n yield req\n\n def get_reqid(self, req):\n prefix = \"\"\n if req.storage_id in self.storage_by_id:\n s = self.storage_by_id[req.storage_id]\n prefix = s.prefix\n return \"{}{}\".format(prefix, req.db_id)\n\n def load_by_reqheaders(self, req):\n reqid = self.get_reqid(req)\n return self.req_by_id(reqid)\n\n # functions that don't just pass through to underlying conn\n\n def add_sqlite_storage(self, path, prefix):\n desc = _serialize_storage(\"sqlite\", prefix)\n sid = self.msg_conn.add_sqlite_storage(path, desc)\n s = ActiveStorage(type=\"sqlite\", storage_id=sid, prefix=prefix)\n self._add_storage(s, prefix)\n return s\n\n def add_in_memory_storage(self, prefix):\n desc = _serialize_storage(\"inmem\", prefix)\n sid = self.msg_conn.add_in_memory_storage(desc)\n s = ActiveStorage(type=\"inmem\", storage_id=sid, prefix=prefix)\n self._add_storage(s, prefix)\n return s\n\n def close_storage(self, storage_id):\n s = self.storage_by_id[storage_id]\n self.msg_conn.close_storage(s.storage_id)\n del self.storage_by_id[s.storage_id]\n del self.storage_by_prefix[s.prefix]\n\n def set_proxy_storage(self, storage_id):\n s = self.storage_by_id[storage_id]\n self.msg_conn.set_proxy_storage(s.storage_id)\n self.proxy_storage = storage_id\n\n def set_storage_prefix(self, storage_id, prefix):\n if prefix in self.storage_by_prefix:\n raise Exception(\"prefix already exists\")\n s = self.storage_by_id[storage_id]\n del self.storage_by_prefix[s.prefix]\n news = ActiveStorage(type=s.type, prefix=prefix, storage_id=s.storage_id)\n self.storage_by_prefix[news.prefix] = news\n self.storage_by_id[storage_id] = news\n\n def save_new(self, req, inmem=False, storage=None):\n if inmem:\n storage = self.inmem_storage\n else:\n storage = self._stg_or_def(storage)\n self.msg_conn.save_new(req, storage=storage)\n\n def submit(self, req, save=False, inmem=False, storage=None):\n if save:\n storage = self._stg_or_def(storage)\n if inmem:\n storage = self.inmem_storage\n self.msg_conn.submit(req, storage=storage)\n\n def query_storage(self, q, max_results=0, headers_only=False, storage=None, conn=None):\n results = []\n conn = conn or self.msg_conn\n if storage is None:\n for s in self.storage_iter():\n results += conn.query_storage(q, max_results=max_results,\n headers_only=headers_only,\n storage=s.storage_id)\n else:\n results += conn.query_storage(q, max_results=max_results,\n headers_only=headers_only,\n storage=storage)\n\n def kfunc(req):\n if req.time_start is None:\n return datetime.datetime.utcfromtimestamp(0)\n return req.time_start\n\n results.sort(key=kfunc)\n results = [r for r in reversed(results)]\n return results\n\n def query_storage_async(self, slot, *args, **kwargs):\n def perform_query():\n try:\n with self.new_conn() as c:\n r = self.query_storage(*args, conn=c, **kwargs)\n slot.emit(r)\n except Exception:\n pass\n ProxyThread(target=perform_query).start()\n\n def req_by_id(self, reqid, storage_id=None, headers_only=False):\n if storage_id is None:\n storage, db_id = self.parse_reqid(reqid)\n storage_id = storage.storage_id\n else:\n db_id = reqid\n retreq = self.msg_conn.req_by_id(db_id, headers_only=headers_only,\n storage=storage_id)\n\n if reqid[0] == 's': # `u` is handled by parse_reqid\n retreq.response = retreq.response.unmangled\n\n return retreq\n\n def check_request(self, query, req=None, reqid=\"\"):\n if req is not None:\n return self.msg_conn.check_request(query, req=req)\n else:\n storage, db_id = self.parse_reqid(reqid)\n storage_id = storage.storage_id\n return self.msg_conn.check_request(query, storage_id=storage_id, db_id=db_id)\n raise Exception(\"check_request requires either a request or reqid\")\n\n # for these and submit, might need storage stored on the request itself\n def add_tag(self, reqid, tag, storage=None):\n self.msg_conn.add_tag(reqid, tag, storage=self._stg_or_def(storage))\n\n def remove_tag(self, reqid, tag, storage=None):\n self.msg_conn.remove_tag(reqid, tag, storage=self._stg_or_def(storage))\n\n def clear_tag(self, reqid, storage=None):\n self.msg_conn.clear_tag(reqid, storage=self._stg_or_def(storage))\n\n def all_saved_queries(self, storage=None):\n self.msg_conn.all_saved_queries(storage=None)\n\n def save_query(self, name, filt, storage=None):\n self.msg_conn.save_query(name, filt, storage=self._stg_or_def(storage))\n\n def load_query(self, name, storage=None):\n self.msg_conn.load_query(name, storage=self._stg_or_def(storage))\n\n def delete_query(self, name, storage=None):\n self.msg_conn.delete_query(name, storage=self._stg_or_def(storage))\n\n def set_plugin_value(self, key, value, storage=None):\n self.msg_conn.set_plugin_value(key, value, self._stg_or_def(storage))\n\n def get_plugin_value(self, key, storage=None):\n return self.msg_conn.get_plugin_value(key, self._stg_or_def(storage))\n\n\ndef decode_req(result, headers_only=False, storage=0):\n if \"StartTime\" in result and result[\"StartTime\"] > 0:\n time_start = time_from_nsecs(result[\"StartTime\"])\n else:\n time_start = None\n\n if \"EndTime\" in result and result[\"EndTime\"] > 0:\n time_end = time_from_nsecs(result[\"EndTime\"])\n else:\n time_end = None\n\n if \"DbId\" in result:\n db_id = result[\"DbId\"]\n else:\n db_id = \"\"\n\n if \"Tags\" in result:\n tags = result[\"Tags\"]\n else:\n tags = \"\"\n\n ret = HTTPRequest(\n method=result[\"Method\"],\n path=result[\"Path\"],\n proto_major=result[\"ProtoMajor\"],\n proto_minor=result[\"ProtoMinor\"],\n headers=copy.deepcopy(result[\"Headers\"]),\n body=base64.b64decode(result[\"Body\"]),\n dest_host=result[\"DestHost\"],\n dest_port=result[\"DestPort\"],\n use_tls=result[\"UseTLS\"],\n time_start=time_start,\n time_end=time_end,\n tags=tags,\n headers_only=headers_only,\n db_id=db_id,\n storage_id=storage)\n\n if \"Unmangled\" in result:\n ret.unmangled = decode_req(result[\"Unmangled\"], headers_only=headers_only, storage=storage)\n if \"Response\" in result:\n ret.response = decode_rsp(result[\"Response\"], headers_only=headers_only, storage=storage)\n if \"WSMessages\" in result:\n for wsm in result[\"WSMessages\"]:\n ret.ws_messages.append(decode_ws(wsm, storage=storage))\n return ret\n\n\ndef decode_rsp(result, headers_only=False, storage=0):\n ret = HTTPResponse(\n status_code=result[\"StatusCode\"],\n reason=result[\"Reason\"],\n proto_major=result[\"ProtoMajor\"],\n proto_minor=result[\"ProtoMinor\"],\n headers=copy.deepcopy(result[\"Headers\"]),\n body=base64.b64decode(result[\"Body\"]),\n headers_only=headers_only,\n storage_id=storage,\n )\n\n if \"Unmangled\" in result:\n ret.unmangled = decode_rsp(result[\"Unmangled\"], headers_only=headers_only, storage=storage)\n return ret\n\n\ndef decode_ws(result, storage=0):\n timestamp = None\n db_id = \"\"\n\n if \"Timestamp\" in result:\n timestamp = time_from_nsecs(result[\"Timestamp\"])\n if \"DbId\" in result:\n db_id = result[\"DbId\"]\n\n ret = WSMessage(\n is_binary=result[\"IsBinary\"],\n message=base64.b64decode(result[\"Message\"]),\n to_server=result[\"ToServer\"],\n timestamp=timestamp,\n db_id=db_id,\n storage_id=storage,\n )\n\n if \"Unmangled\" in result:\n ret.unmangled = decode_ws(result[\"Unmangled\"], storage=storage)\n\n return ret\n\n\ndef encode_req(req, int_rsp=False):\n msg = {\n \"DestHost\": req.dest_host,\n \"DestPort\": req.dest_port,\n \"UseTLS\": req.use_tls,\n \"Method\": req.method,\n \"Path\": req.url.geturl(),\n \"ProtoMajor\": req.proto_major,\n \"ProtoMinor\": req.proto_major,\n \"Headers\": req.headers.dict(),\n \"Tags\": list(req.tags),\n \"Body\": base64.b64encode(copy.copy(req.body)).decode(),\n }\n\n if not int_rsp:\n msg[\"StartTime\"] = time_to_nsecs(req.time_start)\n msg[\"EndTime\"] = time_to_nsecs(req.time_end)\n if req.unmangled is not None:\n msg[\"Unmangled\"] = encode_req(req.unmangled)\n if req.response is not None:\n msg[\"Response\"] = encode_rsp(req.response)\n msg[\"WSMessages\"] = []\n for wsm in req.ws_messages:\n msg[\"WSMessages\"].append(encode_ws(wsm))\n return msg\n\n\ndef encode_rsp(rsp, int_rsp=False):\n msg = {\n \"ProtoMajor\": rsp.proto_major,\n \"ProtoMinor\": rsp.proto_minor,\n \"StatusCode\": rsp.status_code,\n \"Reason\": rsp.reason,\n \"Headers\": rsp.headers.dict(),\n \"Body\": base64.b64encode(copy.copy(rsp.body)).decode(),\n }\n\n if not int_rsp:\n if rsp.unmangled is not None:\n msg[\"Unmangled\"] = encode_rsp(rsp.unmangled)\n return msg\n\n\ndef encode_ws(ws, int_rsp=False):\n msg = {\n \"Message\": base64.b64encode(ws.message).decode(),\n \"IsBinary\": ws.is_binary,\n \"toServer\": ws.to_server,\n }\n if not int_rsp:\n if ws.unmangled is not None:\n msg[\"Unmangled\"] = encode_ws(ws.unmangled)\n msg[\"Timestamp\"] = time_to_nsecs(ws.timestamp)\n msg[\"DbId\"] = ws.db_id\n return msg\n\n\ndef time_from_nsecs(nsecs):\n secs = nsecs / 1000000000\n t = datetime.datetime.utcfromtimestamp(secs)\n return t\n\n\ndef time_to_nsecs(t):\n if t is None:\n return None\n secs = (t - datetime.datetime(1970, 1, 1)).total_seconds()\n return int(math.floor(secs * 1000000000))\n\n\nRequestStatusLine = namedtuple(\"RequestStatusLine\", [\"method\", \"path\", \"proto_major\", \"proto_minor\"])\nResponseStatusLine = namedtuple(\"ResponseStatusLine\", [\"proto_major\", \"proto_minor\", \"status_code\", \"reason\"])\n\n\ndef parse_req_sline(sline):\n if len(sline.split(b' ')) == 3:\n verb, path, version = sline.split(b' ')\n elif len(sline.split(b' ')) == 2:\n verb, version = sline.split(b' ')\n path = b''\n else:\n raise Exception(\"malformed statusline\")\n raw_version = version[5:] # strip HTTP/\n pmajor, pminor = raw_version.split(b'.', 1)\n return RequestStatusLine(verb.decode(), path.decode(), int(pmajor), int(pminor))\n\n\ndef parse_rsp_sline(sline):\n if len(sline.split(b' ')) > 2:\n version, status_code, reason = sline.split(b' ', 2)\n else:\n version, status_code = sline.split(b' ', 1)\n reason = ''\n raw_version = version[5:] # strip HTTP/\n pmajor, pminor = raw_version.split(b'.', 1)\n return ResponseStatusLine(int(pmajor), int(pminor), int(status_code), reason.decode())\n\n\ndef _parse_message(bs, sline_parser):\n header_env, body = re.split(br\"(?:\\r\\n|\\n)(?:\\r\\n|\\n)\", bs, 1)\n status_line, header_bytes = re.split(b\"\\r?\\n\", header_env, 1)\n h = Headers()\n for l in re.split(br\"\\r?\\n\", header_bytes):\n k, v = l.split(b\": \", 1)\n if k.lower != 'content-length':\n h.add(k.decode(), v.decode())\n h.add(\"Content-Length\", str(len(body)))\n return (sline_parser(status_line), h, body)\n\n\ndef parse_request(bs, dest_host='', dest_port=80, use_tls=False):\n req_sline, headers, body = _parse_message(bs, parse_req_sline)\n req = HTTPRequest(\n method=req_sline.method,\n path=req_sline.path,\n proto_major=req_sline.proto_major,\n proto_minor=req_sline.proto_minor,\n headers=headers.dict(),\n body=body,\n dest_host=dest_host,\n dest_port=dest_port,\n use_tls=use_tls)\n return req\n\n\ndef parse_response(bs):\n rsp_sline, headers, body = _parse_message(bs, parse_rsp_sline)\n rsp = HTTPResponse(\n status_code=rsp_sline.status_code,\n reason=rsp_sline.reason,\n proto_major=rsp_sline.proto_major,\n proto_minor=rsp_sline.proto_minor,\n headers=headers.dict(),\n body=body)\n return rsp\n\n\ndef get_full_url(req):\n netloc = req.dest_host\n if req.use_tls:\n scheme = \"https\"\n if req.dest_port != 443:\n netloc = \"%s:%d\" % (req.dest_host, req.dest_port)\n else:\n scheme = \"http\"\n if req.dest_port != 80:\n netloc = \"%s:%d\" % (req.dest_host, req.dest_port)\n rpath = req.url\n u = URL(\"\")\n u.scheme = scheme\n u.netloc = netloc\n u.path = rpath.path\n u.params = rpath.params\n u.query = rpath.query\n u.fragment = rpath.fragment\n return u.geturl()\n" }, { "path": "guppyproxy/repeater.py", "content": "from guppyproxy.util import display_error_box\nfrom guppyproxy.reqview import ReqViewWidget\nfrom PyQt5.QtWidgets import QWidget, QVBoxLayout, QHBoxLayout, QPushButton, QLineEdit, QCheckBox, QLabel, QSizePolicy, QToolButton\nfrom PyQt5.QtCore import pyqtSlot\n\n\nclass RepeaterWidget(QWidget):\n\n def __init__(self, client):\n QWidget.__init__(self)\n self.client = client\n self.history = []\n self.history_pos = 0\n\n self.setLayout(QVBoxLayout())\n self.layout().setSpacing(0)\n self.layout().setContentsMargins(0, 0, 0, 0)\n buttons = QHBoxLayout()\n buttons.setContentsMargins(0, 0, 0, 0)\n buttons.setSpacing(8)\n\n submitButton = QPushButton(\"Submit\")\n submitButton.clicked.connect(self.submit)\n self.dest_host_input = QLineEdit()\n self.dest_port_input = QLineEdit()\n self.dest_port_input.setMaxLength(5)\n self.dest_port_input.setSizePolicy(QSizePolicy.Minimum, QSizePolicy.Preferred)\n self.dest_usetls_input = QCheckBox()\n \n self.back_button = QToolButton()\n self.back_button.setText(\"<\")\n self.back_button.clicked.connect(self.back)\n self.forward_button = QToolButton()\n self.forward_button.setText(\">\")\n self.forward_button.clicked.connect(self.forward)\n\n buttons.addWidget(self.back_button)\n buttons.addWidget(self.forward_button)\n buttons.addWidget(submitButton)\n buttons.addWidget(QLabel(\"Host:\"))\n buttons.addWidget(self.dest_host_input)\n buttons.addWidget(QLabel(\"Port:\"))\n buttons.addWidget(self.dest_port_input)\n buttons.addWidget(QLabel(\"Use TLS:\"))\n buttons.addWidget(self.dest_usetls_input)\n buttons.addStretch()\n\n self.reqview = ReqViewWidget(tag_tab=True)\n self.reqview.set_read_only(False)\n self.reqview.set_tags_read_only(False)\n self.layout().addLayout(buttons)\n self.layout().addWidget(self.reqview)\n\n self.req = None\n self.dest_host = \"\"\n self.dest_port = 80\n self.use_tls = False\n self._update_buttons()\n \n def _set_host(self, host):\n self.dest_host_input.setText(host)\n\n def _set_port(self, port):\n if port is None or port <= 0:\n self.dest_port_input.setText(\"\")\n else:\n self.dest_port_input.setText(str(port))\n\n def _set_usetls(self, usetls):\n if usetls:\n self.dest_usetls_input.setCheckState(2)\n else:\n self.dest_usetls_input.setCheckState(0)\n \n def _set_dest_info(self, host, port, usetls):\n self._set_host(host)\n self._set_port(port)\n self._set_usetls(usetls)\n \n def _get_dest_info(self):\n host = self.dest_host_input.text()\n try:\n port = int(self.dest_port_input.text())\n except:\n port = -1\n if self.dest_usetls_input.checkState() == 0:\n usetls = False\n else:\n usetls = True\n return (host, port, usetls)\n\n def set_request(self, req, update_history=True):\n self._set_dest_info(\"\", -1, False)\n if update_history:\n self.history.append(req)\n self.history_pos = len(self.history)-1\n self._update_buttons()\n if req:\n self.req = req\n self.req.tags = set([\"repeater\"])\n self._set_dest_info(req.dest_host, req.dest_port, req.use_tls)\n self.reqview.set_request(self.req)\n\n @pyqtSlot(set)\n def update_req_tags(self, tags):\n if self.req:\n self.req.tags = tags\n\n @pyqtSlot()\n def submit(self):\n try:\n req = self.reqview.get_request()\n if not req:\n display_error_box(\"Could not parse request\")\n return\n except:\n display_error_box(\"Could not parse request\")\n return\n req.tags.add(\"repeater\")\n host, port, usetls = self._get_dest_info()\n if port is None:\n display_error_box(\"Invalid port\")\n return\n req.dest_host = host\n req.dest_port = port\n req.dest_usetls = usetls\n try:\n self.client.submit(req, save=True)\n self.req = req\n self.set_request(req)\n except Exception as e:\n errmsg = \"Error submitting request:\\n%s\" % str(e)\n display_error_box(errmsg)\n return\n \n @pyqtSlot()\n def back(self):\n if self.history_pos > 0:\n self.history_pos -= 1\n self.set_request(self.history[self.history_pos], update_history=False)\n self._update_buttons()\n\n @pyqtSlot()\n def forward(self):\n if self.history_pos < len(self.history)-1:\n self.history_pos += 1\n self.set_request(self.history[self.history_pos], update_history=False)\n self._update_buttons()\n \n def _update_buttons(self):\n self.forward_button.setEnabled(True)\n self.back_button.setEnabled(True)\n if len(self.history) == 0 or self.history_pos == len(self.history)-1:\n self.forward_button.setEnabled(False)\n if self.history_pos == 0:\n self.back_button.setEnabled(False)\n \n" }, { "path": "guppyproxy/reqlist.py", "content": "import threading\nimport shlex\n\nfrom guppyproxy.util import max_len_str, query_to_str, display_error_box, display_info_box, display_req_context, display_multi_req_context, hostport, method_color, sc_color, DisableUpdates, host_color\nfrom guppyproxy.proxy import HTTPRequest, RequestContext, InvalidQuery, SocketClosed, time_to_nsecs, ProxyThread\nfrom guppyproxy.reqview import ReqViewWidget\nfrom guppyproxy.reqtree import ReqTreeView\nfrom PyQt5.QtWidgets import QWidget, QTableWidget, QTableWidgetItem, QGridLayout, QHeaderView, QAbstractItemView, QVBoxLayout, QHBoxLayout, QComboBox, QTabWidget, QPushButton, QLineEdit, QStackedLayout, QToolButton, QCheckBox, QLabel, QTableView, QMenu\nfrom PyQt5.QtCore import pyqtSlot, pyqtSignal, QObject, QVariant, Qt, QAbstractTableModel, QModelIndex, QItemSelection, QSortFilterProxyModel\nfrom itertools import groupby, count\n\ndef get_field_entry():\n dropdown = QComboBox()\n dropdown.addItem(\"Anywhere\", \"all\")\n dropdown.addItem(\"Req. Body\", \"reqbody\")\n dropdown.addItem(\"Rsp. Body\", \"rspbody\")\n dropdown.addItem(\"Any Body\", \"body\")\n # dropdown.addItem(\"WSMessage\", \"wsmessage\")\n\n dropdown.addItem(\"Req. Header\", \"reqheader\")\n dropdown.addItem(\"Rsp. Header\", \"rspheader\")\n dropdown.addItem(\"Any Header\", \"header\")\n\n dropdown.addItem(\"Method\", \"method\")\n dropdown.addItem(\"Host\", \"host\")\n dropdown.addItem(\"Path\", \"path\")\n dropdown.addItem(\"URL\", \"url\")\n dropdown.addItem(\"Status\", \"statuscode\")\n dropdown.addItem(\"Tag\", \"tag\")\n\n dropdown.addItem(\"Any Param\", \"param\")\n dropdown.addItem(\"URL Param\", \"urlparam\")\n dropdown.addItem(\"Post Param\", \"postparam\")\n dropdown.addItem(\"Rsp. Cookie\", \"rspcookie\")\n dropdown.addItem(\"Req. Cookie\", \"reqcookie\")\n dropdown.addItem(\"Any Cookie\", \"cookie\")\n\n # dropdown.addItem(\"After\", \"\")\n # dropdown.addItem(\"Before\", \"\")\n # dropdown.addItem(\"TimeRange\", \"\")\n # dropdown.addItem(\"Id\", \"\")\n return dropdown\n\n\ndef get_string_cmp_entry():\n dropdown = QComboBox()\n dropdown.addItem(\"cnt.\", \"contains\")\n dropdown.addItem(\"cnt. (rgx)\", \"containsregexp\")\n dropdown.addItem(\"is\", \"is\")\n dropdown.addItem(\"len. >\", \"lengt\")\n dropdown.addItem(\"len. <\", \"lenlt\")\n dropdown.addItem(\"len. =\", \"leneq\")\n return dropdown\n\n\nclass StringCmpWidget(QWidget):\n returnPressed = pyqtSignal()\n\n def __init__(self, *args, **kwargs):\n QWidget.__init__(self, *args, **kwargs)\n layout = QHBoxLayout()\n self.cmp_entry = get_string_cmp_entry()\n self.text_entry = QLineEdit()\n self.text_entry.returnPressed.connect(self.returnPressed)\n layout.addWidget(self.cmp_entry)\n layout.addWidget(self.text_entry)\n self.setLayout(layout)\n self.layout().setContentsMargins(0, 0, 0, 0)\n\n def get_value(self):\n str_cmp = self.cmp_entry.itemData(self.cmp_entry.currentIndex())\n str_val = self.text_entry.text()\n return [str_cmp, str_val]\n\n def reset(self):\n self.cmp_entry.setCurrentIndex(0)\n self.text_entry.setText(\"\")\n\n\ndef dt_sort_key(r):\n if r.time_start:\n return time_to_nsecs(r.time_start)\n return 0\n\n\nclass StringKVWidget(QWidget):\n returnPressed = pyqtSignal()\n\n def __init__(self, *args, **kwargs):\n QWidget.__init__(self, *args, **kwargs)\n self.str2_shown = False\n self.str1 = StringCmpWidget()\n self.str2 = StringCmpWidget()\n self.str1.returnPressed.connect(self.returnPressed)\n self.str2.returnPressed.connect(self.returnPressed)\n self.toggle_button = QToolButton()\n self.toggle_button.setText(\"+\")\n\n self.toggle_button.clicked.connect(self._show_hide_str2)\n\n layout = QHBoxLayout()\n layout.addWidget(self.str1)\n layout.addWidget(self.str2)\n layout.addWidget(self.toggle_button)\n\n self.str2.setVisible(self.str2_shown)\n self.setLayout(layout)\n self.layout().setContentsMargins(0, 0, 0, 0)\n\n @pyqtSlot()\n def _show_hide_str2(self):\n if self.str2_shown:\n self.toggle_button.setText(\"+\")\n self.str2_shown = False\n else:\n self.toggle_button.setText(\"-\")\n self.str2_shown = True\n self.str2.setVisible(self.str2_shown)\n\n def get_value(self):\n retval = self.str1.get_value()\n if self.str2_shown:\n retval += self.str2.get_value()\n return retval\n\n def reset(self):\n self.str1.reset()\n self.str2.reset()\n\n\nclass DropdownFilterEntry(QWidget):\n # a widget that lets you enter filters using ezpz dropdowns/text boxes\n filterEntered = pyqtSignal(list)\n\n def __init__(self, *args, **kwargs):\n QWidget.__init__(self, *args, **kwargs)\n layout = QHBoxLayout()\n confirm = QToolButton()\n confirm.setText(\"OK\")\n confirm.setToolTip(\"Apply the entered filter\")\n self.field_entry = get_field_entry()\n\n # stack containing widgets for string, k/v, date, daterange\n self.str_cmp_entry = StringCmpWidget()\n self.kv_cmp_entry = StringKVWidget()\n self.inv_entry = QCheckBox(\"inv\")\n # date\n # daterange\n\n self.entry_layout = QStackedLayout()\n self.entry_layout.setContentsMargins(0, 0, 0, 0)\n self.current_entry = 0\n self.entry_layout.addWidget(self.str_cmp_entry)\n self.entry_layout.addWidget(self.kv_cmp_entry)\n # add date # 2\n # add daterange # 3\n\n confirm.clicked.connect(self.confirm_entry)\n self.str_cmp_entry.returnPressed.connect(self.confirm_entry)\n self.kv_cmp_entry.returnPressed.connect(self.confirm_entry)\n self.field_entry.currentIndexChanged.connect(self._display_value_widget)\n\n layout.addWidget(confirm)\n layout.addWidget(self.inv_entry)\n layout.addWidget(self.field_entry)\n layout.addLayout(self.entry_layout)\n\n self.setLayout(layout)\n self.setContentsMargins(0, 0, 0, 0)\n self._display_value_widget()\n\n @pyqtSlot()\n def _display_value_widget(self):\n # show the correct value widget in the value stack layout\n field = self.field_entry.itemData(self.field_entry.currentIndex())\n self.current_entry = 0\n if field in (\"all\", \"reqbody\", \"rspbody\", \"body\", \"wsmessage\", \"method\",\n \"host\", \"path\", \"url\", \"statuscode\", \"tag\"):\n self.current_entry = 0\n elif field in (\"reqheader\", \"rspheader\", \"header\", \"param\", \"urlparam\"\n \"postparam\", \"rspcookie\", \"reqcookie\", \"cookie\"):\n self.current_entry = 1\n # elif for date\n # elif for daterange\n self.entry_layout.setCurrentIndex(self.current_entry)\n\n def get_value(self):\n val = []\n if self.inv_entry.isChecked():\n val.append(\"inv\")\n field = self.field_entry.itemData(self.field_entry.currentIndex())\n val.append(field)\n if self.current_entry == 0:\n val += self.str_cmp_entry.get_value()\n elif self.current_entry == 1:\n val += self.kv_cmp_entry.get_value()\n # elif for date\n # elif for daterange\n return [val] # no support for OR\n\n @pyqtSlot()\n def confirm_entry(self):\n phrases = self.get_value()\n self.filterEntered.emit(phrases)\n self.str_cmp_entry.reset()\n self.kv_cmp_entry.reset()\n # reset date\n # reset date range\n\n\nclass TextFilterEntry(QWidget):\n # a text box that can be used to enter filters\n filterEntered = pyqtSignal(list)\n\n def __init__(self, *args, **kwargs):\n QWidget.__init__(self, *args, **kwargs)\n layout = QHBoxLayout()\n self.textEntry = QLineEdit()\n self.textEntry.returnPressed.connect(self.confirm_entry)\n self.textEntry.setToolTip(\"Enter the filter here and press return to apply it\")\n layout.addWidget(self.textEntry)\n self.setLayout(layout)\n self.layout().setContentsMargins(0, 0, 0, 0)\n\n @pyqtSlot()\n def confirm_entry(self):\n args = shlex.split(self.textEntry.text())\n phrases = [list(group) for k, group in groupby(args, lambda x: x == \"OR\") if not k]\n self.filterEntered.emit(phrases)\n self.textEntry.setText(\"\")\n\n\nclass FilterEntry(QWidget):\n # a widget that lets you switch between filter entries\n filterEntered = pyqtSignal(list)\n\n def __init__(self, *args, **kwargs):\n QWidget.__init__(self, *args, **kwargs)\n self.current_entry = 0\n self.max_entries = 2\n self.text_entry = TextFilterEntry()\n dropdown_entry = DropdownFilterEntry()\n\n self.text_entry.filterEntered.connect(self.filterEntered)\n dropdown_entry.filterEntered.connect(self.filterEntered)\n\n self.entry_layout = QStackedLayout()\n self.entry_layout.addWidget(dropdown_entry)\n self.entry_layout.addWidget(self.text_entry)\n\n swap_button = QToolButton()\n swap_button.setText(\">\")\n swap_button.setToolTip(\"Switch between dropdown and text entry\")\n swap_button.clicked.connect(self.next_entry)\n\n hlayout = QHBoxLayout()\n hlayout.addWidget(swap_button)\n hlayout.addLayout(self.entry_layout)\n self.setLayout(hlayout)\n self.layout().setContentsMargins(0, 0, 0, 0)\n self.layout().setSpacing(0)\n\n @pyqtSlot()\n def next_entry(self):\n self.current_entry += 1\n self.current_entry = self.current_entry % self.max_entries\n self.entry_layout.setCurrentIndex(self.current_entry)\n\n def set_entry(self, entry):\n self.current_entry = entry\n self.current_entry = self.current_entry % self.max_entries\n self.entry_layout.setCurrentIndex(self.current_entry)\n\n\nclass FilterListWidget(QTableWidget):\n # list part of the filter tab\n def __init__(self, *args, **kwargs):\n self.client = kwargs.pop(\"client\")\n QTableWidget.__init__(self, *args, **kwargs)\n self.context = RequestContext(self.client)\n\n # Set up table\n self.setColumnCount(1)\n self.horizontalHeader().hide()\n self.horizontalHeader().setSectionResizeMode(QHeaderView.Stretch)\n self.verticalHeader().hide()\n self.verticalHeader().setSectionResizeMode(QHeaderView.ResizeToContents)\n #self.setSelectionMode(QAbstractItemView.NoSelection)\n #self.setEditTriggers(QAbstractItemView.NoEditTriggers)\n\n def append_fstr(self, fstr):\n args = shlex.split(fstr)\n phrase = [list(group) for k, group in groupby(args, lambda x: x == \"OR\") if not k]\n self.context.apply_phrase(phrase)\n self._append_fstr_row(fstr)\n\n def set_query(self, query):\n self.context.set_query(query)\n self.redraw_table()\n\n def pop_phrase(self):\n self.context.pop_phrase()\n self.redraw_table()\n\n def clear_phrases(self):\n self.context.set_query([])\n self.redraw_table()\n\n def _append_fstr_row(self, fstr):\n row = self.rowCount()\n self.insertRow(row)\n self.setItem(row, 0, QTableWidgetItem(fstr))\n\n def redraw_table(self):\n self.setRowCount(0)\n query = self.context.query\n for p in query:\n condstrs = [' '.join(l) for l in p]\n fstr = ' OR '.join(condstrs)\n self._append_fstr_row(fstr)\n\n def get_query(self):\n return self.context.query\n\n\nclass FilterEditor(QWidget):\n # a widget containing a list of filters and the ability to edit the filters in the list\n filtersEdited = pyqtSignal(list)\n\n builtin_filters = (\n ('No Images', ['inv', 'path', 'containsregexp', r'(\\.png$|\\.jpg$|\\.jpeg$|\\.gif$|\\.ico$|\\.bmp$|\\.svg$)']),\n ('No JavaScript/CSS/Fonts', ['inv', 'path', 'containsregexp', r'(\\.js$|\\.css$|\\.woff$)']),\n )\n\n def __init__(self, *args, **kwargs):\n self.client = kwargs.pop(\"client\")\n QWidget.__init__(self, *args, **kwargs)\n layout = QVBoxLayout()\n\n # Manage bar\n manage_bar = QHBoxLayout()\n pop_button = QPushButton(\"Pop\")\n pop_button.setToolTip(\"Remove the most recently applied filter\")\n clear_button = QPushButton(\"Clear\")\n clear_button.setToolTip(\"Remove all active filters\")\n scope_reset_button = QPushButton(\"Scope\")\n scope_reset_button.setToolTip(\"Set the active filters to the current scope\")\n scope_save_button = QPushButton(\"Save Scope\")\n scope_save_button.setToolTip(\"Set the scope to the current filters. Any messages that don't match the active filters will be ignored by the proxy.\")\n\n self.builtin_combo = QComboBox()\n self.builtin_combo.addItem(\"Apply a built-in filter\", None)\n for desc, filt in FilterEditor.builtin_filters:\n self.builtin_combo.addItem(desc, filt)\n self.builtin_combo.currentIndexChanged.connect(self._apply_builtin_filter)\n\n manage_bar.addWidget(clear_button)\n manage_bar.addWidget(pop_button)\n manage_bar.addWidget(scope_reset_button)\n manage_bar.addWidget(scope_save_button)\n manage_bar.addWidget(self.builtin_combo)\n manage_bar.addStretch()\n mbar_widget = QWidget()\n mbar_widget.setLayout(manage_bar)\n pop_button.clicked.connect(self.pop_phrase)\n clear_button.clicked.connect(self.clear_phrases)\n scope_reset_button.clicked.connect(self.reset_to_scope)\n scope_save_button.clicked.connect(self.save_scope)\n\n # Filter list\n self.filter_list = FilterListWidget(client=self.client)\n\n # Filter entry\n self.entry = FilterEntry()\n self.entry.setMaximumHeight(self.entry.sizeHint().height())\n self.entry.filterEntered.connect(self.apply_phrase)\n\n layout.addWidget(mbar_widget)\n layout.addWidget(self.filter_list)\n layout.addWidget(self.entry)\n self.setLayout(layout)\n self.layout().setSpacing(0)\n self.layout().setContentsMargins(0, 0, 0, 0)\n\n @pyqtSlot()\n def save_scope(self):\n query = self.filter_list.get_query()\n self.client.set_scope(query)\n display_info_box(\"Scope updated\")\n\n @pyqtSlot()\n def reset_to_scope(self):\n query = self.client.get_scope().filter\n self.filter_list.set_query(query)\n self.filtersEdited.emit(self.filter_list.get_query())\n\n @pyqtSlot()\n def clear_phrases(self):\n self.filter_list.clear_phrases()\n self.filtersEdited.emit(self.filter_list.get_query())\n\n @pyqtSlot()\n def pop_phrase(self):\n self.filter_list.pop_phrase()\n self.filtersEdited.emit(self.filter_list.get_query())\n\n @pyqtSlot(list)\n def apply_phrase(self, phrase):\n fstr = query_to_str([phrase])\n try:\n self.filter_list.append_fstr(fstr)\n except InvalidQuery as e:\n display_error_box(\"Could not add filter:\\n\\n%s\" % e)\n return\n self.filtersEdited.emit(self.filter_list.get_query())\n\n @pyqtSlot(int)\n def _apply_builtin_filter(self, ind):\n phrase = self.builtin_combo.itemData(ind)\n if phrase:\n self.apply_phrase([phrase])\n self.builtin_combo.setCurrentIndex(0)\n \n def set_is_text(self, is_text):\n if is_text:\n self.entry.set_entry(1)\n else:\n self.entry.set_entry(0)\n \n\nclass ReqListModel(QAbstractTableModel):\n requestsLoading = pyqtSignal()\n requestsLoaded = pyqtSignal()\n \n HD_ID = 0\n HD_VERB = 1\n HD_HOST = 2\n HD_PATH = 3\n HD_SCODE = 4\n HD_REQLEN = 5\n HD_RSPLEN = 6\n HD_TIME = 7\n HD_TAGS = 8\n HD_MNGL = 9\n\n def __init__(self, client, *args, **kwargs):\n QAbstractTableModel.__init__(self, *args, **kwargs)\n self.client = client\n self.header_order = [\n self.HD_ID,\n self.HD_VERB,\n self.HD_HOST,\n self.HD_PATH,\n self.HD_SCODE,\n self.HD_REQLEN,\n self.HD_RSPLEN,\n self.HD_TIME,\n self.HD_TAGS,\n self.HD_MNGL,\n ]\n self.table_headers = {\n self.HD_ID: \"ID\",\n self.HD_VERB: \"Method\",\n self.HD_HOST: \"Host\",\n self.HD_PATH: \"Path\",\n self.HD_SCODE: \"S-Code\",\n self.HD_REQLEN: \"Req Len\",\n self.HD_RSPLEN: \"Rsp Len\",\n self.HD_TIME: \"Time\",\n self.HD_TAGS: \"Tags\",\n self.HD_MNGL: \"Mngl\",\n }\n self.reqs = []\n self.sort_enabled = False\n self.header_count = len(self.header_order)\n self.reqs_loaded = 0\n \n def headerData(self, section, orientation, role):\n if role == Qt.DisplayRole and orientation == Qt.Horizontal:\n hd = self.header_order[section]\n return self.table_headers[hd]\n return QVariant()\n \n def rowCount(self, parent):\n return self.reqs_loaded\n \n def columnCount(self, parent):\n return self.header_count\n \n def _gen_req_row(self, req):\n MAX_PATH_LEN = 60\n MAX_TAG_LEN = 40\n reqid = self.client.get_reqid(req)\n method = req.method\n host = hostport(req)\n path = max_len_str(req.url.path, MAX_PATH_LEN)\n reqlen = str(req.content_length)\n tags = max_len_str(', '.join(sorted(req.tags)), MAX_TAG_LEN)\n \n if req.response:\n scode = str(req.response.status_code) + ' ' + req.response.reason\n rsplen = str(req.response.content_length)\n else:\n scode = \"--\"\n rsplen = \"--\"\n\n if req.time_start and req.time_end:\n time_delt = req.time_end - req.time_start\n reqtime = (\"%.2f\" % time_delt.total_seconds())\n else:\n reqtime = \"--\"\n if req.unmangled and req.response and req.response.unmangled:\n manglestr = \"q/s\"\n elif req.unmangled:\n manglestr = \"q\"\n elif req.response and req.response.unmangled:\n manglestr = \"s\"\n else:\n manglestr = \"N/A\"\n return (req, reqid, method, host, path, scode, reqlen, rsplen, reqtime, tags, manglestr)\n \n \n def data(self, index, role):\n if role == Qt.BackgroundColorRole:\n req = self.reqs[index.row()][0]\n if index.column() == 2:\n return host_color(hostport(req))\n elif index.column() == 4:\n if req.response:\n return sc_color(str(req.response.status_code))\n elif index.column() == 1:\n return method_color(req.method)\n return QVariant()\n elif role == Qt.DisplayRole:\n rowdata = self.reqs[index.row()]\n return rowdata[index.column()+1]\n return QVariant()\n \n def canFetchMore(self, parent):\n if parent.isValid():\n return False\n return (self.reqs_loaded < len(self.reqs))\n \n def fetchMore(self, parent):\n if parent.isValid():\n return\n if self.reqs_loaded == len(self.reqs):\n return\n n_to_fetch = 50\n if self.reqs_loaded + n_to_fetch > len(self.reqs):\n n_to_fetch = len(self.reqs) - self.reqs_loaded\n self.beginInsertRows(QModelIndex(), self.reqs_loaded, self.reqs_loaded + n_to_fetch)\n self.reqs_loaded += n_to_fetch\n self.endInsertRows()\n\n def _sort_reqs(self):\n def skey(rowdata):\n return dt_sort_key(rowdata[0])\n if self.sort_enabled:\n self.reqs = sorted(self.reqs, key=skey, reverse=True)\n \n def _req_ind(self, req=None, reqid=None):\n if not reqid:\n reqid = self.client.get_reqid(req)\n for ind, rowdata in zip(count(), self.reqs):\n req = rowdata[0]\n if self.client.get_reqid(req) == reqid:\n return ind\n return -1\n \n def _emit_all_data(self):\n self.dataChanged.emit(self.createIndex(0, 0), self.createIndex(self.rowCount(None), self.columnCount(None)))\n \n def _set_requests(self, reqs):\n self.reqs = [self._gen_req_row(req) for req in reqs]\n self.reqs_loaded = 0\n \n def set_requests(self, reqs):\n self.beginResetModel()\n self._set_requests(reqs)\n self._sort_reqs()\n self._emit_all_data()\n self.endResetModel()\n \n def clear(self):\n self.beginResetModel()\n self.reqs = []\n self.reqs_loaded = 0\n self._emit_all_data()\n self.endResetModel()\n\n def add_request_head(self, req):\n self.beginInsertRows(QModelIndex(), 0, 0)\n self.reqs = [self._gen_req_row(req)] + self.reqs\n self.reqs_loaded += 1\n self.endInsertRows()\n \n def add_request(self, req):\n self.beginResetModel()\n self.reqs.append(self._gen_req_row(req))\n self.reqs_loaded = 0\n self._sort_reqs()\n self._emit_all_data()\n self.endResetModel()\n \n def add_requests(self, reqs):\n self.beginResetModel()\n for req in reqs:\n self.reqs.append(self._gen_req_row(req))\n self.reqs_loaded = 0\n self._sort_reqs()\n self._emit_all_data()\n self.endResetModel()\n \n def update_request(self, req):\n ind = self._req_ind(req)\n if ind < 0:\n return\n self.reqs[ind] = self._gen_req_row(req)\n self.dataChanged.emit(self.createIndex(ind, 0), self.createIndex(ind, self.rowCount(None)))\n\n def delete_request(self, req=None, reqid=None):\n ind = self._req_ind(req, reqid)\n if ind < 0:\n return\n self.beginRemoveRows(QModelIndex(), ind, ind)\n self.reqs_loaded -= 1\n self.reqs = self.reqs[:ind] + self.reqs[(ind+1):]\n self.endRemoveRows()\n \n def has_request(self, req=None, reqid=None):\n if self._req_ind(req, reqid) < 0:\n return False\n return True\n \n def get_requests(self):\n return [row[0] for row in self.reqs]\n \n def disable_sort(self):\n self.sort_enabled = False\n\n def enable_sort(self):\n self.sort_enabled = True\n self._sort_reqs()\n \n def req_by_ind(self, ind):\n return self.reqs[ind][0]\n\n \nclass ReqBrowser(QWidget):\n # Widget containing request viewer, tabs to view list of reqs, filters, and (evevntually) site map\n # automatically updated with requests as they're saved\n def __init__(self, client, repeater_widget=None, macro_widget=None, reload_reqs=True, update=False, filter_tab=True, is_client_context=False):\n QWidget.__init__(self)\n self.client = client\n self.filters = []\n self.reload_reqs = reload_reqs\n\n self.mylayout = QGridLayout()\n self.mylayout.setSpacing(0)\n self.mylayout.setContentsMargins(0, 0, 0, 0)\n\n # reqtable updater\n if update:\n self.updater = ReqListUpdater(self.client)\n else:\n self.updater = None\n\n # reqtable/search\n self.listWidg = ReqTableWidget(client, repeater_widget=repeater_widget, macro_widget=macro_widget)\n if self.updater:\n self.updater.add_reqlist_widget(self.listWidg)\n self.listWidg.requestsSelected.connect(self.update_viewer)\n self.listLayout = QVBoxLayout()\n self.listLayout.setContentsMargins(0, 0, 0, 0)\n self.listLayout.setSpacing(0)\n self.listButtonLayout = QHBoxLayout()\n self.listButtonLayout.setContentsMargins(0, 0, 0, 0)\n clearSelectionBut = QPushButton(\"Clear Selection\")\n clearSelectionBut.clicked.connect(self.listWidg.clear_selection)\n self.listButtonLayout.addWidget(clearSelectionBut)\n self.listButtonLayout.addStretch()\n self.listLayout.addWidget(self.listWidg)\n self.listLayout.addLayout(self.listButtonLayout)\n\n # Filter widget\n self.filterWidg = FilterEditor(client=self.client)\n self.filterWidg.filtersEdited.connect(self.listWidg.set_filter)\n if is_client_context:\n self.filterWidg.filtersEdited.connect(self.set_client_context)\n self.filterWidg.reset_to_scope()\n\n # Tree widget\n self.treeWidg = ReqTreeView()\n\n # add tabs\n self.listTabs = QTabWidget()\n lwidg = QWidget()\n lwidg.setLayout(self.listLayout)\n self.listTabs.addTab(lwidg, \"List\")\n self.tree_ind = self.listTabs.count()\n self.listTabs.addTab(self.treeWidg, \"Tree\")\n if filter_tab:\n self.listTabs.addTab(self.filterWidg, \"Filters\")\n self.listTabs.currentChanged.connect(self._tab_changed)\n\n # reqview\n self.reqview = ReqViewWidget(info_tab=True, param_tab=True, tag_tab=True)\n self.reqview.set_tags_read_only(False)\n self.reqview.tag_widg.tagsUpdated.connect(self._tags_updated)\n self.listWidg.req_view_widget = self.reqview\n\n self.mylayout.addWidget(self.reqview, 0, 0, 3, 1)\n self.mylayout.addWidget(self.listTabs, 4, 0, 2, 1)\n\n self.setLayout(self.mylayout)\n \n def show_filters(self):\n self.listTabs.setCurrentIndex(2)\n\n def show_history(self):\n self.listTabs.setCurrentIndex(0)\n\n def show_tree(self):\n self.listTabs.setCurrentIndex(1)\n\n @pyqtSlot(list)\n def set_client_context(self, query):\n self.client.context.set_query(query)\n \n @pyqtSlot()\n def reset_to_scope(self):\n self.filterWidg.reset_to_scope()\n\n @pyqtSlot(list)\n def update_viewer(self, reqs):\n self.reqview.set_request(None)\n if len(reqs) > 0:\n if self.reload_reqs:\n reqh = reqs[0]\n req = self.client.req_by_id(reqh.db_id)\n else:\n req = reqs[0]\n self.reqview.set_request(req)\n\n @pyqtSlot(list)\n def update_filters(self, query):\n self.filters = query\n\n @pyqtSlot(HTTPRequest)\n def add_request_item(self, req):\n self.listWidg.add_request_item(req)\n self.treeWidg.add_request_item(req)\n\n @pyqtSlot(list)\n def set_requests(self, reqs):\n self.listWidg.set_requests(reqs)\n self.treeWidg.set_requests(reqs)\n\n @pyqtSlot(int)\n def _tab_changed(self, i):\n if i == self.tree_ind:\n self.treeWidg.set_requests(self.listWidg.get_requests())\n\n @pyqtSlot(set)\n def _tags_updated(self, tags):\n req = self.reqview.req\n req.tags = tags\n if req.db_id:\n reqid = self.client.get_reqid(req)\n self.client.clear_tag(reqid)\n for tag in tags:\n self.client.add_tag(reqid, tag)\n \n def set_filter_is_text(self, is_text):\n self.filterWidg.set_is_text(is_text)\n \n\nclass ReqListUpdater(QObject):\n\n newRequest = pyqtSignal(HTTPRequest)\n requestUpdated = pyqtSignal(HTTPRequest)\n requestDeleted = pyqtSignal(str)\n\n def __init__(self, client):\n QObject.__init__(self)\n self.mtx = threading.Lock()\n self.client = client\n self.reqlist_widgets = []\n self.t = ProxyThread(target=self.run_updater)\n self.t.start()\n\n def add_reqlist_widget(self, widget):\n self.mtx.acquire()\n try:\n self.newRequest.connect(widget.add_request)\n self.requestUpdated.connect(widget.update_request)\n self.requestDeleted.connect(widget.delete_request)\n self.reqlist_widgets.append(widget)\n finally:\n self.mtx.release()\n\n def run_updater(self):\n conn = self.client.new_conn()\n try:\n try:\n for msg in conn.watch_storage():\n self.mtx.acquire()\n try:\n if msg[\"Action\"] == \"NewRequest\":\n self.newRequest.emit(msg[\"Request\"])\n elif msg[\"Action\"] == \"RequestUpdated\":\n self.requestUpdated.emit(msg[\"Request\"])\n elif msg[\"Action\"] == \"RequestDeleted\":\n self.requestDeleted.emit(msg[\"MessageId\"])\n finally:\n self.mtx.release()\n except SocketClosed:\n return\n finally:\n conn.close()\n\n def stop(self):\n self.conn.close()\n\n\nclass ReqTableWidget(QWidget):\n requestsChanged = pyqtSignal(list)\n requestsSelected = pyqtSignal(list)\n\n def __init__(self, client, repeater_widget=None, macro_widget=None, *args, **kwargs):\n QWidget.__init__(self, *args, **kwargs)\n self.allow_save = False\n\n self.client = client\n self.repeater_widget = repeater_widget\n self.macro_widget = macro_widget\n self.query = []\n self.req_view_widget = None\n\n self.setLayout(QStackedLayout())\n self.layout().setContentsMargins(0, 0, 0, 0)\n \n self.tableModel = ReqListModel(self.client)\n self.tableView = QTableView()\n self.tableView.setModel(self.tableModel)\n\n self.tableView.verticalHeader().setSectionResizeMode(QHeaderView.ResizeToContents)\n self.tableView.horizontalHeader().setSectionResizeMode(QHeaderView.ResizeToContents)\n self.tableView.verticalHeader().hide()\n self.tableView.setSelectionBehavior(QAbstractItemView.SelectRows)\n #self.tableView.setSelectionMode(QAbstractItemView.SingleSelection)\n self.tableView.horizontalHeader().setStretchLastSection(True)\n \n self.tableView.selectionModel().selectionChanged.connect(self.on_select_change)\n self.tableModel.dataChanged.connect(self._paint_view)\n self.tableModel.rowsInserted.connect(self._on_rows_inserted)\n self.requestsChanged.connect(self.set_requests)\n self.requestsSelected.connect(self._updated_selected_request)\n \n self.selected_reqs = []\n \n self.layout().addWidget(self.tableView)\n self.layout().addWidget(QLabel(\"Loading requests from data file...\"))\n \n @pyqtSlot(HTTPRequest)\n def add_request(self, req):\n with DisableUpdates(self.tableView):\n if req.db_id != \"\":\n reqid = self.client.get_reqid(req)\n if self.client.check_request(self.query, reqid=reqid):\n self.tableModel.add_request_head(req)\n if req.unmangled and req.unmangled.db_id != \"\" and self.tableModel.has_request(req.unmangled):\n self.tableModel.delete_request(req.unmangled)\n else:\n if self.client.check_request(self.query, req=req):\n self.tableModel.add_request_head(req)\n \n @pyqtSlot()\n def clear(self):\n self.tableModel.clear()\n \n def get_requests(self):\n return self.tableModel.get_requests()\n\n @pyqtSlot(list)\n def set_requests(self, reqs, check_filter=False):\n to_add = []\n if not check_filter:\n to_add = reqs\n else:\n for req in reqs:\n if req.db_id != \"\":\n reqid = self.client.get_reqid(req)\n if self.client.check_request(self.query, reqid=reqid):\n to_add.append(req)\n else:\n if self.client.check_request(self.query, req=req):\n to_add.append(req)\n with DisableUpdates(self.tableView):\n self.clear()\n self.tableModel.disable_sort()\n self.tableModel.add_requests(to_add)\n self.tableModel.enable_sort()\n self.set_is_not_loading()\n\n @pyqtSlot(HTTPRequest)\n def update_request(self, req):\n with DisableUpdates(self.tableView):\n self.tableModel.update_request(req)\n if req.db_id != \"\":\n if req.unmangled and req.unmangled.db_id != \"\":\n self.tableModel.delete_request(reqid=self.client.get_reqid(req.unmangled))\n\n @pyqtSlot(str)\n def delete_request(self, reqid):\n with DisableUpdates(self.tableView):\n self.tableModel.delete_request(reqid=reqid)\n\n @pyqtSlot(list)\n def set_filter(self, query):\n self.query = query\n self.set_is_loading()\n self.client.query_storage_async(self.requestsChanged, self.query, headers_only=True)\n\n @pyqtSlot(list)\n def _updated_selected_request(self, reqs):\n if len(reqs) > 0:\n self.selected_reqs = reqs\n else:\n self.selected_reqs = []\n \n @pyqtSlot(QModelIndex, int, int)\n def _on_rows_inserted(self, parent, first, last):\n rows = self.tableView.selectionModel().selectedRows()\n if len(rows) > 0:\n row = rows[0].row()\n idx = self.tableModel.index(row, 0, QModelIndex())\n self.tableView.scrollTo(idx)\n\n @pyqtSlot(QItemSelection, QItemSelection)\n def on_select_change(self, newSelection, oldSelection):\n reqs = []\n added = set()\n for rowidx in self.tableView.selectionModel().selectedRows():\n row = rowidx.row()\n if row not in added:\n reqs.append(self.tableModel.req_by_ind(row))\n added.add(row)\n self.requestsSelected.emit(reqs)\n\n @pyqtSlot()\n def clear_selection(self):\n self.tableView.clearSelection()\n \n def get_selected_request(self):\n # load the full request\n if len(self.selected_reqs) > 0:\n return self.client.load_by_reqheaders(self.selected_reqs[0])\n else:\n return None\n\n def get_selected_requests(self):\n ret = []\n for hreq in self.selected_reqs:\n ret.append(self.client.load_by_reqheaders(hreq))\n return ret\n\n def get_all_requests(self):\n return [self.client.req_by_id(self.client.get_reqid(req)) for req in self.tableModel.get_requests()]\n\n def contextMenuEvent(self, event):\n if len(self.selected_reqs) > 1:\n reqs = self.get_selected_requests()\n display_multi_req_context(self, self.client, reqs, event,\n macro_widget=self.macro_widget,\n save_option=self.allow_save)\n elif len(self.selected_reqs) == 1:\n req = self.get_selected_request()\n display_req_context(self, self.client, req, event,\n repeater_widget=self.repeater_widget,\n req_view_widget=self.req_view_widget,\n macro_widget=self.macro_widget,\n save_option=self.allow_save)\n\n def set_is_loading(self):\n self.set_loading(True)\n\n def set_is_not_loading(self):\n self.set_loading(False)\n\n def set_loading(self, is_loading):\n with DisableUpdates(self.tableView):\n if is_loading:\n self.layout().setCurrentIndex(1)\n else:\n self.layout().setCurrentIndex(0)\n \n @pyqtSlot(QModelIndex, QModelIndex)\n def _paint_view(self, indA, indB):\n self.tableView.repaint()\n \n @pyqtSlot()\n def delete_selected(self):\n with DisableUpdates(self.tableView):\n for req in self.selected_reqs:\n self.tableModel.delete_request(req=req)\n\n" }, { "path": "guppyproxy/reqtree.py", "content": "from guppyproxy.proxy import HTTPRequest\nfrom PyQt5.QtWidgets import QWidget, QTreeView, QVBoxLayout\nfrom PyQt5.QtGui import QStandardItem, QStandardItemModel\nfrom PyQt5.QtCore import pyqtSlot, Qt\n\n\ndef _include_req(req):\n if not req.response:\n return False\n if req.response.status_code == 404:\n return False\n return True\n\n\nclass PathNodeItem(QStandardItem):\n\n def __init__(self, text, *args, **kwargs):\n QStandardItem.__init__(self, *args, **kwargs)\n self.text = text\n self.children = {}\n\n def add_child(self, text):\n if text not in self.children:\n newitem = PathNodeItem(text, text)\n newitem.setFlags(newitem.flags() ^ Qt.ItemIsEditable)\n self.children[text] = newitem\n self.appendRow(newitem)\n\n def get_child(self, text):\n return self.children[text]\n\n def add_child_path(self, texts):\n if not texts:\n return\n childtext = texts[0]\n self.add_child(childtext)\n child = self.get_child(childtext)\n child.add_child_path(texts[1:])\n\n\nclass ReqTreeView(QWidget):\n def __init__(self):\n QWidget.__init__(self)\n\n self.setLayout(QVBoxLayout())\n self.layout().setSpacing(0)\n self.layout().setContentsMargins(0, 0, 0, 0)\n\n self.nodes = {}\n self.tree_view = QTreeView()\n self.tree_view.header().close()\n self.root = QStandardItemModel()\n self.tree_view.setModel(self.root)\n self.layout().addWidget(self.tree_view)\n\n @pyqtSlot(HTTPRequest)\n def add_request_item(self, req):\n path_parts = req.url.geturl(False).split(\"/\")\n path_parts = path_parts[1:]\n path_parts = [\"/\" + p for p in path_parts]\n path_parts = [req.dest_host] + path_parts\n if path_parts[0] not in self.nodes:\n item = PathNodeItem(path_parts[0], path_parts[0])\n item.setFlags(item.flags() ^ Qt.ItemIsEditable)\n self.nodes[path_parts[0]] = item\n self.root.appendRow(item)\n else:\n item = self.nodes[path_parts[0]]\n item.add_child_path(path_parts[1:])\n\n @pyqtSlot(list)\n def set_requests(self, reqs):\n self.clear()\n for req in reqs:\n if _include_req(req):\n self.add_request_item(req)\n self.tree_view.expandAll()\n\n def clear(self):\n self.nodes = {}\n self.root = QStandardItemModel()\n self.tree_view.setModel(self.root)\n" }, { "path": "guppyproxy/reqview.py", "content": "import re\n\nfrom guppyproxy.util import datetime_string, DisableUpdates\nfrom guppyproxy.proxy import HTTPRequest, get_full_url, parse_request\nfrom guppyproxy.hexteditor import ComboEditor\nfrom PyQt5.QtWidgets import QWidget, QTableWidget, QTableWidgetItem, QGridLayout, QHeaderView, QAbstractItemView, QLineEdit, QTabWidget, QVBoxLayout, QToolButton, QHBoxLayout, QStackedLayout\nfrom PyQt5.QtCore import pyqtSlot, pyqtSignal, Qt\nfrom pygments.lexer import Lexer\nfrom pygments.lexers import get_lexer_for_mimetype, TextLexer\nfrom pygments.lexers.textfmts import HttpLexer\nfrom pygments.util import ClassNotFound\nfrom pygments.token import Token\n\n\nclass HybridHttpLexer(Lexer):\n tl = TextLexer()\n hl = HttpLexer()\n \n def __init__(self, max_len=50000, *args, **kwargs):\n self.max_len = max_len\n Lexer.__init__(self, *args, **kwargs)\n\n def get_tokens_unprocessed(self, text):\n try:\n split = re.split(r\"(?:\\r\\n|\\n)(?:\\r\\n|\\n)\", text, 1)\n if len(split) == 2:\n h = split[0]\n body = split[1]\n else:\n h = split[0]\n body = ''\n except Exception as e:\n for v in self.tl.get_tokens_unprocessed(text):\n yield v\n raise e\n\n for token in self.hl.get_tokens_unprocessed(h):\n yield token\n\n if len(body) > 0:\n if len(body) <= self.max_len or self.max_len < 0:\n second_parser = None\n if \"Content-Type\" in h:\n try:\n ct = re.search(\"Content-Type: (.*)\", h)\n if ct is not None:\n hval = ct.groups()[0]\n mime = hval.split(\";\")[0]\n second_parser = get_lexer_for_mimetype(mime)\n except ClassNotFound:\n pass\n if second_parser is None:\n yield (len(h), Token.Text, text[len(h):])\n else:\n for index, tokentype, value in second_parser.get_tokens_unprocessed(text[len(h):]):\n yield (index + len(h), tokentype, value)\n else:\n yield (len(h), Token.Text, text[len(h):])\n\n\nclass InfoWidget(QWidget):\n def __init__(self, *args, **kwargs):\n QWidget.__init__(self, *args, **kwargs)\n self.request = None\n self.setLayout(QVBoxLayout())\n self.layout().setSpacing(0)\n self.layout().setContentsMargins(0, 0, 0, 0)\n self.infotable = QTableWidget()\n self.infotable.setColumnCount(2)\n\n self.infotable.verticalHeader().hide()\n self.infotable.horizontalHeader().hide()\n self.infotable.horizontalHeader().setSectionResizeMode(QHeaderView.ResizeToContents)\n self.infotable.verticalHeader().setSectionResizeMode(QHeaderView.ResizeToContents)\n self.infotable.horizontalHeader().setStretchLastSection(True)\n\n self.layout().addWidget(self.infotable)\n\n def _add_info(self, k, v):\n row = self.infotable.rowCount()\n self.infotable.insertRow(row)\n item1 = QTableWidgetItem(k)\n item1.setFlags(item1.flags() ^ Qt.ItemIsEditable)\n self.infotable.setItem(row, 0, item1)\n self.infotable.setItem(row, 1, QTableWidgetItem(v))\n\n def set_request(self, req):\n with DisableUpdates(self.infotable):\n self.request = req\n self.infotable.setRowCount(0)\n if self.request is None:\n return\n reqlen = len(self.request.body)\n reqlen = '%d bytes' % reqlen\n rsplen = 'No response'\n\n mangle_str = 'Nothing mangled'\n if self.request.unmangled:\n mangle_str = 'Request'\n\n if self.request.response:\n response_code = str(self.request.response.status_code) + \\\n ' ' + self.request.response.reason\n rsplen = self.request.response.content_length\n rsplen = '%d bytes' % rsplen\n\n if self.request.response.unmangled:\n if mangle_str == 'Nothing mangled':\n mangle_str = 'Response'\n else:\n mangle_str += ' and Response'\n else:\n response_code = ''\n\n time_str = '--'\n if self.request.time_end is not None and self.request.time_start is not None:\n time_delt = self.request.time_end - self.request.time_start\n time_str = \"%.2f sec\" % time_delt.total_seconds()\n\n if self.request.use_tls:\n is_ssl = 'YES'\n else:\n is_ssl = 'NO'\n\n if self.request.time_start:\n time_made_str = datetime_string(self.request.time_start)\n else:\n time_made_str = '--'\n\n verb = self.request.method\n host = self.request.dest_host\n\n self._add_info('Made on', time_made_str)\n self._add_info('URL', get_full_url(self.request))\n self._add_info('Host', host)\n self._add_info('Path', self.request.url.path)\n self._add_info('Verb', verb)\n self._add_info('Status Code', response_code)\n self._add_info('Request Length', reqlen)\n self._add_info('Response Length', rsplen)\n if self.request.response and self.request.response.unmangled:\n self._add_info('Unmangled Response Length', self.request.response.unmangled.content_length)\n self._add_info('Time', time_str)\n self._add_info('Port', str(self.request.dest_port))\n self._add_info('SSL', is_ssl)\n self._add_info('Mangled', mangle_str)\n self._add_info('Tags', ', '.join(self.request.tags))\n\n\nclass ParamWidget(QWidget):\n def __init__(self, *args, **kwargs):\n QWidget.__init__(self, *args, **kwargs)\n self.request = None\n self.setLayout(QVBoxLayout())\n self.tab_widget = QTabWidget()\n\n self.urltable = QTableWidget()\n self.urltable.setColumnCount(2)\n self.posttable = QTableWidget()\n self.posttable.setColumnCount(2)\n self.cookietable = QTableWidget()\n self.cookietable.setColumnCount(2)\n\n self.tab_widget.addTab(self.urltable, \"URL\")\n self.tab_widget.addTab(self.posttable, \"POST\")\n self.tab_widget.addTab(self.cookietable, \"Cookies\")\n\n self.format_table(self.urltable)\n self.format_table(self.posttable)\n self.format_table(self.cookietable)\n\n self.layout().addWidget(self.tab_widget)\n\n def _add_info(self, table, k, v):\n row = table.rowCount()\n table.insertRow(row)\n item1 = QTableWidgetItem(k)\n item1.setFlags(item1.flags() ^ Qt.ItemIsEditable)\n table.setItem(row, 0, item1)\n table.setItem(row, 1, QTableWidgetItem(v))\n\n def format_table(self, table):\n table.verticalHeader().hide()\n table.horizontalHeader().hide()\n table.horizontalHeader().setSectionResizeMode(QHeaderView.ResizeToContents)\n table.verticalHeader().setSectionResizeMode(QHeaderView.ResizeToContents)\n table.horizontalHeader().setStretchLastSection(True)\n\n def clear_tables(self):\n self.urltable.setRowCount(0)\n self.posttable.setRowCount(0)\n self.cookietable.setRowCount(0)\n\n def set_request(self, req):\n with DisableUpdates(self.urltable, self.posttable, self.cookietable):\n self.clear_tables()\n if req is None:\n return\n post_params = req.parameters()\n url_params = req.url.parameters()\n cookies = [(k, v) for k, v in req.cookie_iter()]\n\n if url_params:\n for k, vv in url_params.items():\n for v in vv:\n self._add_info(self.urltable, k, v)\n if post_params:\n for k, vv in post_params.items():\n for v in vv:\n self._add_info(self.posttable, k, v)\n if cookies:\n for k, v in cookies:\n self._add_info(self.cookietable, k, v)\n\n\nclass TagList(QTableWidget):\n tagsUpdated = pyqtSignal(set)\n\n # list part of the tag tab\n def __init__(self, *args, **kwargs):\n QTableWidget.__init__(self, *args, **kwargs)\n self.tags = set()\n\n # Set up table\n self.setColumnCount(1)\n self.horizontalHeader().hide()\n self.horizontalHeader().setSectionResizeMode(QHeaderView.Stretch)\n self.verticalHeader().hide()\n self.verticalHeader().setSectionResizeMode(QHeaderView.ResizeToContents)\n self.setEditTriggers(QAbstractItemView.NoEditTriggers)\n\n def add_tag(self, tag):\n self.tags.add(tag)\n self.redraw_table()\n self.tagsUpdated.emit(set(self.tags))\n\n def set_tags(self, tags, emit=True):\n self.tags = set(tags)\n self.redraw_table()\n if emit:\n self.tagsUpdated.emit(set(self.tags))\n\n def clear_tags(self):\n self.tags = set()\n self.redraw_table()\n self.tagsUpdated.emit(set(self.tags))\n\n def _append_str_row(self, fstr):\n row = self.rowCount()\n self.insertRow(row)\n self.setItem(row, 0, QTableWidgetItem(fstr))\n\n def redraw_table(self):\n self.setRowCount(0)\n for tag in sorted(self.tags):\n self._append_str_row(tag)\n\n @pyqtSlot()\n def delete_selected(self):\n rows = self.selectionModel().selectedRows()\n if len(rows) == 0:\n return\n for idx in rows:\n tag = self.item(idx.row(), 0).text()\n self.tags.remove(tag)\n self.redraw_table()\n self.tagsUpdated.emit(set(self.tags))\n\n def get_tags(self):\n return set(self.tags)\n\n\nclass TagWidget(QWidget):\n tagsUpdated = pyqtSignal(set)\n\n def __init__(self, *args, **kwargs):\n QWidget.__init__(self, *args, **kwargs)\n self.setLayout(QVBoxLayout())\n self.taglist = TagList()\n self.taglist.tagsUpdated.connect(self.tagsUpdated)\n self.layout().addWidget(self.taglist)\n\n self.taginput = QLineEdit()\n self.taginput.returnPressed.connect(self.add_tag)\n self.addbutton = QToolButton()\n self.addbutton.setText(\"+\")\n self.removebutton = QToolButton()\n self.removebutton.setText(\"-\")\n editbar = QHBoxLayout()\n editbar.addWidget(self.addbutton)\n editbar.addWidget(self.removebutton)\n editbar.addWidget(self.taginput)\n\n self.removebutton.clicked.connect(self.taglist.delete_selected)\n self.addbutton.clicked.connect(self.add_tag)\n\n self.layout().addLayout(editbar)\n\n @pyqtSlot()\n def add_tag(self):\n if self.readonly:\n return\n tag = self.taginput.text()\n if tag == \"\":\n return\n self.taglist.add_tag(tag)\n self.taginput.setText(\"\")\n\n def set_read_only(self, readonly):\n self.readonly = readonly\n self.addbutton.setEnabled(not readonly)\n self.removebutton.setEnabled(not readonly)\n\n\nclass ReqViewWidget(QWidget):\n requestEdited = pyqtSignal(HTTPRequest)\n\n def __init__(self, info_tab=False, param_tab=False, tag_tab=False, *args, **kwargs):\n QWidget.__init__(self, *args, **kwargs)\n self.request = None\n self.setLayout(QVBoxLayout())\n self.layout().setSpacing(0)\n self.layout().setContentsMargins(0, 0, 0, 0)\n\n view_layout = QGridLayout()\n view_layout.setSpacing(3)\n view_layout.setContentsMargins(0, 0, 0, 0)\n\n self.req_edit = ComboEditor()\n self.rsp_edit = ComboEditor()\n self.req_edit.setReadOnly(True)\n self.rsp_edit.setReadOnly(True)\n\n view_layout.addWidget(self.req_edit, 0, 0)\n view_layout.addWidget(self.rsp_edit, 0, 1)\n view_widg = QWidget()\n view_widg.setLayout(view_layout)\n\n use_tab = False\n if info_tab or tag_tab: # or or \n use_tab = True\n self.tab_widget = QTabWidget()\n self.tab_widget.addTab(view_widg, \"Message\")\n\n self.info_tab = False\n self.info_widg = None\n if info_tab:\n self.info_tab = True\n self.info_widg = InfoWidget()\n self.tab_widget.addTab(self.info_widg, \"Info\")\n\n self.param_tab = False\n self.param_widg = None\n if param_tab:\n self.param_tab = True\n self.param_widg = ParamWidget()\n self.tab_widget.addTab(self.param_widg, \"Params\")\n\n self.tag_tab = False\n self.tag_widg = None\n if tag_tab:\n self.tag_tab = True\n self.tag_widg = TagWidget()\n self.tab_widget.addTab(self.tag_widg, \"Tags\")\n\n if use_tab:\n self.layout().addWidget(self.tab_widget)\n else:\n self.layout().addWidget(view_widg)\n\n def set_read_only(self, ro):\n self.req_edit.setReadOnly(ro)\n\n def set_tags_read_only(self, ro):\n if self.tag_tab:\n self.tag_widg.set_read_only(ro)\n\n def get_request(self):\n try:\n req = parse_request(self.req_edit.get_bytes())\n req.dest_host = self.dest_host\n req.dest_port = self.dest_port\n req.use_tls = self.use_tls\n if self.tag_widg:\n req.tags = self.tag_widg.taglist.get_tags()\n return req\n except Exception as e:\n raise e\n return None\n\n @pyqtSlot(HTTPRequest)\n def set_request(self, req):\n self.req = req\n self.dest_host = \"\"\n self.dest_port = -1\n self.use_tls = False\n if req:\n self.dest_host = req.dest_host\n self.dest_port = req.dest_port\n self.use_tls = req.use_tls\n self.update_editors()\n if self.info_tab:\n self.info_widg.set_request(req)\n if self.tag_tab:\n if req:\n self.tag_widg.taglist.set_tags(req.tags, emit=False)\n if self.param_tab:\n self.param_widg.set_request(req)\n\n def update_editors(self):\n self.req_edit.set_bytes(b\"\")\n self.rsp_edit.set_bytes(b\"\")\n lex = HybridHttpLexer()\n if self.req is not None:\n self.req_edit.set_bytes_highlighted(self.req.full_message(), lexer=lex)\n if self.req.response is not None:\n self.rsp_edit.set_bytes_highlighted(self.req.response.full_message(), lexer=lex)\n \n def show_message(self):\n self.tab_widget.setCurrentIndex(0)\n" }, { "path": "guppyproxy/settings.py", "content": "from guppyproxy.util import list_remove, display_error_box, set_default_dialog_dir, default_dialog_dir, save_dialog, open_dialog\nfrom guppyproxy.proxy import MessageError\nfrom guppyproxy.config import ProxyConfig\nfrom PyQt5.QtWidgets import QWidget, QTableWidget, QTableWidgetItem, QHeaderView, QAbstractItemView, QFormLayout, QVBoxLayout, QHBoxLayout, QPushButton, QLineEdit, QSizePolicy, QToolButton, QCheckBox, QLabel\nfrom PyQt5.QtCore import pyqtSlot, pyqtSignal\nimport os\nimport copy\n\n\nclass ListenerList(QTableWidget):\n listenersUpdated = pyqtSignal(list)\n\n # list part of the listener tab\n def __init__(self, *args, **kwargs):\n QTableWidget.__init__(self, *args, **kwargs)\n self.listeners = []\n\n # Set up table\n self.setColumnCount(1)\n self.horizontalHeader().hide()\n self.horizontalHeader().setSectionResizeMode(QHeaderView.Stretch)\n self.verticalHeader().hide()\n self.verticalHeader().setSectionResizeMode(QHeaderView.ResizeToContents)\n self.setEditTriggers(QAbstractItemView.NoEditTriggers)\n\n def _add_listener(self, interface, port):\n self.listeners.append((interface, port))\n\n def add_listener(self, interface, port):\n self.listeners.append((interface, port))\n self.redraw_table()\n self.listenersUpdated.emit(self.listeners[:])\n\n def set_listeners(self, listeners):\n self.listeners = []\n for interface, port in listeners:\n self._add_listener(interface, port)\n self.redraw_table()\n self.listenersUpdated.emit(copy.deepcopy(self.listeners))\n\n def _append_row(self, interface, port):\n row = self.rowCount()\n self.insertRow(row)\n self.setItem(row, 0, QTableWidgetItem(\"%s:%s\" % (interface, port)))\n\n def redraw_table(self):\n self.setRowCount(0)\n for interface, port in self.listeners:\n self._append_row(interface, port)\n\n @pyqtSlot()\n def delete_selected(self):\n rows = self.selectionModel().selectedRows()\n if len(rows) == 0:\n return\n rownums = [idx.row() for idx in rows]\n self.listeners = list_remove(self.listeners, rownums)\n self.redraw_table()\n self.listenersUpdated.emit(self.listeners[:])\n\n def clear(self):\n self.listeners = []\n self.redraw_table()\n self.listenersUpdated.emit(self.listeners[:])\n\n def get_listeners(self):\n return self.listeners[:]\n\n\nclass ListenerWidget(QWidget):\n listenersUpdated = pyqtSignal(list)\n\n def __init__(self, *args, **kwargs):\n QWidget.__init__(self, *args, **kwargs)\n self.setLayout(QVBoxLayout())\n self.layout().setContentsMargins(0, 0, 0, 0)\n self.listenerlist = ListenerList()\n self.listenerlist.listenersUpdated.connect(self.listenersUpdated)\n self.layout().addWidget(self.listenerlist)\n\n self.hostinput = QLineEdit()\n self.hostinput.setText(\"127.0.0.1\")\n self.hostinput.returnPressed.connect(self.add_listener)\n self.portinput = QLineEdit()\n self.portinput.setMaxLength(5)\n self.portinput.setSizePolicy(QSizePolicy.Minimum, QSizePolicy.Preferred)\n self.portinput.returnPressed.connect(self.add_listener)\n self.addbutton = QToolButton()\n self.addbutton.setText(\"+\")\n self.removebutton = QToolButton()\n self.removebutton.setText(\"-\")\n editbar = QHBoxLayout()\n editbar.addWidget(self.addbutton)\n editbar.addWidget(self.removebutton)\n editbar.addWidget(QLabel(\"Interface:\"))\n editbar.addWidget(self.hostinput)\n editbar.addWidget(QLabel(\"Port:\"))\n editbar.addWidget(self.portinput)\n\n self.removebutton.clicked.connect(self.listenerlist.delete_selected)\n self.addbutton.clicked.connect(self.add_listener)\n\n self.layout().addLayout(editbar)\n\n @pyqtSlot()\n def add_listener(self):\n host = self.hostinput.text()\n port = self.portinput.text()\n if host == \"\":\n return\n if port == \"\":\n return\n try:\n port = int(port)\n except Exception:\n return\n self.listenerlist.add_listener(host, port)\n self.hostinput.setText(\"127.0.0.1\")\n self.portinput.setText(\"\")\n\n def set_listeners(self, listeners):\n self.listenerlist.set_listeners(listeners)\n\n\nclass DatafileWidget(QWidget):\n datafileLoaded = pyqtSignal(str)\n\n def __init__(self, *args, **kwargs):\n QWidget.__init__(self, *args, **kwargs)\n self.setLayout(QHBoxLayout())\n self.layout().setContentsMargins(0, 0, 0, 0)\n self.datapath = QLineEdit()\n newbutton = QPushButton(\"New\")\n newbutton.clicked.connect(self.new_datafile)\n browsebutton = QPushButton(\"Open\")\n browsebutton.clicked.connect(self.open_datafile)\n confirmbutton = QPushButton(\"Go!\")\n confirmbutton.clicked.connect(self._load_datafile)\n self.layout().addWidget(self.datapath)\n self.layout().addWidget(newbutton)\n self.layout().addWidget(browsebutton)\n self.layout().addWidget(confirmbutton)\n\n @pyqtSlot()\n def _load_datafile(self):\n path = self.datapath.text()\n self.datafileLoaded.emit(path)\n\n @pyqtSlot()\n def new_datafile(self):\n fname = save_dialog(self, filter_string=\"Database File (*.gpy)\")\n if not fname:\n return\n if len(fname) < 4 and fname[:-4] != \".gpy\":\n fname += \".gpy\"\n set_default_dialog_dir(fname)\n self.datapath.setText(fname)\n self._load_datafile()\n\n @pyqtSlot()\n def open_datafile(self):\n fname = open_dialog(self)\n if not fname:\n return\n self.datapath.setText(fname)\n set_default_dialog_dir(fname)\n self._load_datafile()\n\n\nclass ProxyInfoWidget(QWidget):\n proxyInfoUpdated = pyqtSignal(dict)\n\n def __init__(self, *args, **kwargs):\n QWidget.__init__(self, *args, **kwargs)\n self.setLayout(QFormLayout())\n\n self.enablebox = QCheckBox()\n self.enablebox.stateChanged.connect(self._enable_cb_statechange)\n self.hostinput = QLineEdit()\n self.portinput = QLineEdit()\n self.credsbox = QCheckBox()\n self.credsbox.stateChanged.connect(self._login_cb_statechange)\n self.credsbox.setCheckState(0)\n self.usernameinput = QLineEdit()\n self.passwordinput = QLineEdit()\n self.passwordinput.setEchoMode(QLineEdit.Password)\n self.socksbox = QCheckBox()\n self.confirmbutton = QPushButton(\"Confirm\")\n self.confirmbutton.clicked.connect(self._confirm_entry)\n\n self.layout().addRow(QLabel(\"Use Proxy\"), self.enablebox)\n self.layout().addRow(QLabel(\"Host\"), self.hostinput)\n self.layout().addRow(QLabel(\"Port\"), self.portinput)\n self.layout().addRow(QLabel(\"Use Login\"), self.credsbox)\n self.layout().addRow(QLabel(\"Username\"), self.usernameinput)\n self.layout().addRow(QLabel(\"Password\"), self.passwordinput)\n self.layout().addRow(QLabel(\"Use SOCKS\"), self.socksbox)\n self.layout().addRow(QLabel(\"\"), self.confirmbutton)\n\n self._set_enabled(False)\n self._set_login_enabled(False)\n\n @pyqtSlot(int)\n def _login_cb_statechange(self, state):\n if state == 0:\n self._set_login_enabled(False)\n else:\n self._set_login_enabled(True)\n\n @pyqtSlot(int)\n def _enable_cb_statechange(self, state):\n if state == 0:\n self._set_enabled(False)\n else:\n self._set_enabled(True)\n\n def _set_enabled(self, enabled):\n self.all_enabled = enabled\n self.hostinput.setEnabled(enabled)\n self.portinput.setEnabled(enabled)\n self.credsbox.setEnabled(enabled)\n self.socksbox.setEnabled(enabled)\n if enabled:\n self._set_login_enabled(self.loginenabled)\n else:\n self._set_login_enabled(False)\n\n def _set_login_enabled(self, enabled):\n self.loginenabled = enabled\n self.usernameinput.setEnabled(enabled)\n self.passwordinput.setEnabled(enabled)\n\n def _fill_form(self, enabled, host, port, need_creds, username, password, use_socks):\n if enabled:\n self.enablebox.setCheckState(2)\n else:\n self.enablebox.setCheckState(0)\n self.hostinput.setText(host)\n if port == 0:\n self.portinput.setText(\"\")\n else:\n self.portinput.setText(str(port))\n if need_creds:\n self.credsbox.setCheckState(2)\n else:\n self.credsbox.setCheckState(0)\n self.usernameinput.setText(username)\n self.passwordinput.setText(password)\n if use_socks:\n self.socksbox.setCheckState(2)\n else:\n self.socksbox.setCheckState(0)\n\n def _confirm_entry(self):\n use_proxy = not (self.enablebox.checkState() == 0)\n if use_proxy:\n host = self.hostinput.text()\n port = self.portinput.text()\n try:\n port = int(port)\n except Exception:\n return\n is_socks = not (self.socksbox.checkState() == 0)\n if self.credsbox.checkState() == 0:\n username = \"\"\n password = \"\"\n else:\n username = self.usernameinput.text()\n password = self.passwordinput.text()\n entry = {\"use_proxy\": use_proxy, \"host\": host, \"port\": port,\n \"is_socks\": is_socks, \"username\": username, \"password\": password}\n else:\n entry = {\"use_proxy\": False, \"host\": \"\", \"port\": 0,\n \"is_socks\": False, \"username\": \"\", \"password\": \"\"}\n self.proxyInfoUpdated.emit(entry)\n\n\nclass SettingsWidget(QWidget):\n datafileLoaded = pyqtSignal()\n\n def __init__(self, client, *args, **kwargs):\n QWidget.__init__(self, *args, **kwargs)\n self.client = client\n self.setLayout(QFormLayout())\n\n # Datafile\n self.datafilewidg = DatafileWidget()\n self.datafilewidg.datafileLoaded.connect(self._load_datafile)\n self.layout().addRow(QLabel(\"Datafile\"), self.datafilewidg)\n\n # Listeners\n self.listenerwidg = ListenerWidget()\n self.listenerwidg.listenersUpdated.connect(self._listeners_updated)\n self.layout().addRow(QLabel(\"Listeners\"), self.listenerwidg)\n\n # Proxy settings\n self.proxywidg = ProxyInfoWidget()\n self.proxywidg.proxyInfoUpdated.connect(self._set_proxy_settings)\n self.layout().addRow(QLabel(\"Proxy Settings\"), self.proxywidg)\n\n self.load_config()\n\n def load_config(self):\n # Load config\n self.config = ProxyConfig()\n try:\n configs = self.client.get_plugin_value(ProxyConfig.PLUGIN_KEY)\n except MessageError:\n configs = self.config.dumps()\n self.client.set_plugin_value(ProxyConfig.PLUGIN_KEY, configs)\n self.config.loads(configs)\n\n new_listeners = [(vals[0], vals[1]) for vals in self.config.listeners]\n self.listenerwidg.set_listeners(new_listeners)\n # fill proxy\n self.proxywidg._fill_form(self.config.use_proxy,\n self.config.proxy_host,\n self.config.proxy_port,\n not (self.config.proxy_username == \"\" and self.config.proxy_password == \"\"),\n self.config.proxy_username,\n self.config.proxy_password,\n self.config.is_socks_proxy)\n self.reload_listeners()\n\n @pyqtSlot(str)\n def _load_datafile(self, path):\n old_storage = self.client.proxy_storage\n try:\n storage = self.client.add_sqlite_storage(path, \"tmpprefix\")\n except MessageError as e:\n display_error_box(\"Could not load datafile:\\n%s\" % e)\n return\n self.client.close_storage(old_storage)\n self.client.set_storage_prefix(storage.storage_id, \"\")\n self.client.set_proxy_storage(storage.storage_id)\n self.client.disk_storage = storage\n self.load_config()\n self.datafileLoaded.emit()\n\n @pyqtSlot(list)\n def _listeners_updated(self, new_listeners):\n old_listensers = self.client.get_listeners()\n parsedold = {}\n for lid, addr in old_listensers:\n iface, port = addr.rsplit(':', 1)\n port = int(port)\n parsedold[(iface, port)] = lid\n oldset = set(parsedold.keys())\n newset = set(new_listeners)\n hosts_to_remove = oldset.difference(new_listeners)\n ids_to_remove = [parsedold[i] for i in hosts_to_remove]\n hosts_to_add = newset.difference(oldset)\n\n failed_listeners = []\n for i in ids_to_remove:\n self.client.remove_listener(i)\n for iface, port in hosts_to_add:\n try:\n self.client.add_listener(iface, port)\n except MessageError as e:\n err = \"%s:%s: %s\" % (iface, port, e)\n failed_listeners.append(err)\n if failed_listeners:\n errmsg = \"Failed to create listener(s):\\n\\n%s\" % ('\\n'.join(failed_listeners))\n display_error_box(errmsg)\n self.config.set_listeners([(host, port, None) for host, port in new_listeners]) # ignore transparent\n self.save_config()\n\n @pyqtSlot(dict)\n def _set_proxy_settings(self, proxy_data):\n self.config.proxy = proxy_data\n use_creds = (self.config.proxy_username != \"\" or self.config.proxy_password != \"\")\n self.client.set_proxy(self.config.use_proxy,\n self.config.proxy_host,\n self.config.proxy_port,\n use_creds,\n self.config.proxy_username,\n self.config.proxy_password,\n self.config.is_socks_proxy)\n self.save_config()\n\n def reload_listeners(self):\n hosts = self.client.get_listeners()\n pairs = []\n for lid, iface in hosts:\n host, port = iface.rsplit(\":\", 1)\n pairs.append((host, port))\n self.listenerwidg.blockSignals(True)\n self.listenerwidg.set_listeners(pairs)\n self.listenerwidg.blockSignals(False)\n\n def save_config(self):\n self.client.set_plugin_value(ProxyConfig.PLUGIN_KEY, self.config.dumps())\n" }, { "path": "guppyproxy/shortcuts.py", "content": "from guppyproxy.util import display_info_box, paste_clipboard\nfrom PyQt5.QtCore import pyqtSlot, QObject, Qt\nfrom PyQt5.QtWidgets import QShortcut \nfrom PyQt5.QtGui import QKeySequence\n\nclass GuppyShortcuts(QObject):\n \n ACT_NAV_FILTER_TEXT = 0\n ACT_NAV_FILTER_DROPDOWN = 1\n ACT_NAV_HISTORY = 2\n ACT_NAV_TREE = 3\n ACT_NAV_REPEATER = 4\n ACT_NAV_INTERCEPTOR = 5\n ACT_NAV_DECODER = 6\n ACT_NAV_DECODER_PASTE = 7\n ACT_NAV_FILTER_POP = 8\n ACT_OPEN = 9\n ACT_NEW = 10\n ACT_NAV_MACRO_ACTIVE = 11\n ACT_NAV_MACRO_INT = 12\n\n def __init__(self, guppy_window):\n QObject.__init__(self)\n self.guppy_window = guppy_window\n self.combos = {}\n\n self.add_shortcut(self.ACT_NAV_FILTER_TEXT,\n \"Navigate to filter text input\",\n self.nav_to_filter_text,\n QKeySequence(Qt.CTRL+Qt.Key_U))\n\n self.add_shortcut(self.ACT_NAV_FILTER_DROPDOWN,\n \"Navigate to filter dropdown input\",\n self.nav_to_filter_dropdown,\n QKeySequence(Qt.CTRL+Qt.Key_I))\n\n self.add_shortcut(self.ACT_NAV_FILTER_POP,\n \"Navigate to filters and pop most recent filter\",\n self.nav_to_filter_pop,\n QKeySequence(Qt.CTRL+Qt.Key_P))\n\n self.add_shortcut(self.ACT_NAV_HISTORY,\n \"Navigate to request list\",\n self.nav_to_history,\n QKeySequence(Qt.CTRL+Qt.Key_J))\n\n self.add_shortcut(self.ACT_NAV_TREE,\n \"Navigate to tree view\",\n self.nav_to_tree,\n QKeySequence(Qt.CTRL+Qt.Key_T))\n\n self.add_shortcut(self.ACT_NAV_REPEATER,\n \"Navigate to repeater\",\n self.nav_to_repeater,\n QKeySequence(Qt.CTRL+Qt.Key_R))\n\n self.add_shortcut(self.ACT_NAV_INTERCEPTOR,\n \"Navigate to interceptor\",\n self.nav_to_interceptor,\n QKeySequence(Qt.CTRL+Qt.Key_E))\n\n self.add_shortcut(self.ACT_NAV_DECODER,\n \"Navigate to decoder\",\n self.nav_to_decoder,\n QKeySequence(Qt.CTRL+Qt.Key_D))\n\n self.add_shortcut(self.ACT_NAV_DECODER_PASTE,\n \"Navigate to decoder and fill with clipboard\",\n self.nav_to_decoder_and_paste,\n QKeySequence(Qt.CTRL+Qt.SHIFT+Qt.Key_D))\n\n self.add_shortcut(self.ACT_OPEN,\n \"Open datafile\",\n self.open_datafile,\n QKeySequence(Qt.CTRL+Qt.SHIFT+Qt.Key_O))\n\n self.add_shortcut(self.ACT_NEW,\n \"New datafile\",\n self.new_datafile,\n QKeySequence(Qt.CTRL+Qt.SHIFT+Qt.Key_N))\n\n self.add_shortcut(self.ACT_NAV_MACRO_ACTIVE,\n \"Navigate to active macros\",\n self.nav_to_active_macros,\n QKeySequence(Qt.CTRL+Qt.Key_M))\n\n self.add_shortcut(self.ACT_NAV_MACRO_INT,\n \"Navigate to intercepting macros\",\n self.nav_to_int_macros,\n QKeySequence(Qt.CTRL+Qt.Key_N))\n\n\n def add_shortcut(self, action, desc, func, key=None):\n sc = QShortcut(self.guppy_window)\n self.combos[action] = (sc, desc)\n sc.activated.connect(func)\n if key:\n sc.setKey(key)\n\n def set_key(self, action, key):\n sc = self.combos[action][0]\n sc.setKey(key)\n\n def get_desc(self, action):\n return self.combos[action][1]\n \n @pyqtSlot()\n def nav_to_filter_text(self):\n self.guppy_window.show_hist_tab()\n self.guppy_window.historyWidget.show_filters()\n self.guppy_window.historyWidget.set_filter_is_text(True)\n self.guppy_window.historyWidget.filterWidg.entry.text_entry.textEntry.setFocus()\n\n @pyqtSlot()\n def nav_to_filter_dropdown(self):\n self.guppy_window.show_hist_tab()\n self.guppy_window.historyWidget.show_filters()\n self.guppy_window.historyWidget.set_filter_is_text(False)\n\n @pyqtSlot()\n def nav_to_filter_pop(self):\n self.guppy_window.show_hist_tab()\n self.guppy_window.historyWidget.show_filters()\n self.guppy_window.historyWidget.filterWidg.pop_phrase()\n\n @pyqtSlot()\n def nav_to_history(self):\n self.guppy_window.show_hist_tab()\n self.guppy_window.historyWidget.show_history()\n self.guppy_window.historyWidget.reqview.show_message()\n\n @pyqtSlot()\n def nav_to_tree(self):\n self.guppy_window.show_hist_tab()\n self.guppy_window.historyWidget.show_tree()\n\n @pyqtSlot()\n def nav_to_repeater(self):\n self.guppy_window.show_repeater_tab()\n\n @pyqtSlot()\n def nav_to_interceptor(self):\n self.guppy_window.show_interceptor_tab()\n\n @pyqtSlot()\n def nav_to_decoder(self):\n self.guppy_window.show_decoder_tab()\n\n @pyqtSlot()\n def nav_to_decoder_and_paste(self):\n self.guppy_window.show_decoder_tab()\n text = paste_clipboard()\n self.guppy_window.decoderWidget.decoder_input.editor.set_bytes(text.encode())\n\n @pyqtSlot()\n def open_datafile(self):\n self.guppy_window.settingsWidget.datafilewidg.open_datafile()\n \n @pyqtSlot()\n def new_datafile(self):\n self.guppy_window.settingsWidget.datafilewidg.new_datafile()\n\n @pyqtSlot()\n def nav_to_active_macros(self):\n self.guppy_window.show_active_macro_tab()\n\n @pyqtSlot()\n def nav_to_int_macros(self):\n self.guppy_window.show_int_macro_tab()\n" }, { "path": "guppyproxy/util.py", "content": "import os\nimport string\nimport time\nimport datetime\nimport random\nfrom guppyproxy.proxy import get_full_url, Headers\nfrom pygments.formatters import HtmlFormatter\nfrom pygments.styles import get_style_by_name\nfrom PyQt5.QtWidgets import QMessageBox, QMenu, QApplication, QFileDialog\nfrom PyQt5.QtGui import QColor\n\n\nstr_colorcache = {}\n\n_last_file_dialog_dir = \"\"\n_is_app = False\n\nclass DisableUpdates:\n def __init__(self, *args):\n self.prevs = [(obj, obj.updatesEnabled()) for obj in args]\n self.undoredo = []\n self.readonly = []\n for obj, _ in self.prevs:\n obj.setUpdatesEnabled(False)\n if hasattr(obj, 'setReadOnly'):\n self.undoredo.append((obj, obj.isUndoRedoEnabled()))\n obj.setUndoRedoEnabled(False)\n if hasattr(obj, 'setReadOnly'):\n self.readonly.append((obj, obj.isReadOnly()))\n obj.setReadOnly(True)\n\n def __enter__(self):\n return self\n\n def __exit__(self, exc_type, exc_value, traceback):\n for obj, prev in self.prevs:\n obj.setUpdatesEnabled(prev)\n for obj, prev in self.undoredo:\n obj.setUndoRedoEnabled(prev)\n for obj, prev in self.readonly:\n obj.setReadOnly(prev)\n\n\ndef str_hash_code(s):\n h = 0\n n = len(s) - 1\n for c in s.encode():\n h += c * 31 ** n\n n -= 1\n return h\n\n\nqtprintable = string.digits + string.ascii_letters + string.punctuation + ' ' + '\\t' + '\\n'\n\n\ndef dbgline():\n from inspect import currentframe, getframeinfo\n cf = currentframe()\n print(getframeinfo(cf.f_back).filename, cf.f_back.f_lineno)\n\n\ndef is_printable(s):\n for c in s:\n if c not in qtprintable:\n return False\n return True\n\n\ndef printable_data(data, include_newline=True):\n chars = []\n printable = string.printable\n if not include_newline:\n printable = [c for c in printable if c != '\\n']\n for c in data:\n if chr(c) in printable:\n chars.append(chr(c))\n else:\n chars.append('.')\n return ''.join(chars)\n\n\ndef max_len_str(s, ln):\n if ln <= 3:\n return \"...\"\n if len(s) <= ln:\n return s\n return s[:(ln - 3)] + \"...\"\n\n\ndef display_error_box(msg, title=\"Error\"):\n msgbox = QMessageBox()\n msgbox.setIcon(QMessageBox.Warning)\n msgbox.setText(msg)\n msgbox.setWindowTitle(title)\n msgbox.setStandardButtons(QMessageBox.Ok)\n return msgbox.exec_()\n\n\ndef display_info_box(msg, title=\"Message\"):\n msgbox = QMessageBox()\n msgbox.setIcon(QMessageBox.Information)\n msgbox.setText(msg)\n msgbox.setWindowTitle(title)\n msgbox.setStandardButtons(QMessageBox.Ok)\n return msgbox.exec_()\n\n\ndef copy_to_clipboard(s):\n QApplication.clipboard().setText(s)\n \n\ndef paste_clipboard():\n return QApplication.clipboard().text()\n\n\ndef running_as_app():\n global _is_app\n return _is_app\n\n\ndef set_running_as_app(is_app):\n global _is_app\n _is_app = is_app\n\n \ndef default_dialog_dir():\n global _last_file_dialog_dir\n if _last_file_dialog_dir != \"\":\n return _last_file_dialog_dir\n \n if running_as_app():\n return os.path.expanduser('~')\n else:\n return os.getcwd()\n\n\ndef set_default_dialog_dir(s):\n global _last_file_dialog_dir\n _last_file_dialog_dir = os.path.dirname(s)\n\n\ndef save_dialog(parent, filter_string=\"Any File (*)\", caption=\"Save File\", default_dir=None, default_name=None):\n default_dir = default_dir or default_dialog_dir()\n fname, _ = QFileDialog.getSaveFileName(parent, caption, default_dir, filter_string)\n if not fname:\n return None\n set_default_dialog_dir(os.path.abspath(fname))\n return fname\n\ndef open_dialog(parent, filter_string=\"Any File (*)\", default_dir=None):\n fname, _ = QFileDialog.getOpenFileName(parent, \"Save File\", default_dialog_dir(), filter_string)\n if not fname:\n return None\n set_default_dialog_dir(os.path.abspath(fname))\n return fname\n\n\ndef display_req_context(parent, client, req, event, repeater_widget=None, req_view_widget=None, macro_widget=None, save_option=False):\n from guppyproxy.macros import create_macro_template\n\n menu = QMenu(parent)\n repeaterAction = None\n displayUnmangledReq = None\n displayUnmangledRsp = None\n viewInBrowser = None\n macroAction = None\n saveToHistAction = None\n\n if save_option:\n saveToHistAction = menu.addAction(\"Save request to history\")\n\n if repeater_widget is not None:\n repeaterAction = menu.addAction(\"Send to repeater\")\n\n if req.unmangled and req_view_widget:\n displayUnmangledReq = menu.addAction(\"View unmangled request\")\n if req.response and req.response.unmangled and req_view_widget:\n displayUnmangledRsp = menu.addAction(\"View unmangled response\")\n\n if req.db_id != \"\":\n viewInBrowser = menu.addAction(\"View response in browser\")\n\n curlAction = menu.addAction(\"Copy as cURL command\")\n saveAction = menu.addAction(\"Save response to file\")\n saveFullActionReq = menu.addAction(\"Save request to file (full message)\")\n saveFullActionRsp = menu.addAction(\"Save response to file (full message)\")\n saveMacroAction = menu.addAction(\"Create active macro with selected requests\")\n\n if macro_widget is not None:\n macroAction = menu.addAction(\"Add to active macro input\")\n\n action = menu.exec_(parent.mapToGlobal(event.pos()))\n if save_option and action == saveToHistAction:\n client.save_new(req)\n if repeaterAction and action == repeaterAction:\n repeater_widget.set_request(req)\n if displayUnmangledReq and action == displayUnmangledReq:\n req_view_widget.set_request(req.unmangled)\n if displayUnmangledRsp and action == displayUnmangledRsp:\n new_req = req.copy()\n new_req.response = req.response.unmangled\n req_view_widget.set_request(new_req)\n if viewInBrowser and action == viewInBrowser:\n url = \"http://puppy/rsp/%s\" % req.db_id\n copy_to_clipboard(url)\n display_info_box(\"URL copied to clipboard.\\n\\nPaste the URL into the browser being proxied\")\n if action == curlAction:\n curl = curl_command(req)\n if curl is None:\n display_error_box(\"Request could not be converted to cURL command\")\n try:\n copy_to_clipboard(curl)\n except Exception:\n display_error_box(\"Error copying command to clipboard\")\n if action == saveAction:\n if not req.response:\n display_error_box(\"No response associated with request\")\n return\n fname = req.url.path.rsplit('/', 1)[-1]\n saveloc = save_dialog(parent, default_name=fname)\n if not saveloc:\n return\n with open(saveloc, 'wb') as f:\n f.write(req.response.body)\n if action == saveFullActionRsp:\n if not req.response:\n display_error_box(\"No response associated with request\")\n return\n fname = req.url.path.rsplit('/', 1)[-1] + \".response\"\n saveloc = save_dialog(parent, default_name=fname)\n if not saveloc:\n return\n with open(saveloc, 'wb') as f:\n f.write(req.response.full_message())\n if action == saveFullActionReq:\n fname = req.url.path.rsplit('/', 1)[-1] + \".request\"\n saveloc = save_dialog(parent, default_name=fname)\n if not saveloc:\n return\n with open(saveloc, 'wb') as f:\n f.write(req.full_message())\n if macroAction and action == macroAction:\n macro_widget.add_requests([req])\n if action == saveMacroAction:\n saveloc = save_dialog(parent, default_name=\"macro.py\")\n if saveloc == None:\n return\n with open(saveloc, 'w') as f:\n f.write(create_macro_template([req]))\n\ndef display_multi_req_context(parent, client, reqs, event, macro_widget=None, save_option=False):\n from guppyproxy.macros import create_macro_template\n\n menu = QMenu(parent)\n if macro_widget:\n macroAction = menu.addAction(\"Add to active macro input\")\n if save_option:\n saveAction = menu.addAction(\"Save requests to history\")\n saveMacroAction = menu.addAction(\"Create active macro with selected requests\")\n action = menu.exec_(parent.mapToGlobal(event.pos()))\n if macro_widget and action == macroAction:\n if macro_widget:\n macro_widget.add_requests(reqs)\n if save_option and action == saveAction:\n for req in reqs:\n client.save_new(req)\n if action == saveMacroAction:\n saveloc = save_dialog(parent, default_name=\"macro.py\")\n if saveloc == None:\n return\n with open(saveloc, 'w') as f:\n f.write(create_macro_template(reqs))\n\ndef method_color(method):\n if method.lower() == 'get':\n return QColor(240, 240, 255)\n\n if method.lower() == 'post':\n return QColor(255, 255, 230)\n\n if method.lower() == 'put':\n return QColor(255, 240, 240)\n\n return QColor(255, 255, 255)\n\ndef sc_color(sc):\n if sc[0] == '2':\n return QColor(240, 255, 240)\n\n if sc[0] == '3':\n return QColor(255, 240, 255)\n\n if sc[0] == '4':\n return QColor(255, 240, 240)\n\n if sc[0] == '5':\n return QColor(255, 255, 230)\n\n return QColor(255, 255, 255)\n\ndef host_color(hostport):\n return str_color(hostport, lighten=150, seed=1)\n\ndef str_color(s, lighten=0, seed=0):\n global str_colorcache\n if s in str_colorcache:\n return str_colorcache[s]\n hashval = str_hash_code(s)+seed\n gen = random.Random()\n gen.seed(hashval)\n r = gen.randint(lighten, 255)\n g = gen.randint(lighten, 255)\n b = gen.randint(lighten, 255)\n\n col = QColor(r, g, b)\n str_colorcache[s] = col\n return col\n\ndef hostport(req):\n # returns host:port if to a port besides 80 or 443\n host = req.dest_host\n if req.use_tls and req.dest_port == 443:\n return host\n if (not req.use_tls) and req.dest_port == 80:\n return host\n return \"%s:%d\" % (host, req.dest_port)\n\n\ndef _sh_esc(s):\n sesc = s.replace(\"\\\\\", \"\\\\\\\\\")\n sesc = sesc.replace(\"\\\"\", \"\\\\\\\"\")\n return sesc\n\n\ndef curl_command(req):\n # Creates a curl command that submits a given request\n command = \"curl\"\n if req.method != \"GET\":\n command += \" -X %s\" % req.method\n for k, v in req.headers.pairs():\n if k.lower == \"content-length\":\n continue\n kesc = _sh_esc(k)\n vesc = _sh_esc(v)\n command += ' --header \"%s: %s\"' % (kesc, vesc)\n if req.body:\n if not is_printable(req.body):\n return None\n besc = _sh_esc(req.body)\n command += ' -d \"%s\"' % besc\n command += ' \"%s\"' % _sh_esc(get_full_url(req))\n return command\n\n\ndef list_remove(lst, inds):\n return [i for j, i in enumerate(lst) if j not in inds]\n\n\ndef hexdump(src, length=16):\n FILTER = ''.join([(len(repr(chr(x))) == 3) and chr(x) or '.' for x in range(256)])\n lines = []\n for c in range(0, len(src), length):\n chars = src[c:c + length]\n hex = ' '.join([\"%02x\" % x for x in chars])\n printable = ''.join([\"%s\" % ((x <= 127 and FILTER[x]) or '.') for x in chars])\n lines.append(\"%04x %-*s %s\\n\" % (c, length * 3, hex, printable))\n return ''.join(lines)\n\n\ndef confirm(message, default='n'):\n \"\"\"\n A helper function to get confirmation from the user. It prints ``message``\n then asks the user to answer yes or no. Returns True if the user answers\n yes, otherwise returns False.\n \"\"\"\n if 'n' in default.lower():\n default = False\n else:\n default = True\n\n print(message)\n if default:\n answer = input('(Y/n) ')\n else:\n answer = input('(y/N) ')\n\n if not answer:\n return default\n\n if answer[0].lower() == 'y':\n return True\n else:\n return False\n\n\n# Taken from http://stackoverflow.com/questions/4770297/python-convert-utc-datetime-string-to-local-datetime\ndef utc2local(utc):\n epoch = time.mktime(utc.timetuple())\n offset = datetime.datetime.fromtimestamp(epoch) - datetime.datetime.utcfromtimestamp(epoch)\n return utc + offset\n\n\ndef datetime_string(dt):\n dtobj = utc2local(dt)\n time_made_str = dtobj.strftime('%a, %b %d, %Y, %I:%M:%S.%f %p')\n return time_made_str\n\n\ndef query_to_str(query):\n retstr = \"\"\n for p in query:\n fstrs = []\n for f in p:\n fstrs.append(' '.join(f))\n\n retstr += (' OR '.join(fstrs))\n return retstr\n\n\ndef textedit_highlight(text, lexer):\n from pygments import highlight\n wrapper_head = \"\"\"
\"\"\"\n    wrapper_foot = \"
\"\n highlighted = highlight(text, lexer,\n HtmlFormatter(noclasses=True, style=get_style_by_name(\"colorful\"), nowrap=True))\n highlighted = wrapper_head + highlighted + wrapper_foot\n return highlighted\n" }, { "path": "install.sh", "content": "#!/bin/bash\n\nprompt_yn() {\n read -p \"$1 (yN) \" yn;\n case $yn in\n [Yy]* ) return 0;;\n * ) return 1;;\n esac\n}\n\nrequire() {\n if ! $@; then\n echo \"Error running $@, exiting...\";\n exit 1;\n fi\n}\n\nGO=\"$(which go)\"\nBUILDFLAGS=\"\"\nPUPPYREPO=\"https://github.com/roglew/puppy.git\"\nPUPPYVERSION=\"tags/0.2.6\"\n\nINSTALLDIR=\"$( cd \"$( dirname \"${BASH_SOURCE[0]}\" )\" && pwd )\"\nTMPGOPATH=\"$INSTALLDIR/gopath\"\nDATADIR=\"$HOME/.guppy\"\nVIRTUALENVNAME=\"guppyenv\"\n\nwhile getopts \"g:f:r:dph\" opt; do\n case $opt in\n g)\n GO=\"$OPTARG\"\n ;;\n f)\n BUILDFLAGS=\"${OPTARG}\"\n ;;\n r)\n PUPPYREPO=\"${OPTARG}\"\n DEV=\"yes\"\n ;;\n d)\n DEV=\"yes\"\n ;;\n p)\n DOPUPPY=\"yes\"\n ;;\n h)\n echo -e \"Build script flags:\"\n echo -e \"-p\\tCompile puppy from source rather than using pre-built binaries\"\n echo -e \"-g [path to go]\\tUse specific go binary to compile puppy\"\n echo -e \"-f [arguments]\\tArguments to pass to \\\"go build\\\". ie -f \\\"-ldflags -s\\\"\"\n echo -e \"-r [git repository link]\\t download puppy from an alternate repository\"\n echo -e \"-d\\tinstall puppy in development mode by using \\\"pip install -e\\\" to install puppy\"\n echo -e \"-h\\tprint this help message\"\n echo -e \"\"\n exit 0;\n ;;\n\n \\?)\n echo \"Invalid option: -$OPTARG\" >&2\n exit 1;\n ;;\n esac\ndone\n\nif ! type \"python3\" > /dev/null; then\n echo \"python3 not installed. Please install python3 and try again\"\n exit 1;\nfi\n\nif ! type \"pip\" > /dev/null; then\n if ! type \"easy_install\" > /dev/null; then\n echo \"pip not available. Please install pip then try again.\"\n exit 1;\n fi\n\n if prompt_yn \"Installation requires pip. Install pip using \\\"sudo easy_install pup\\\"?\"; then\n require sudo easy_install pip;\n else\n echo \"Please install pip and try the installation again\"\n exit 1;\n fi\nfi\n\ncd \"$INSTALLDIR\"\nmkdir -p $DATADIR\n\nif [ $DOPUPPY ]; then\n # Compile puppy from source\n\n if [ ! $GO ]; then\n if ! type \"go\" > /dev/null; then\n echo \"go not installed. Please install go and try again\"\n exit 1;\n fi\n fi\n\n # Set up fake gopath\n export GOPATH=\"$TMPGOPATH\";\n require mkdir -p \"$GOPATH/src\"\n\n # Clone the repo\n REPODIR=\"$GOPATH/src/puppy\";\n if [ ! -d \"$REPODIR\" ]; then\n # Clone the repo if it doesn't exist\n require mkdir -p \"$REPODIR\";\n echo git clone \"$PUPPYREPO\" \"$REPODIR\";\n require git clone \"$PUPPYREPO\" \"$REPODIR\";\n fi\n \n # Check out the correct version\n cd \"$REPODIR\";\n if [ $DEV ] || [ $REPODIR ]; then\n # If it's development, get the most recent version of puppy\n require git pull;\n else\n # if it's not development, get the specified version\n require git checkout \"$PUPPYVERSION\";\n fi\n cd \"$INSTALLDIR\"\n \n # Get dependencies\n cd \"$REPODIR\";\n echo \"Getting puppy dependencies...\"\n require \"$GO\" get ./...;\n \n # Build puppy into the data dir\n echo \"Building puppy into $DATADIR/puppy...\";\n require mkdir -p \"$DATADIR\";\n require \"$GO\" build -o \"$DATADIR\"/puppy $BUILDFLAGS \"puppy/cmd/main\";\nelse\n # copy the pre-compiled binary\n UNAME=\"$(uname -s)\"\n PUPPYFILE=\"\"\n if [ \"$UNAME\" = \"Darwin\" ]; then\n echo \"copying mac version of pre-built puppy to $DATADIR/puppy\"\n PUPPYFILE=\"puppy.osx\"\n elif [ \"$UNAME\" = \"Linux\" ]; then\n if [ \"$(uname -m)\" = \"x86_64\" ]; then\n echo \"copying 64-bit linux version of pre-built puppy to $DATADIR/puppy\"\n PUPPYFILE=\"puppy.linux64\"\n else\n echo \"copying 32-bit linux version of pre-built puppy to $DATADIR/puppy\"\n PUPPYFILE=\"puppy.linux32\"\n fi\n else\n echo \"could not detect system type. Please use -p to compile puppy from source (requires go installation)\"\n exit 1;\n fi\n cp \"$INSTALLDIR/puppyrsc/$PUPPYFILE\" \"$DATADIR/puppy\"\nfi\n\n# Clear out old .pyc files\nrequire find \"$INSTALLDIR/guppyproxy\" -iname \"*.pyc\" -exec rm -f {} \\;\n\n# Set up the virtual environment\nif ! type \"virtualenv\" > /dev/null; then\n if prompt_yn \"\\\"virtualenv\\\" not installed. Install using pip?\"; then\n require sudo pip install virtualenv\n else\n exit 1;\n fi\nfi\n\nVENVDIR=\"$DATADIR/venv\";\nrequire mkdir -p \"$VENVDIR\";\nrequire virtualenv -p \"$(which python3)\" \"$VENVDIR\";\ncd \"$VENVDIR\";\nrequire source bin/activate;\ncd \"$INSTALLDIR\";\n\nif [ -z $DEV ]; then\n require pip install -e .\nelse\n require pip install .\nfi\n\necho -e \"#!/bin/bash\\nsource \\\"$VENVDIR/bin/activate\\\";\\nguppy \\$@ || killall puppy;\\n\" > start\nchmod +x start;\n\necho \"\"\necho \"Guppy installed. Run guppy by executing the generated \\\"start\\\" script.\"\n" }, { "path": "puppyrsc/NOTE.md", "content": "These binaries are pre-built versions of puppy () which is used to proxy HTTP requests as they pass through the proxy. By default, installation will use these pre-built binaries. If you would like to compile the binary yourself, ensure you have a compatible go installation and use the `-p` flag with the install script.\n" }, { "path": "setup.py", "content": "#!/usr/bin/env python\n\nimport pkgutil\nfrom setuptools import setup, find_packages\ntry:\n import py2app\nexcept ImportError:\n pass\n\nVERSION = \"0.1.0\"\n\nsetup(name='GuppyProxy',\n version=VERSION,\n description='The Guppy Intercepting Proxy',\n author='Rob Glew',\n author_email='rglew56@gmail.com',\n packages=['guppyproxy'],\n app=['guppyproxy/gup.py'],\n include_package_data = True,\n license='MIT',\n options={'py2app': {\n 'packages': ['lxml','pygments','PyQt5'],\n 'iconfile': 'img/shark.icns',\n 'resources': ['puppyrsc'],\n }\n },\n entry_points = {\n 'console_scripts':['guppy = guppyproxy.gup:start'],\n },\n long_description=\"The Guppy Proxy\",\n keywords='http proxy hacking 1337hax pwnurmum',\n install_requires=[\n 'lxml>=4.1.1',\n 'Pygments>=2.0.2',\n 'PyQt5>=5.9',\n ],\n classifiers=[\n 'Intended Audience :: Developers',\n 'Intended Audience :: Information Technology',\n 'Operating System :: MacOS',\n 'Operating System :: POSIX :: Linux',\n 'Development Status :: 2 - Pre-Alpha',\n 'Programming Language :: Python :: 3.6',\n 'License :: OSI Approved :: MIT License',\n 'Topic :: Security',\n ]\n)\n" } ]