Showing preview only (254K chars total). Download the full file or copy to clipboard to get everything.
Repository: roglew/guppy-proxy
Branch: master
Commit: 01df16be71dd
Files: 26
Total size: 30.2 MB
Directory structure:
gitextract_l9ewhzbc/
├── .gitignore
├── LICENSE
├── README.md
├── guppyproxy/
│ ├── __init__.py
│ ├── config.py
│ ├── decoder.py
│ ├── gui.py
│ ├── gup.py
│ ├── hexteditor.py
│ ├── interceptor.py
│ ├── macros.py
│ ├── proxy.py
│ ├── repeater.py
│ ├── reqlist.py
│ ├── reqtree.py
│ ├── reqview.py
│ ├── settings.py
│ ├── shortcuts.py
│ └── util.py
├── img/
│ └── shark.icns
├── install.sh
├── puppyrsc/
│ ├── NOTE.md
│ ├── puppy.linux32
│ ├── puppy.linux64
│ └── puppy.osx
└── setup.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
*.pyc
data.db
.coverage
.cache
tests/.cache
.DS_Store
TAGS
config.json
build/*
*.egg-info/*
.#*
*notes*
*.org
start
================================================
FILE: LICENSE
================================================
MIT License
Copyright (c) 2018 Rob Glew
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: README.md
================================================
# Guppy Proxy
The Guppy Proxy is an intercepting proxy for performing web application security testing. Its features are often similar to, or straight up rippoffs from [Burp Suite](https://portswigger.net/burp/). However, Burp Suite has its own issues (search, licensing) which led to the creation of Guppy.
# Installation
## Dependencies
Make sure the following commands are available:
* `python3`
* `pip`
* `virtualenv` (can be installed with pip)
## Installing
### Mac
1. Download the .app of version of guppy [available here](https://guppydist.s3-us-west-2.amazonaws.com/GuppyProxy-0.0.15.zip)
1. Start the application
1. Add the CA cert in `~/.guppy/certs` to your browser as a CA
1. Configure your browser to use `localhost:8080` as a proxy
1. Navigate to a site and look at the history in the main window
### Linux / Alternative for Mac
1. Clone this repo somewhere it won't get deleted: `git clone https://github.com/roglew/guppy-proxy.git`
1. `cd /path/to/guppy-proxy`
1. `./install.sh` to use pre-built binary or `./install.sh -p` to compile the go component from source (requires a [go installation](https://golang.org/doc/install))
1. Test that the application starts up and generate certs: `./start` (keep the window open and continue to test it works)
1. Copy/symlink the generated `start` script somewhere in your PATH (i.e. `~/bin` if you have that included) and rename it to `guppy` if you want
1. Add the CA cert in `~/.guppy/certs` to your browser as a CA
1. Configure your browser to use `localhost:8080` as a proxy
1. Navigate to a site and look at the history in the main window
## Updating
1. Navigate to the guppy-proxy folder with this repo in it
1. `git pull` to pull the latest version
1. run `./install.sh` again
The same start script as before should still work
## Uninstalling
1. Delete the guppy-proxy directory you made during installation
1. Delete `~/.guppy`
1. Remove the start script from wherever you put it
# How to Use Guppy
## History View
The first thing you see when you open Guppy is the history view. As requests pass through the proxy they are displayed in the lower half of the window. You can click a request to view the full request/response in the windows on the upper half or right click them for more options. The tabs on the upper half will let you view additional information about the selected request:
* Messages - The full request/response
* Info - A list of values associated with the message
* Tags - Lets you view/edit the tags currently associated with the request
The bottom half has tabs which relate to all of the requests that have been recorded by the proxy:
* List - A list of all of the requests that have been recorded by the proxy
* Tree - A site map of all of the endpoints visited
* Filters - An advanced search interface which is described below in the Filters section
## Filters and Search
Guppy's main selling point over other similar proxies is its search. You can search for a wide variety of fields within a request or response and apply more than one search condition at the same time. This allows you to perform complex searches over your history so that you can always find the request that you want. You would be surprised what you can find when searching for paths, headers, and body contents. For example you can find potential CSRF targets by finding requests which are not GET requests and also do not have a header with "CSRF" in it.
How to apply a filter to your search:
1. Select the field you want to search by
1. Select how you want to search it (whether it contains a value, matches a regexp, is an exact value, etc)
1. Enter the value to search by in the text box
1. Click "Ok" or press enter in the text box
Once you apply a filter, the "list" and "tree" tabs will only include requests which match ALL of the active filters.
In addition, you can apply different filters for the key and value of key/value fields (such as headers or request parameters). This can be done by:
1. Select a key/value field such as "Rsp. Header" or "URL Param"
1. Click the "+" button on the right
1. Enter the filter for the key on the left and the filter for the value on the right
1. Click "Ok" or press enter in one of the text boxes
And that's it! The filter tab has the following additional controls:
1. Clear - Delete all active filters
1. Pop - Delete the most recent filter
1. Scope - Set the active search to your project's scope (see below)
1. Save Scope - Set your project's scope to the currently active filters (see below)
1. Apply a built-in filter dropdown - Guppy has a list of commonly useful filters. Select one from this list to apply it
### Text Filter Entry
Along with the provided dropdowns you can manually type in a filter by clicking the `>` button. In some cases it may be faster to type your filter out rather than clicking on dropdowns. In addition it allows you to create filter statements that contain an `OR` and will pass a request that matches any one of the given filters. In fact, all the dropdown input does is generate these strings for you.
Most filter strings have the following format:
```
<field> <comparer> <value>
```
Where `<field>` is some part of the request/response, `<comparer>` is some comparison to `<value>`. For example, if you wanted a filter that only matches requests to `target.org`, you could use the following filter string:
```
host is target.org
field = "host"
comparer = "is"
value = "target.org"
```
For fields that are a list of key/value pairs (headers, get params, post params, and cookies) you can use the following format:
```
<field> <comparer1> <value1>[ <comparer2> <value2>]
```
This is a little more complicated. If you don't give comparer2/value2, the filter will pass any pair where the key or the value matches comparer1 and value1. If you do give comparer2/value2, the key must match comparer1/value1 and the value must match comparer2/value2 For example:
```
Filter A:
cookie contains Session
Filter B:
cookie contains Session contains 456
Filter C:
inv cookie contains Ultra
Cookie: SuperSession=abc123
Matches A and C but not B
Cookie: UltraSession=abc123456
Matches both A and B but not C
```
#### List of fields
| Field Name | Aliases | Description | Format |
|:--------|:------------|:-----|:------|
| all | all | Anywhere in the request, response, or a websocket message | String |
| reqbody | reqbody, reqbd, qbd, qdata, qdt | The body of the request | String |
| rspbody | rspbody, rspbd, sbd, sdata, sdt | The body of the response | String |
| body | body, bd, data, dt | The body in either the request or the response | String |
| wsmessage | wsmessage, wsm | In a websocket message | String |
| method | method, verb, vb | The request method (GET, POST, etc) | String |
| host | host, domain, hs, dm | The host that the request was sent to | String |
| path | path, pt | The path of the request | String |
| url | url | The full URL of the request | String |
| statuscode | statuscode, sc | The status code of the response (200, 404, etc) | String |
| tag | tag | Any of the tags of the request | String |
| reqheader | reqheader, reqhd, qhd | A header in the request | Key/Value |
| rspheader | rspheader, rsphd, shd | A header in the response | Key/Value |
| header | header, hd | A header in the request or the response | Key/Value |
| param | param, pm | Either a URL or a POST parameter | Key/Value |
| urlparam | urlparam, uparam | A URL parameter of the request | Key/Value |
| postparam | postparam, pparam | A post parameter of the request | Key/Value |
| rspcookie | rspcookie, rspck, sck | A cookie set by the response | Key/Value |
| reqcookie | reqcookie, reqck, qck | A cookie submitted by the request | Key/Value |
| cookie | cookie, ck | A cookie sent by the request or a cookie set by the response | Key/Value |
#### List of comparers
| Field Name | Aliases | Description |
|:--------|:------------|:-----|
| is | is | Exact string match |
| contains | contains, ct | A contain B is true if B is a substring of A |
| containsr | containsr, ctr | A containr B is true if A matches regexp B |
| leneq | leneq | A Leq B if A's length equals B (B must be a number) |
| lengt | lengt | A Lgt B if A's length is greater than B (B must be a number ) |
| lenlt | lenlt | A Llt B if A's length is less than B (B must be a number) |
#### Special form filters
A few filters don't conform to the field, comparer, value format. You can still negate these.
| Format | Aliases | Description |
|:--|:--|:--|
| invert <filter string> | invert, inv | Inverts a filter string. Anything that matches the filter string will not pass the filter. |
Examples:
```
Show state-changing requests
inv method is GET
Show requests without a csrf parameter
inv param ct csrf
```
#### Using OR
If you want to create a filter that will pass a request if it matches any of one of a few filters you can create `OR` statements. This is done by entering in each filter on the same line and separating them with an `OR` (It's case sensitive!).
Examples:
```
Show requests to target.org or example.com:
host is target.org OR host is example.com
Show requests that either are to /foobar or have foobar in the response or is a 404
path is /foobar OR sbd ct foobar OR sc is 404
```
### Scope
The scope of your project describes which requests should be recorded as they pass through the proxy. Guppy allows you to define a set of filters which describe which requests are in scope. For example, if your scope is just `host ctr example.com$` only requests to example.com will be recorded in history.
To set the scope of your project:
1. Enter the filters you want to be your scope
1. Press the "Save Scope" button
And you're done! Requests that do not match this set of filters will no longer be saved. You can also set your current search to your scope by clicking the "Scope" button. The scope can be deleted by pressing the "Clear" button to delete all active filters and then clicking "Save Scope".
# Repeater
The repeater lets you repeatedly tweak and submit a request. You can use a request in the repeater by:
1. Find the request you which to resubmit in the history list view
1. Right click the request and click "Send to Repeater"
1. Navigate to the repeater tab
1. Edit the request on the left
1. Click the submit button
When you click submit:
* The request will be submitted
* The request and response will be saved in history
* Any tags under the "tag" tab will be applied to the request
# Interceptor
The interceptor lets you edit requests and responses as they pass through the proxy. To use this:
1. Navigate to the interceptor tab
1. Click the "Int. Requests" and/or the "Int. Responses" buttons
1. Wait for a request to pass through the proxy
1. Edit the message in the text box then click "Forward" to forward the edited message or click "Cancel" to just drop the message altogether
# Decoder
The decoder allows you to perform common encoding/decoding actions. You use the decoder by:
1. Paste the data that you want to encode/decode
1. Select how you wish to encode/decode it
1. Press "Go!"
The text will be processed and it will appear in the same text box. Easy!
# Macros
Guppy includes support for loading and executing Python scripts in order to allow for more complex attacks than can be performed by hand with the repeater. It is worth noting that **this feature is not user friendly. Use it at your own risk.** No attempt is made to make this feature user-friendly, stable, or good. The main reason it exists is to make it easier to write python scripts which integrate with Guppy history and to provide some way to extend Guppy's features without a pull request. If you haven't been scared away yet, read on.
There are two types of macros that you can write:
* Active macros: Take requests as an input, make more requests, edit the input requests, etc, then output a new set of requests for review
* Intercepting macros: Modify requests and responses as they pass through the proxy
Most features that you want will fall into one of those categories. Both macros are created by creating a `.py` file and defining specific functions which will be run when the macro is executed. For example an active macro must define `run_macro` and an intercepting macro must define `mangle_request` and/or `mangle_response`. See their respective sections below for more details.
## The API
Unfortunately since this feature was pretty much just thrown together for my own use, the documentation is looking at the source. Hopefully it will become more stable once Guppy development slows down, but for now you'll have to look at the relevant classes to figure out how to do stuff on your own. The following classes are the most important when writing a macro:
### MacroClient
`MacroClient` is defined in `(guppyproxy/macro.py)` and is the interface that macros use to submit requests, save requests to history, and produce output. At the time of writing the class provides:
```
MacroClient.submit(req, save=False): Submits a request to the server and sets req.response to the response. req is the HTTPRequest to submit. req.dest_host, req.dest_port, and req.use_tls will be used to determine the location to submit the request to
MacroClient.save(req): Permenantly saves an HTTPRequest to history
MacroClient.output(s): Prints a string to the output tab in the macros interface
MacroClient.output_req(req): Adds a request to the output request table in the macros interface
MacroClient.new_request(method="GET", path="/", proto_major=1, proto_minor=1,
headers=None, body=bytes(), dest_host="", dest_port=80,
use_tls=False, tags=None): Creates a new HTTPRequest from scratch that can be submitted with the client
```
### HTTPRequest and HTTPResponse
`HTTPRequest` and `HTTPResponse` are defined in `guppyproxy/proxy.py`. These classes represent HTTP messages. `HTTPRequest` contains both the contents of the message and information about its intended destination (host, port, whether to use TLS). Below are a few examples on how to use these classes, however for more deails you will need to consult `proxy.py`:
```python
req = HTTPRequest()
rsp = HTTPResponse()
req2 = req.copy() # Copy a request
rsp2 = rsp.copy() # Copy a response
# Refer to the messages associated with a messages
rsp3 = req.response # Response to a request, will be `None` if there was no response
unm = req.unmangled # Unmangled version of a request, is `None` if none exist
unm2 = rsp.unmangled # Unmangled version of a response, is `None` if none exist
# Get the full message of an object (is a bytes())
full_req = req.full_message()
full_rsp = rsp.full_message()
# Get timing info for a request
tstart = req.time_start # datetime.datetime when the request was made
tend = req.time_end # datetime.datetime when the request's response was received
# Get destination info from a request
dest_host = req.dest_host
dest_port = req.dest_port
use_tls = req.use_tls
# Get/set the method of a request
m = req.method # Get the method of the request
req.method = "POST" # Set the method of the request
# Get/set url info of a request
requrl = req.full_url() # get the full URL of a request
path = req.url.path # get the path of a request
req.url.path = "/foo/bar/baz" # set the path of a request
v = req.url.get_param("foo") # get the value of the "foo" URL parameter
req.url.set_param("foo", "bar") # set the value of the "foo" URL parameter to "bar"
req.url.add_param("foo", "bar2") # add a URL parameter allowing duplicates
req.url.del_param("foo") # delete a url parameter
[(k, v) for k, v in req.url.param_iter] # iterate over all the key/value pairs in the URL parameters
frag = req.url.fragment # get the fragment of the url (the bit after the #)
req.url.fragment = "frag" # set the url fragment of the request
# Manage headers in a message
req.headers.set("Foo", "Bar") # set a header, repalcing existing value
hd = req.headers.get("Foo") # get the value of a header (for duplicates, returns first value)
req.headers.add("Foo", "Bar2") # add a header without replacing an existing one
pairs = req.headers.pairs() # returns all the key/value pairs of the headers in the message
req.headers.delete("Foo") # delete a header
req.headers.dict() # Returns a dict of the headers in the form of {"key1": ["val1", "val2"], "key2": ["val3", "val4"]}
# Same for responses
rsp.headers.set("Foo", "Bar")
hd = rsp.headers.get("Foo")
rsp.headers.add("Foo", "Bar2")
pairs = rsp.headers.pairs()
rsp.headers.delete("Foo")
rsp.headers.dict()
# Manage body of a message
req.body = "foo=bar" # set the body of the message to a string
req.body = b"\x01\x02\x03" # set the body to bytes
bd = req.body # Get the value of the body (always is bytes())
# Same for responses
rsp.body = "foo=bar"
rsp.body = b"\x01\x02\x03"
bd = rsp.body
# Manage POST parameters of a request
params = req.parameters() # Returns a dict of the POST parameters in the form of {"key1": ["val1", "val2"], "key2": ["val3", "val4"]}
[(k, v) for k, v in req.param_iter()] # Iterate through all the key/value pairs of the request parameters
req.set_param("Foo", "Bar") # Set the "Foo" parameter to "Bar"
req.add_param("Foo", "Bar2") # Add a POST parameter to the request allowing duplicates
req.del_param("Foo") # Delete a parameter from the request
# NOTE: Setting a POST parameter will not change the request method to POST
# Managing the cookies of a message
cookie = req.cookies() # Returns an http.cookies.BaseCookie representing the request's cookies
req.set_cookie("foo", "bar") # set a cookie in the request
req.del_cookie("foo") # delete a cookie from the request
[(k, v) for k, v in req.cookie_iter()] # Iterate over the key/value pairs of the cookies in a request
req.set_cookies({"cookie1": "val1", "cookie2": "val2"}) # Set the cookies in the request
req.set_cookies(req2) # Set the requests on req to the cookies in req2
req.add_cookies({"cookie1": "val1", "cookie2": "val2"}) # Add cookies to the request replacing existing values
req.add_cookies(req2) # Add cookies from req2 to the request replacing existing values
# Same for responses
cookie = rsp.cookies()
rsp.set_cookie("foo", "bar")
rsp.del_cookie("foo")
[(k, v) for k, v in rsp.cookie_iter()]
rsp.set_cookies({"cookie1": "val1", "cookie2": "val2"})
rsp.set_cookies(rsp2)
rsp.add_cookies({"cookie1": "val1", "cookie2": "val2"})
rsp.add_cookies(rsp2)
# Manage tags of a request
hastag = ("tagname" in req.tags) # check if a request has a tag
req.tags.add("tagname") # add a tag to a request
req.tags.remove("tagname") # remove a tag from the request
# NOTE: req.tags is a regular set() and you can do whatever you want to it
```
## Macro Arguments
Both active and intercepting macros can optionally have Guppy prompt for a set of arguments before running. These arguments will be passed as a dict in the `args` variable when calling the relevant function. A macro can request arguments by defining a `get_args` function and returning a list of strings. For example if a macro defines the following `get_args` function:
```python
def get_args():
return ["foo", "bar"]
```
the proxy will prompt for values for foo and bar. If the user enters "FOOARG" and "BARARG" for the values `args` will have a value of:
```python
{"foo": "FOOARG", "bar": "BARARG"}
```
See below for examples on how to use arguments in macros. If `get_args` is not defined, `None` will be passed in for `args`.
## Active Macros
Active macros are a Python script that define a `run_macro` function that takes in two arguments. A `MacroClient` (as defined in `guppyproxy/macros.py`) and a list of requests (`HTTPRequest` and `HTTPResponse` are defined in `guppyproxy/proxy.py`). The following is an example of a macro that resubmits all of the input requests but adds a new header:
```python
# addheader.py
def get_args():
return ["header_key", "header_val"]
def run_macro(client, args, reqs):
for req in reqs:
client.output("Submitting request to %s..." % req.full_url())
req.headers.set(args["header_key"], args["header_val"])
client.submit(req)
client.output_req(req)
```
Macros such as this can be used for things such as testing auth controls or brute forcing paths/filenames.
## Intercepting Macros
Intercepting macros are used to look at/modify requests as they pass through the proxy. This is done by defining `mangle_request` and/or `mangle_response`:
```
mangle_request(client, req): Takes in a client and an HTTPRequest and returns an HTTPRequest. The returned HTTPRequest will be sent to the server instead of the original.
mangle_response(client, req, rsp): Takes in a client, HTTPRequest, and HTTResponse and returns an HTTPResponse. The returned HTTPResponse will be sent to the browser instead of the original.
```
As an example, the following macro will ask for a find/replace value. When run, it will set the `session` cookie in the request to `bar` before submitting it to the server and then perform the given find and replace on the body of the response.
```python
# intexample.py
def get_args():
return ["find", "replace"]
def mangle_request(client, args, req):
req.set_cookie("session", "bar")
return req
def mangle_response(client, args, req, rsp):
rsp.body = rsp.body.replace(args['find'].encode(), args['replace'].encode())
return rsp
```
# Settings
This tab allows you to edit your proxy settings. It lets you select a file to store your history/settings in and configure what ports the proxy listens on. It also allows you to configure an upstream proxy to use. You can add a listener by entering the interface and port into the text boxes and clicking the "+" button. They can be deleted by selecting them from the list and clicking the "-" button.
You can also specify settings for an upstream proxy by checking the "Use Proxy" box, filling out the appropriate info, and clicking "confirm".
## Data Files
Your entire request history and your settings can be stored in a data file on disk. This allows you to save your work for later and even send your work to someone else. You can start a new project with a new data file by clicking the "New" button in the settings tab. Once you do this, your settings, scope, and all the messages that pass through the proxy will be saved to the specified file. You can also load an existing project by using the "Open" button. Finally, you can specify a data file by typing the path into the text box and clicking "Go!"
# Keybindings
Guppy has the following keybindings:
| Key | Action |
|:--------|:------------|
| `Ctrl+J` | Navigate to request list |
| `Ctrl+T` | Navigate to tree view |
| `Ctrl+R` | Navigate to repeater |
| `Ctrl+N` | Navigate to interceptor |
| `Ctrl+D` | Navigate to decoder |
| `Ctrl+U` | Navigate to filter text input |
| `Ctrl+I` | Navigate to filter dropdown input |
| `Ctrl+P` | Navigate to filters and pop most recent filter |
| `Ctrl+Shift+D` | Navigate to decoder and fill with clipboard |
| `Ctrl+Shift+N` | Create new datafile |
| `Ctrl+Shift+O` | Open existing datafile |
================================================
FILE: guppyproxy/__init__.py
================================================
================================================
FILE: guppyproxy/config.py
================================================
import copy
import json
default_config = """{
"listeners": [
{"iface": "127.0.0.1", "port": 8080}
],
"proxy": {"use_proxy": false, "host": "", "port": 0, "is_socks": false}
}"""
class ProxyConfig:
PLUGIN_KEY = "guppy_config"
def __init__(self):
self._listeners = [('127.0.0.1', 8080, None)]
self._proxy = {'use_proxy': False, 'host': '', 'port': 0, 'is_socks': False}
def loads(self, js):
config_info = json.loads(js)
self._set_config(config_info)
def dumps(self):
listeners = []
for l in self._listeners:
listener = {"host": l[0], "port": l[1]}
listeners.append(listener)
_config_info = {"listeners": listeners,
"proxy": self._proxy}
return json.dumps(_config_info)
def load(self, fname):
try:
with open(fname, 'r') as f:
config_info = json.loads(f.read())
except IOError:
config_info = json.loads(default_config)
with open(fname, 'w') as f:
f.write(default_config)
self._set_config(config_info)
def _set_config(self, config_info):
# Listeners
if 'listeners' in config_info:
self._parse_listeners(config_info['listeners'])
if 'proxy' in config_info:
self._proxy = config_info['proxy']
def _parse_listeners(self, listeners):
self._listeners = []
for info in listeners:
if 'port' in info:
port = info['port']
else:
port = 8080
if 'interface' in info:
iface = info['interface']
elif 'iface' in info:
iface = info['iface']
else:
iface = '127.0.0.1'
if "transparent" in info:
trans_info = info['transparent']
transparent_dest = (trans_info.get('host', ""),
trans_info.get('port', 0),
trans_info.get('use_tls', False))
else:
transparent_dest = None
self._listeners.append((iface, port, transparent_dest))
def set_listeners(self, listeners):
self._listeners = listeners
@property
def listeners(self):
return copy.deepcopy(self._listeners)
@listeners.setter
def listeners(self, val):
self._parse_listeners(val)
@property
def proxy(self):
# don't use this, use the getters to get the parsed values
return self._proxy
@proxy.setter
def proxy(self, val):
self._proxy = val
@property
def use_proxy(self):
if self._proxy is None:
return False
if 'use_proxy' in self._proxy:
if self._proxy['use_proxy']:
return True
return False
@property
def proxy_host(self):
if self._proxy is None:
return ''
if 'host' in self._proxy:
return self._proxy['host']
return ''
@property
def proxy_port(self):
if self._proxy is None:
return ''
if 'port' in self._proxy:
return self._proxy['port']
return ''
@property
def proxy_username(self):
if self._proxy is None:
return ''
if 'username' in self._proxy:
return self._proxy['username']
return ''
@property
def proxy_password(self):
if self._proxy is None:
return ''
if 'password' in self._proxy:
return self._proxy['password']
return ''
@property
def use_proxy_creds(self):
return ('username' in self._proxy or 'password' in self._proxy)
@property
def is_socks_proxy(self):
if self._proxy is None:
return False
if 'is_socks' in self._proxy:
if self._proxy['is_socks']:
return True
return False
================================================
FILE: guppyproxy/decoder.py
================================================
import html
import base64
import urllib
import json
from guppyproxy.util import display_error_box
from guppyproxy.hexteditor import ComboEditor
from PyQt5.QtWidgets import QWidget, QHBoxLayout, QVBoxLayout, QComboBox, QPlainTextEdit, QPushButton
from PyQt5.QtCore import pyqtSlot, pyqtSignal
from datetime import datetime
class DecodeError(Exception):
pass
def asciihex_encode_helper(s):
return ''.join('{0:x}'.format(c) for c in s).encode()
def asciihex_decode_helper(s):
ret = []
try:
for a, b in zip(s[0::2], s[1::2]):
c = chr(a) + chr(b)
ret.append(chr(int(c, 16)))
return ''.join(ret).encode()
except Exception as e:
raise DecodeError("Unable to decode asciihex")
def base64_decode_helper(s):
s = s.decode()
for i in range(0, 8):
try:
s_padded = base64.b64decode(s + '=' * i)
return s_padded
except Exception as e2:
pass
raise DecodeError("Unable to base64 decode string: %s" % s)
def url_decode_helper(s):
bs = s.decode()
return urllib.parse.unquote(bs).encode()
def url_encode_helper(s):
bs = s.decode()
return urllib.parse.quote_plus(bs).encode()
def html_encode_helper(s):
return ''.join(['&#x{0:x};'.format(c) for c in s]).encode()
def html_decode_helper(s):
return html.unescape(s.decode()).encode()
def pp_json(s):
d = json.loads(s.strip())
return json.dumps(d, indent=2, sort_keys=True).encode()
def decode_jwt(s):
# in case they paste the whole auth header or the token with "bearer"
s = s.strip()
fields = s.split(b' ')
s = fields[-1].strip()
parts = s.split(b'.')
ret = b''
for part in parts:
try:
ret += base64_decode_helper(part.decode()) + b'\n\n'
except:
ret += b"[error decoding]\n\n"
return ret
def decode_unixtime(s):
ts = int(s)
dfmt = '%b %d, %Y %I:%M:%S %p'
try:
return datetime.utcfromtimestamp(ts).strftime(dfmt).encode()
except ValueError:
ts = ts/1000
return datetime.utcfromtimestamp(ts).strftime(dfmt).encode()
class DecoderWidget(QWidget):
def __init__(self):
QWidget.__init__(self)
layout = QVBoxLayout()
self.decoder_input = DecoderInput()
layout.addWidget(self.decoder_input)
self.setLayout(layout)
self.layout().setContentsMargins(0, 0, 0, 0)
class DecoderInput(QWidget):
decodeRun = pyqtSignal(bytes)
decoders = {
"encode_b64": ("Encode Base64", base64.b64encode),
"decode_b64": ("Decode Base64", base64_decode_helper),
"encode_ah": ("Encode Asciihex", asciihex_encode_helper),
"decode_ah": ("Decode Asciihex", asciihex_decode_helper),
"encode_url": ("URL Encode", url_encode_helper),
"decode_url": ("URL Decode", url_decode_helper),
"encode_html": ("HTML Encode", html_encode_helper),
"decode_html": ("HTML Decode", html_decode_helper),
"decode_unixtime": ("Format Unix Timestamp", decode_unixtime),
"pp_json": ("Pretty-Print JSON", pp_json),
"decode_jwt": ("Decode JWT Token", decode_jwt),
}
def __init__(self, *args, **kwargs):
QWidget.__init__(self)
layout = QVBoxLayout()
tool_layout = QHBoxLayout()
self.editor = ComboEditor(pretty_tab=False, enable_pretty=False)
self.encode_entry = QComboBox()
encode_button = QPushButton("Go!")
encode_button.clicked.connect(self.encode)
for k, v in self.decoders.items():
self.encode_entry.addItem(v[0], k)
layout.addWidget(self.editor)
tool_layout.addWidget(self.encode_entry)
tool_layout.addWidget(encode_button)
tool_layout.addStretch()
layout.addLayout(tool_layout)
self.setLayout(layout)
self.layout().setContentsMargins(0, 0, 0, 0)
@pyqtSlot()
def encode(self):
text = self.editor.get_bytes()
encode_type = self.encode_entry.itemData(self.encode_entry.currentIndex())
encode_func = DecoderInput.decoders[encode_type][1]
try:
encoded = encode_func(text)
except Exception as e:
display_error_box("Error processing string:\n" + str(e))
return
self.editor.set_bytes(encoded)
================================================
FILE: guppyproxy/gui.py
================================================
import random
from guppyproxy.reqlist import ReqBrowser, ReqListModel
from guppyproxy.repeater import RepeaterWidget
from guppyproxy.interceptor import InterceptorWidget
from guppyproxy.decoder import DecoderWidget
from guppyproxy.settings import SettingsWidget
from guppyproxy.shortcuts import GuppyShortcuts
from guppyproxy.macros import MacroWidget
from PyQt5.QtWidgets import QWidget, QTabWidget, QVBoxLayout, QTableView
from PyQt5.QtCore import Qt, QTimer, QObject, pyqtSlot
class GuppyWindow(QWidget):
titles = (
"Guppy Proxy",
)
def __init__(self, client):
QWidget.__init__(self)
self.client = client
self.delayTimeout = 100
self._resizeTimer = QTimer(self)
self._resizeTimer.timeout.connect(self._delayedUpdate)
self.setFocusPolicy(Qt.StrongFocus)
self.shortcuts = GuppyShortcuts(self)
self.tabWidget = QTabWidget()
self.repeaterWidget = RepeaterWidget(self.client)
self.interceptorWidget = InterceptorWidget(self.client)
self.macroWidget = MacroWidget(self.client)
self.historyWidget = ReqBrowser(self.client,
repeater_widget=self.repeaterWidget,
macro_widget=self.macroWidget,
is_client_context=True,
update=True)
self.decoderWidget = DecoderWidget()
self.settingsWidget = SettingsWidget(self.client)
self.settingsWidget.datafileLoaded.connect(self.historyWidget.reset_to_scope)
self.history_ind = self.tabWidget.count()
self.tabWidget.addTab(self.historyWidget, "History")
self.repeater_ind = self.tabWidget.count()
self.tabWidget.addTab(self.repeaterWidget, "Repeater")
self.interceptor_ind = self.tabWidget.count()
self.tabWidget.addTab(self.interceptorWidget, "Interceptor")
self.decoder_ind = self.tabWidget.count()
self.tabWidget.addTab(self.decoderWidget, "Decoder")
self.macro_ind = self.tabWidget.count()
self.tabWidget.addTab(self.macroWidget, "Macros")
self.settings_ind = self.tabWidget.count()
self.tabWidget.addTab(self.settingsWidget, "Settings")
self.mainLayout = QVBoxLayout()
self.mainLayout.addWidget(self.tabWidget)
self.mainWidget = QWidget()
self.mainWidget.setLayout(self.mainLayout)
self.wrapperLayout = QVBoxLayout()
self.wrapperLayout.addWidget(self.mainWidget)
self.wrapperLayout.setContentsMargins(0, 0, 0, 0)
self.setLayout(self.wrapperLayout)
self.setWindowTitle(random.choice(GuppyWindow.titles))
self.show()
def show_hist_tab(self):
self.tabWidget.setCurrentIndex(self.history_ind)
def show_repeater_tab(self):
self.tabWidget.setCurrentIndex(self.repeater_ind)
def show_interceptor_tab(self):
self.tabWidget.setCurrentIndex(self.interceptor_ind)
def show_decoder_tab(self):
self.tabWidget.setCurrentIndex(self.decoder_ind)
def show_active_macro_tab(self):
self.tabWidget.setCurrentIndex(self.macro_ind)
self.macroWidget.show_active()
def show_int_macro_tab(self):
self.tabWidget.setCurrentIndex(self.macro_ind)
self.macroWidget.show_int()
def resizeEvent(self, event):
QWidget.resizeEvent(self, event)
self._resizeTimer.stop()
self._resizeTimer.start(self.delayTimeout)
self.mainWidget.setVisible(False)
@pyqtSlot()
def _delayedUpdate(self):
self._resizeTimer.stop()
self.mainWidget.setVisible(True)
def close(self):
self.interceptorWidget.close()
================================================
FILE: guppyproxy/gup.py
================================================
import argparse
import sys
import os
from PyQt5.QtWidgets import QApplication
from PyQt5.QtCore import Qt
from guppyproxy.gui import GuppyWindow
from guppyproxy.proxy import ProxyClient, MessageError, ProxyThread
from guppyproxy.util import confirm, set_running_as_app
from guppyproxy.macros import MacroClient
def load_certificates(client, path):
client.load_certificates(os.path.join(path, "server.pem"),
os.path.join(path, "server.key"))
def generate_certificates(client, path):
try:
os.makedirs(path, 0o755)
except os.error as e:
if not os.path.isdir(path):
raise e
pkey_file = os.path.join(path, 'server.key')
cert_file = os.path.join(path, 'server.pem')
client.generate_certificates(pkey_file, cert_file)
def main():
parser = argparse.ArgumentParser(description="Guppy debug flags. Don't worry about most of these")
parser.add_argument("--binary", nargs=1, help="location of the backend binary")
parser.add_argument("--attach", nargs=1, help="attach to an already running backend")
parser.add_argument("--dbgattach", nargs=1, help="attach to an already running backend and also perform setup")
parser.add_argument('--debug', help='run in debug mode', action='store_true')
parser.add_argument('--dog', help='dog', action='store_true')
args = parser.parse_args()
if args.binary is not None and args.attach is not None:
print("Cannot provide both a binary location and an address to connect to")
exit(1)
data_dir = os.path.join(os.path.expanduser('~'), '.guppy')
if args.binary is not None:
binloc = args.binary[0]
msg_addr = None
elif args.attach is not None or args.dbgattach:
binloc = None
if args.attach is not None:
msg_addr = args.attach[0]
if args.dbgattach is not None:
msg_addr = args.dbgattach[0]
else:
msg_addr = None
binloc = os.path.join(data_dir, "puppy")
if 'RESOURCEPATH' in os.environ:
rpath = os.environ['RESOURCEPATH']
checkloc = os.path.join(rpath, 'puppyrsc', 'puppy.osx')
if os.path.exists(checkloc):
set_running_as_app(True)
binloc = checkloc
if not os.path.exists(binloc):
print("Could not find puppy binary. Please ensure that it has been compiled and placed in ~/.guppy/, or pass in the binary location from the command line")
exit(1)
cert_dir = os.path.join(data_dir, "certs")
with ProxyClient(binary=binloc, conn_addr=msg_addr, debug=args.debug) as client:
try:
load_certificates(client, cert_dir)
except MessageError as e:
generate_certificates(client, cert_dir)
print("Certificates generated to {}".format(cert_dir))
print("Be sure to add {} to your trusted CAs in your browser!".format(os.path.join(cert_dir, "server.pem")))
load_certificates(client, cert_dir)
try:
# Only try and listen/set default storage if we're not attaching
if args.attach is None:
storage = client.add_in_memory_storage("")
client.disk_storage = storage
client.inmem_storage = client.add_in_memory_storage("m")
client.set_proxy_storage(storage.storage_id)
app = QApplication(sys.argv)
window = GuppyWindow(client)
try:
app.exec_()
finally:
window.close()
except MessageError as e:
print(str(e))
MacroClient._ded = True # pray this kills the threads
ProxyThread.waitall()
def start():
main()
if __name__ == '__main__':
main()
================================================
FILE: guppyproxy/hexteditor.py
================================================
import base64
from guppyproxy.util import printable_data, qtprintable, textedit_highlight, DisableUpdates
from guppyproxy.proxy import _parse_message, Headers
from itertools import count
from PyQt5.QtWidgets import QWidget, QTextEdit, QTableWidget, QVBoxLayout, QTableWidgetItem, QTabWidget, QStackedLayout, QLabel, QComboBox
from PyQt5.QtGui import QTextCursor, QTextCharFormat, QImage, QColor, QTextImageFormat, QTextDocument, QTextDocumentFragment, QTextBlockFormat
from PyQt5.QtCore import Qt, pyqtSlot, QUrl
from pygments import highlight
from pygments.formatters import HtmlFormatter
from pygments.lexers import get_lexer_for_mimetype, TextLexer
from pygments.lexers.data import JsonLexer
from pygments.lexers.html import HtmlLexer
from pygments.styles import get_style_by_name
class PrettyPrintWidget(QWidget):
VIEW_NONE = 0
VIEW_HIGHLIGHTED = 1
VIEW_JSON = 2
VIEW_HTMLXML = 3
def __init__(self, *args, **kwargs):
QWidget.__init__(self, *args, **kwargs)
self.headers = Headers()
self.data = b''
self.view = 0
self.setLayout(QVBoxLayout())
self.layout().setContentsMargins(0, 0, 0, 0)
self.stack = QStackedLayout()
self.stack.setContentsMargins(0, 0, 0, 0)
self.nopp_widg = QLabel("No pretty version available")
self.stack.addWidget(self.nopp_widg)
self.highlighted_widg = QTextEdit()
self.highlighted_widg.setReadOnly(True)
self.stack.addWidget(self.highlighted_widg)
self.json_widg = QTextEdit()
self.json_widg.setReadOnly(True)
self.stack.addWidget(self.json_widg)
self.htmlxml_widg = QTextEdit()
self.htmlxml_widg.setReadOnly(True)
self.stack.addWidget(self.htmlxml_widg)
self.selector = QComboBox()
self.selector.addItem("Manually Select Printer", self.VIEW_NONE)
self.selector.addItem("Highlighted", self.VIEW_HIGHLIGHTED)
self.selector.addItem("JSON", self.VIEW_JSON)
self.selector.addItem("HTML/XML", self.VIEW_HTMLXML)
self.selector.currentIndexChanged.connect(self._combo_changed)
self.layout().addWidget(self.selector)
self.layout().addLayout(self.stack)
def guess_format(self):
if 'Content-Type' in self.headers:
ct = self.headers.get('Content-Type').lower()
if 'json' in ct:
self.set_view(self.VIEW_JSON)
elif 'html' in ct or 'xml' in ct:
self.set_view(self.VIEW_HTMLXML)
else:
self.set_view(self.VIEW_HIGHLIGHTED)
else:
self.set_view(self.VIEW_NONE)
@pyqtSlot()
def _combo_changed(self):
field = self.selector.itemData(self.selector.currentIndex())
old = self.selector.blockSignals(True)
self.set_view(field)
self.selector.blockSignals(old)
def set_view(self, view):
if view == self.VIEW_NONE:
self.clear_output()
self.stack.setCurrentIndex(self.VIEW_NONE)
elif view == self.VIEW_JSON:
self.clear_output()
self.fill_json()
self.stack.setCurrentIndex(self.VIEW_JSON)
elif view == self.VIEW_HTMLXML:
self.clear_output()
self.fill_htmlxml()
self.stack.setCurrentIndex(self.VIEW_HTMLXML)
elif view == self.VIEW_HIGHLIGHTED:
self.clear_output()
self.fill_highlighted()
self.stack.setCurrentIndex(self.VIEW_HIGHLIGHTED)
else:
return
self.selector.setCurrentIndex(view)
self.view = view
def clear_output(self):
self.json_widg.setPlainText("")
self.htmlxml_widg.setPlainText("")
def set_bytes(self, bs):
self.clear_output()
self.headers = Headers()
self.data = b''
if not bs:
return
_, h, body = _parse_message(bs, lambda x: None)
self.headers = h
self.data = body
def fill_json(self):
from .decoder import pp_json
with DisableUpdates(self.json_widg):
self.json_widg.setPlainText("")
if not self.data:
return
try:
j = pp_json(self.data.decode())
except Exception:
return
highlighted = textedit_highlight(j, JsonLexer())
self.json_widg.setHtml(highlighted)
def fill_htmlxml(self):
from lxml import etree, html
with DisableUpdates(self.htmlxml_widg):
self.htmlxml_widg.setPlainText("")
if not self.data:
return
try:
fragments = html.fragments_fromstring(self.data.decode())
parsed_frags = []
for f in fragments:
parsed_frags.append(etree.tostring(f, pretty_print=True))
pretty = b''.join(parsed_frags)
except Exception:
return
highlighted = textedit_highlight(pretty, HtmlLexer())
self.htmlxml_widg.setHtml(highlighted)
def fill_highlighted(self):
with DisableUpdates(self.htmlxml_widg):
self.highlighted_widg.setPlainText("")
if not self.data:
return
ct = self.headers.get('Content-Type').lower()
if ";" in ct:
ct = ct.split(";")[0]
try:
lexer = get_lexer_for_mimetype(ct)
highlighted = textedit_highlight(self.data, lexer)
except:
highlighted = printable_data(self.data)
self.highlighted_widg.setHtml(highlighted)
class HextEditor(QWidget):
byte_image = QImage()
byte_image.loadFromData(base64.b64decode("iVBORw0KGgoAAAANSUhEUgAAAAgAAAAICAYAAADED76LAAAAQklEQVQYlYWPMQoAMAgD82jpSzL613RpqG2FBly8QwywQlJ1UENSipAilIAS2FKFziFZ8LIVOjg6ocJx/+ELD/zVnJcqe5vHUAJgAAAAAElFTkSuQmCC"))
byte_url = "data://byte.png"
byte_property = 0x100000 + 1
nonce_property = 0x100000 + 2
byte_nonce = count()
def __init__(self, enable_pretty=True):
QWidget.__init__(self)
layout = QVBoxLayout()
self.enable_pretty = enable_pretty
self.setLayout(layout)
self.layout().setSpacing(0)
self.layout().setContentsMargins(0, 0, 0, 0)
self.lexer = TextLexer()
self.textedit = QTextEdit()
self.textedit.setAcceptRichText(False)
doc = self.textedit.document()
font = doc.defaultFont()
font.setFamily("Courier New")
font.setPointSize(10)
doc.setDefaultFont(font)
doc.addResource(QTextDocument.ImageResource,
QUrl(self.byte_url),
HextEditor.byte_image)
self.textedit.focusInEvent = self.focus_in_event
self.textedit.focusOutEvent = self.focus_left_event
self.data = b''
self.pretty_mode = False
self.layout().addWidget(self.textedit)
def focus_in_event(self, e):
QTextEdit.focusInEvent(self.textedit, e)
if not self.textedit.isReadOnly():
self.set_bytes(self.data)
self.pretty_mode = False
def focus_left_event(self, e):
QTextEdit.focusOutEvent(self.textedit, e)
if not self.textedit.isReadOnly():
self.data = self.get_bytes()
self.set_bytes_highlighted(self.data)
self.pretty_mode = True
def setReadOnly(self, ro):
self.textedit.setReadOnly(ro)
def _insert_byte(self, cursor, b):
f = QTextImageFormat()
f2 = QTextCursor().charFormat()
cursor.document().addResource(QTextDocument.ImageResource,
QUrl(self.byte_url),
HextEditor.byte_image)
f.setName(self.byte_url)
f.setProperty(HextEditor.byte_property, b + 1)
f.setProperty(HextEditor.nonce_property, next(self.byte_nonce))
cursor.insertImage(f)
cursor.setCharFormat(QTextCursor().charFormat())
def clear(self):
self.textedit.setPlainText("")
def set_lexer(self, lexer):
self.lexer = lexer
def set_bytes(self, bs):
with DisableUpdates(self.textedit):
self.pretty_mode = False
self.data = bs
chunks = HextEditor._split_by_printables(bs)
self.clear()
cursor = QTextCursor(self.textedit.document())
cursor.beginEditBlock()
try:
cursor.select(QTextCursor.Document)
cursor.setCharFormat(QTextCharFormat())
cursor.clearSelection()
for chunk in chunks:
if chr(chunk[0]) in qtprintable:
cursor.insertText(chunk.decode())
else:
for b in chunk:
self._insert_byte(cursor, b)
finally:
cursor.endEditBlock()
self.repaint() # needed to fix issue with py2app
def set_bytes_highlighted(self, bs, lexer=None):
if not self.enable_pretty:
self.set_bytes(bs)
return
with DisableUpdates(self.textedit):
self.pretty_mode = True
self.clear()
self.data = bs
if lexer:
self.lexer = lexer
printable = printable_data(bs)
highlighted = textedit_highlight(printable, self.lexer)
self.textedit.setHtml(highlighted)
self.repaint() # needed to fix issue with py2app
def get_bytes(self):
if not self.pretty_mode:
self.data = self._get_bytes()
return self.data
def _get_bytes(self):
from .util import hexdump
bs = bytearray()
block = self.textedit.document().firstBlock()
newline = False
while block.length() > 0:
if newline:
bs.append(ord('\n'))
newline = True
it = block.begin()
while not it.atEnd():
f = it.fragment()
fmt = f.charFormat()
byte = fmt.intProperty(HextEditor.byte_property)
if byte > 0:
text = f.text().encode()
if text == b"\xef\xbf\xbc":
bs.append(byte - 1)
else:
bs += text
else:
text = f.text()
bs += text.encode()
it += 1
block = block.next()
return bytes(bs)
@classmethod
def _split_by_printables(cls, bs):
if len(bs) == 0:
return []
def is_printable(c):
return c in qtprintable
chunks = []
printable = is_printable(chr(bs[0]))
a = 0
b = 1
while b < len(bs):
if is_printable(chr(bs[b])) != printable:
chunks.append(bs[a:b])
a = b
printable = not printable
b += 1
chunks.append(bs[a:b])
return chunks
class HexEditor(QWidget):
def __init__(self):
QWidget.__init__(self)
self.setLayout(QVBoxLayout())
self.layout().setContentsMargins(0, 0, 0, 0)
self.layout().setSpacing(0)
self.data = bytearray()
self.datatable = QTableWidget()
self.datatable.cellChanged.connect(self._cell_changed)
self.datatable.horizontalHeader().setStretchLastSection(True)
self.row_size = 16
self.read_only = False
self.redraw_table()
self.layout().addWidget(self.datatable)
def set_bytes(self, bs):
self.data = bytearray(bs)
self.redraw_table()
def get_bytes(self):
return bytes(self.data)
def setReadOnly(self, ro):
self.read_only = ro
self.redraw_table()
def _redraw_strcol(self, row):
start = self.row_size * row
end = start + self.row_size
data = self.data[start:end]
print_data = printable_data(data, include_newline=False)
item = QTableWidgetItem(print_data)
item.setFlags(item.flags() ^ Qt.ItemIsEditable)
self.datatable.setItem(row, self.str_col, item)
def redraw_table(self, length=None):
with DisableUpdates(self.datatable):
oldsig = self.datatable.blockSignals(True)
self.row_size = length or self.row_size
self.datatable.setColumnCount(self.row_size + 1)
self.datatable.setRowCount(0)
self.str_col = self.row_size
self.datatable.horizontalHeader().hide()
self.datatable.verticalHeader().hide()
rows = int(len(self.data) / self.row_size)
if len(self.data) % self.row_size > 0:
rows += 1
self.datatable.setRowCount(rows)
for i in range(rows * self.row_size):
row = i / self.row_size
col = i % self.row_size
if i < len(self.data):
dataval = "%02x" % self.data[i]
item = QTableWidgetItem(dataval)
if self.read_only:
item.setFlags(item.flags() ^ Qt.ItemIsEditable)
else:
item = QTableWidgetItem("")
item.setFlags(item.flags() ^ Qt.ItemIsEditable)
self.datatable.setItem(row, col, item)
for row in range(rows):
self._redraw_strcol(row)
self.datatable.blockSignals(oldsig)
self.datatable.resizeColumnsToContents()
self.datatable.resizeRowsToContents()
@classmethod
def _format_hex(cls, n):
return ("%02x" % n).upper()
@pyqtSlot(int, int)
def _cell_changed(self, row, col):
oldsig = self.datatable.blockSignals(True)
if col == self.str_col:
return
if len(self.data) == 0:
return
data_ind = self.row_size * row + col
if data_ind >= len(self.data):
return
data_text = self.datatable.item(row, col).text()
try:
data_val = int(data_text, 16)
if data_val < 0x0 or data_val > 0xff:
raise Exception()
except Exception as e:
item = QTableWidgetItem(self._format_hex(self.data[data_ind]))
self.datatable.setItem(row, col, item)
self.datatable.blockSignals(oldsig)
return
if data_text != self._format_hex(data_val):
self.datatable.setItem(row, col, QTableWidgetItem(self._format_hex(data_val)))
self.data[data_ind] = data_val
self._redraw_strcol(row)
self.datatable.blockSignals(oldsig)
class ComboEditor(QWidget):
def __init__(self, pretty_tab=True, enable_pretty=True):
QWidget.__init__(self)
self.setLayout(QVBoxLayout())
self.layout().setSpacing(0)
self.layout().setContentsMargins(0, 0, 0, 0)
self.data = b''
self.enable_pretty = enable_pretty
self.tabWidget = QTabWidget()
self.hexteditor = HextEditor(enable_pretty=self.enable_pretty)
self.hexeditor = HexEditor()
self.ppwidg = PrettyPrintWidget()
self.hexteditor_ind = self.tabWidget.count()
self.tabWidget.addTab(self.hexteditor, "Text")
self.hexeditor_ind = self.tabWidget.count()
self.tabWidget.addTab(self.hexeditor, "Hex")
self.pp_ind = -1
if pretty_tab:
self.pp_ind = self.tabWidget.count()
self.tabWidget.addTab(self.ppwidg, "Pretty")
self.tabWidget.currentChanged.connect(self._tab_changed)
self.previous_tab = self.tabWidget.currentIndex()
self.layout().addWidget(self.tabWidget)
@pyqtSlot(int)
def _tab_changed(self, i):
# commit data from old tab
if self.previous_tab == self.hexteditor_ind:
self.data = self.hexteditor.get_bytes()
if self.previous_tab == self.hexeditor_ind:
self.data = self.hexeditor.get_bytes()
# set up new tab
if i == self.hexteditor_ind:
if self.hexteditor.pretty_mode:
self.hexteditor.set_bytes_highlighted(self.data)
else:
self.hexteditor.set_bytes(self.data)
if i == self.hexeditor_ind:
self.hexeditor.set_bytes(self.data)
if i == self.pp_ind:
self.ppwidg.set_bytes(self.data)
self.ppwidg.guess_format()
# update previous tab
self.previous_tab = self.tabWidget.currentIndex()
@pyqtSlot(bytes)
def set_bytes(self, bs):
self.data = bs
self.tabWidget.setCurrentIndex(0)
if self.tabWidget.currentIndex() == self.hexteditor_ind:
self.hexteditor.set_bytes(bs)
elif self.tabWidget.currentIndex() == self.hexeditor_ind:
self.hexeditor.set_bytes(bs)
elif self.tabWidget.currentIndex() == self.pp_ind:
self.ppwidg.set_bytes(bs)
@pyqtSlot(bytes)
def set_bytes_highlighted(self, bs, lexer=None):
self.data = bs
self.tabWidget.setCurrentIndex(0)
if self.enable_pretty:
self.hexteditor.set_bytes_highlighted(bs, lexer=lexer)
else:
self.set_bytes(bs)
def get_bytes(self):
if self.tabWidget.currentIndex() == self.hexteditor_ind:
self.data = self.hexteditor.get_bytes()
elif self.tabWidget.currentIndex() == self.hexeditor_ind:
self.data = self.hexeditor.get_bytes()
return self.data
def setReadOnly(self, ro):
self.hexteditor.setReadOnly(ro)
self.hexeditor.setReadOnly(ro)
================================================
FILE: guppyproxy/interceptor.py
================================================
from guppyproxy.util import display_error_box
from guppyproxy.proxy import InterceptMacro, parse_request, parse_response
from guppyproxy.hexteditor import ComboEditor
from PyQt5.QtWidgets import QWidget, QVBoxLayout, QHBoxLayout, QPushButton
from PyQt5.QtCore import pyqtSlot, pyqtSignal, QObject
import threading
edit_queue = []
class InterceptEvent:
def __init__(self):
self.e = threading.Event()
self.canceled = False
self.message = None
def wait(self):
self.e.wait()
return self.message
def set(self, message):
self.message = message
self.e.set()
def cancel(self):
self.canceled = True
self.set(None)
class InterceptedMessage:
def __init__(self, request=None, response=None, wsmessage=None):
self.request = request
self.response = response
self.wsmessage = wsmessage
self.event = InterceptEvent()
self.message_type = None
if self.request:
self.message_type = "request"
elif self.response:
self.message_type = "response"
elif self.wsmessage:
self.message_type = "wsmessage"
class InterceptorMacro(InterceptMacro, QObject):
"""
A class representing a macro that modifies requests as they pass through the
proxy
"""
messageReceived = pyqtSignal(InterceptedMessage)
def __init__(self, int_widget):
InterceptMacro.__init__(self)
QObject.__init__(self)
self.int_widget = int_widget
self.messageReceived.connect(self.int_widget.message_received)
self.name = "InterceptorMacro"
def mangle_request(self, request):
int_msg = InterceptedMessage(request=request)
self.messageReceived.emit(int_msg)
req = int_msg.event.wait()
if int_msg.event.canceled:
return request
req.dest_host = request.dest_host
req.dest_port = request.dest_port
req.use_tls = request.use_tls
return req
def mangle_response(self, request, response):
int_msg = InterceptedMessage(response=response)
self.messageReceived.emit(int_msg)
rsp = int_msg.event.wait()
if int_msg.event.canceled:
return response
return rsp
def mangle_websocket(self, request, response, message):
# just don't do this right now
pass
class InterceptorWidget(QWidget):
def __init__(self, client):
QWidget.__init__(self)
self.client = client
self.int_conn = None
self.queued_messages = []
self.editing_message = None
self.editing = False
self.int_req = False
self.int_rsp = False
self.int_ws = False
# layouts
self.setLayout(QVBoxLayout())
self.layout().setSpacing(0)
self.layout().setContentsMargins(0, 0, 0, 0)
buttons = QHBoxLayout()
buttons.setContentsMargins(0, 0, 0, 0)
buttons.setSpacing(10)
# widgets
intReqButton = QPushButton("Int. Requests")
intRspButton = QPushButton("Int. Responses")
intWsButton = QPushButton("Int. Websocket")
forwardButton = QPushButton("Forward")
cancelButton = QPushButton("Cancel")
self.editor = ComboEditor()
intReqButton.setCheckable(True)
intRspButton.setCheckable(True)
intWsButton.setCheckable(True)
intWsButton.setEnabled(False)
forwardButton.clicked.connect(self.forward_message)
cancelButton.clicked.connect(self.cancel_edit)
intReqButton.toggled.connect(self.int_req_toggled)
intRspButton.toggled.connect(self.int_rsp_toggled)
intWsButton.toggled.connect(self.int_ws_toggled)
buttons.addWidget(forwardButton)
buttons.addWidget(cancelButton)
buttons.addWidget(intReqButton)
buttons.addWidget(intRspButton)
buttons.addWidget(intWsButton)
# checkbox for req/rsp/ws
self.layout().addLayout(buttons)
self.layout().addWidget(self.editor)
@pyqtSlot(bool)
def int_req_toggled(self, state):
self.int_req = state
self.restart_intercept()
@pyqtSlot(bool)
def int_rsp_toggled(self, state):
self.int_rsp = state
self.restart_intercept()
@pyqtSlot(bool)
def int_ws_toggled(self, state):
self.int_ws = state
self.restart_intercept()
@pyqtSlot(InterceptedMessage)
def message_received(self, msg):
self.queued_messages.append(msg)
# Update queue list
self.edit_next_message()
def set_edited_message(self, msg):
if msg.message_type == "request":
self.editor.set_bytes(msg.request.full_message())
elif msg.message_type == "response":
self.editor.set_bytes(msg.response.full_message())
elif msg.message_type == "wsmessage":
# this is not gonna work
self.editor.set_bytes(msg.wsmessage.message)
def edit_next_message(self):
if self.editing:
return
self.editor.set_bytes(b"")
if not self.queued_messages:
return
self.editing_message = self.queued_messages.pop()
self.set_edited_message(self.editing_message)
self.editing = True
@pyqtSlot()
def forward_message(self):
if not self.editing:
return
if self.editing_message.message_type == "request":
try:
req = parse_request(self.editor.get_bytes())
except Exception:
display_error_box("Could not parse request")
return
self.editing_message.event.set(req)
elif self.editing_message.message_type == "response":
try:
rsp = parse_response(self.editor.get_bytes())
except Exception:
display_error_box("Could not parse response")
return
self.editing_message.event.set(rsp)
elif self.editing_message.message_type == "wsmessage":
pass
self.editing = False
self.edit_next_message()
@pyqtSlot()
def cancel_edit(self):
if self.editing_message:
self.editing_message.event.cancel()
self.editing = False
self.edit_next_message()
def clear_edit_queue(self):
while self.queued_messages or self.editing_message:
if self.editing_message:
self.editing_message.event.cancel()
self.editing_message = False
if self.queued_messages:
self.editing_message = self.queued_messages.pop()
def restart_intercept(self):
self.close()
self.editor.set_bytes("")
self.editing = False
if not (self.int_req or self.int_rsp or self.int_ws):
return
mangle_macro = InterceptorMacro(self)
mangle_macro.intercept_requests = self.int_req
mangle_macro.intercept_responses = self.int_rsp
mangle_macro.intercept_ws = self.int_ws
self.int_conn = self.client.new_conn()
self.int_conn.intercept(mangle_macro)
def close(self):
if self.int_conn:
self.int_conn.close()
self.int_conn = None
self.clear_edit_queue()
================================================
FILE: guppyproxy/macros.py
================================================
import glob
import imp
import os
import random
import re
import stat
import sys
import traceback
from guppyproxy.proxy import InterceptMacro, HTTPRequest, ProxyThread
from guppyproxy.util import display_error_box, qtprintable, set_default_dialog_dir, default_dialog_dir, open_dialog, save_dialog, display_info_box
from collections import namedtuple
from itertools import count
from PyQt5.QtWidgets import QWidget, QTableWidget, QTableWidgetItem, QGridLayout, QHeaderView, QAbstractItemView, QVBoxLayout, QHBoxLayout, QComboBox, QTabWidget, QPushButton, QLineEdit, QStackedLayout, QToolButton, QCheckBox, QLabel, QTableView, QPlainTextEdit, QFormLayout, QSizePolicy, QDialog, QPlainTextEdit, QTextEdit
from PyQt5.QtCore import pyqtSlot, pyqtSignal, QObject, QVariant, Qt, QAbstractTableModel, QModelIndex, QItemSelection, QSortFilterProxyModel
errwins = set()
class MacroException(Exception):
pass
class MacroClient(QObject):
# A wrapper around proxy.ProxyClient that provides a simplified interface
# to a macro to prevent it from accidentally making the proxy unstable.
# Will add to it as needed/requested
_macroOutput = pyqtSignal(str)
_requestOutput = pyqtSignal(HTTPRequest)
_ded = False
def __init__(self, client):
QObject.__init__(self)
self._client = client
def check_dead(self):
"""
Raises an exception if the program is trying to close. Use this in your loops so that your macro doesn't keep the program from quitting
"""
if self._ded:
raise Exception("program over=very yes")
def submit(self, req, save=False):
"""
Submit a request. If save == True, it will be saved to history
"""
self.check_dead()
self._client.submit(req, save=save)
def save(self, req):
"""
Manually save a request to history. This can be used to perform a request and only save
requests with interesting responses
"""
self.check_dead()
self._client.save_new(req)
def output(self, s):
"""
Write text to the "output" tab
"""
self.check_dead()
self._macroOutput.emit(str(s)+"\n")
def output_req(self, req):
"""
Add a request/response to the list of outputted requests
"""
self.check_dead()
self._requestOutput.emit(req)
def new_request(self, method="GET", path="/", proto_major=1, proto_minor=1,
headers=None, body=bytes(), dest_host="", dest_port=80,
use_tls=False, tags=None):
"""
Manually create a request object that can be submitted with client.submit()
"""
self.check_dead()
return HTTPRequest(method=method, path=path, proto_major=proto_major, proto_minor=proto_minor,
headers=headers, body=body, dest_host=dest_host, dest_port=dest_port,
use_tls=use_tls, tags=tags)
class FileInterceptMacro(InterceptMacro, QObject):
"""
An intercepting macro that loads a macro from a file.
"""
macroError = pyqtSignal(str)
def __init__(self, parent, client, filename):
InterceptMacro.__init__(self)
QObject.__init__(self)
self.fname = filename or None # name from the file
self.source = None
self.client = client
self.parent = parent
self.mclient = MacroClient(self.client)
self.cached_args = {}
self.used_args = {}
if filename:
self.load(filename)
def __repr__(self):
s = self.fname or "(No loaded macro)"
return "<InterceptingMacro %s>" % s
def load(self, fname):
if fname:
self.fname = fname
# yes there's a race condition here, but it's better than nothing
st = os.stat(self.fname)
if (st.st_mode & stat.S_IWOTH):
raise MacroException("Refusing to load world-writable macro: %s" % self.fname)
module_name = self.fname
try:
if module_name in sys.modules and self.source != None:
del sys.modules[module_name]
del self.source
self.source = imp.load_source(module_name, self.fname)
except Exception as e:
self.macroError.emit(make_err_str(self, e))
else:
self.fname = None
self.source = None
# Update what we can do
if self.source and hasattr(self.source, 'mangle_request'):
self.intercept_requests = True
else:
self.intercept_requests = False
if self.source and hasattr(self.source, 'mangle_response'):
self.intercept_responses = True
else:
self.intercept_responses = False
if self.source and hasattr(self.source, 'mangle_websocket'):
self.intercept_ws = True
else:
self.intercept_ws = False
def prompt_args(self):
if not hasattr(self.source, "get_args"):
self.used_args = {}
return True
try:
spec = self.source.get_args()
except Exception as e:
self.macroError.emit(make_err_str(self, e))
return False
args = get_macro_args(self.parent, spec, cached=self.cached_args)
if args is None:
return False
self.cached_args = args
self.used_args = args
return True
def init(self, args):
if hasattr(self.source, 'init'):
try:
self.source.init(self.mclient, args)
except Exception as e:
self.macroError.emit(make_err_str(self, e))
return False
return True
def mangle_request(self, request):
if hasattr(self.source, 'mangle_request'):
try:
return self.source.mangle_request(self.mclient, self.used_args, request)
except Exception as e:
self.macroError.emit(make_err_str(self, e))
return request
def mangle_response(self, request, response):
if hasattr(self.source, 'mangle_response'):
try:
return self.source.mangle_response(self.mclient, self.used_args, request, response)
except Exception as e:
self.macroError.emit(make_err_str(self, e))
return response
def mangle_websocket(self, request, response, message):
if hasattr(self.source, 'mangle_websocket'):
try:
return self.source.mangle_websocket(self.mclient, self.used_args, request, response, message)
except Exception as e:
self.macroError.emit(make_err_str(self, e))
return message
class FileMacro(QObject):
macroError = pyqtSignal(str)
macroComplete = pyqtSignal(str)
requestOutput = pyqtSignal(HTTPRequest)
macroOutput = pyqtSignal(str)
def __init__(self, parent, filename='', resultSlot=None):
QObject.__init__(self)
self.fname = filename or None # filename we load from
self.source = None
self.parent = parent
self.cached_args = {}
self.load()
def load(self):
if self.fname:
st = os.stat(self.fname)
if (st.st_mode & stat.S_IWOTH):
raise MacroException("Refusing to load world-writable macro: %s" % self.fname)
module_name = self.fname
try:
if module_name in sys.modules and self.source != None:
del sys.modules[module_name]
del self.source
self.source = imp.load_source('%s'%module_name, self.fname)
except Exception as e:
self.macroError.emit(make_err_str(self, e))
def execute(self, client, reqs):
self.load()
# Execute the macro
if self.source:
args = None
if hasattr(self.source, "get_args"):
try:
spec = self.source.get_args()
except Exception as e:
self.macroError.emit(make_err_str(self, e))
return
args = get_macro_args(self.parent, spec, cached=self.cached_args)
if args is None:
return
self.cached_args = args
def perform_macro():
mclient = MacroClient(client)
mclient._macroOutput.connect(self.macroOutput)
mclient._requestOutput.connect(self.requestOutput)
try:
self.source.run_macro(mclient, args, reqs)
_, fname = os.path.split(self.fname)
self.macroComplete.emit("%s has finished running" % fname)
except Exception as e:
self.macroError.emit(make_err_str(self, e))
ProxyThread(target=perform_macro).start()
class MacroWidget(QWidget):
# Tabs containing both int and active macros
def __init__(self, client, *args, **kwargs):
self.client = client
QWidget.__init__(self, *args, **kwargs)
self.setLayout(QVBoxLayout())
self.layout().setContentsMargins(0, 0, 0, 0)
self.tab_widg = QTabWidget()
self.active_widg = ActiveMacroWidget(client)
self.active_ind = self.tab_widg.count()
self.tab_widg.addTab(self.active_widg, "Active")
self.int_widg = IntMacroWidget(client)
self.int_ind = self.tab_widg.count()
self.tab_widg.addTab(self.int_widg, "Intercepting")
self.warning_widg = QLabel("<h1>Warning! Macros may cause instability</h1><p>Macros load and run python files into the Guppy process. If you're not careful when you write them you may cause Guppy to crash. If an active macro ends up in an infinite loop you may need to force kill the application when you quit.</p><p><b>PROCEED WITH CAUTION</b></p>")
self.warning_widg.setWordWrap(True)
self.tab_widg.addTab(self.warning_widg, "Warning")
self.layout().addWidget(self.tab_widg)
def show_active(self):
self.tab_widg.setCurrentIndex(self.active_ind)
def show_int(self):
self.tab_widg.setCurrentIndex(self.int_ind)
def add_requests(self, reqs):
# Add requests to active macro inputw
self.active_widg.add_requests(reqs)
class IntMacroListModel(QAbstractTableModel):
err_window = None
def __init__(self, parent, client, *args, **kwargs):
self.client = client
QAbstractTableModel.__init__(self, *args, **kwargs)
self.macros = []
self.int_conns = {}
self.conn_ids = count()
self.parent = parent
self.headers = ["On", "Path"]
def _emit_all_data(self):
self.dataChanged.emit(self.createIndex(0, 0), self.createIndex(self.columnCount(None), self.rowCount(None)))
def headerData(self, section, orientation, role):
if role == Qt.DisplayRole and orientation == Qt.Horizontal:
return self.headers[section]
return QVariant()
def rowCount(self, parent):
return len(self.macros)
def columnCount(self, parent):
return len(self.headers)
def data(self, index, role):
if role == Qt.DisplayRole:
if index.column() == 1:
rowdata = self.macros[index.row()]
macro = rowdata[index.column()]
return macro.fname
if role == Qt.CheckStateRole:
if index.column() == 0:
if self.macros[index.row()][0]:
return 2
return 0
return QVariant()
def flags(self, index):
f = Qt.ItemIsEnabled | Qt.ItemIsSelectable
if index.column() == 0:
f = f | Qt.ItemIsUserCheckable | Qt.ItemIsEditable
return f
def setData(self, index, value, role):
if role == Qt.CheckStateRole and index.column() == 0:
if value:
self.enable_macro(index.row())
else:
self.disable_macro(index.row())
return True
return False
# Non model functions
@pyqtSlot(str)
def add_macro_exception(self, estr):
if not self.err_window:
self.err_window = MacroErrWindow()
self.err_window.add_error(estr)
def add_macro(self, macro_path):
self.beginResetModel()
macro = FileInterceptMacro(self.parent, self.client, macro_path)
macro.macroError.connect(self.add_macro_exception)
self.macros.append([False, macro, -1])
self._emit_all_data()
self.endResetModel()
def remove_macro(self, ind):
self.beginResetModel()
self.disable_macro(ind)
self.macros = self.macros[:ind] + self.macros[ind+1:]
self._emit_all_data()
self.endResetModel()
def enable_macro(self, ind):
self.beginResetModel()
macro = self.macros[ind][1]
if not macro.init(None):
return
try:
macro.load(macro.fname)
except MacroException as e:
display_error_box("Macro could not be loaded: %s", e)
return
except Exception as e:
self.add_macro_exception(make_err_str(macro, e))
return
if not (macro.intercept_requests or macro.intercept_responses or macro.intercept_ws):
display_error_box("Macro must implement mangle_request or mangle_response")
return
if not macro.prompt_args():
return
conn = self.client.new_conn()
conn_id = next(self.conn_ids)
self.macros[ind][2] = conn_id
self.int_conns[conn_id] = conn
conn.intercept(macro)
self.macros[ind][0] = True
self._emit_all_data()
self.endResetModel()
def disable_macro(self, ind):
self.beginResetModel()
conn_id = self.macros[ind][2]
if conn_id >= 0:
conn = self.int_conns[conn_id]
conn.close()
del self.int_conns[conn_id]
self.macros[ind][2] = -1
self.macros[ind][0] = False
self._emit_all_data()
self.endResetModel()
class IntMacroWidget(QWidget):
# Lets the user enable/disable int. macros
def __init__(self, client, *args, **kwargs):
self.client = client
self.macros = []
QWidget.__init__(self, *args, **kwargs)
self.setLayout(QVBoxLayout())
self.layout().setContentsMargins(0, 0, 0, 0)
buttonLayout = QHBoxLayout()
new_button = QPushButton("New")
add_button = QPushButton("Add...")
remove_button = QPushButton("Remove")
new_button.clicked.connect(self.new_macro)
add_button.clicked.connect(self.browse_macro)
remove_button.clicked.connect(self.remove_selected)
# Set up table
self.macroListModel = IntMacroListModel(self, self.client)
self.macroListView = QTableView()
self.macroListView.setModel(self.macroListModel)
self.macroListView.verticalHeader().setSectionResizeMode(QHeaderView.ResizeToContents)
self.macroListView.verticalHeader().hide()
self.macroListView.horizontalHeader().setSectionResizeMode(QHeaderView.ResizeToContents)
self.macroListView.horizontalHeader().hide()
self.macroListView.horizontalHeader().setStretchLastSection(True)
self.macroListView.setSelectionBehavior(QAbstractItemView.SelectRows)
self.macroListView.setSelectionMode(QAbstractItemView.SingleSelection)
buttonLayout.addWidget(new_button)
buttonLayout.addWidget(add_button)
buttonLayout.addWidget(remove_button)
buttonLayout.addStretch()
self.layout().addWidget(self.macroListView)
self.layout().addLayout(buttonLayout)
def add_macro(self, fname):
self.macroListModel.add_macro(fname)
def reload_macros(self):
self.macroListModel.reload_macros()
@pyqtSlot()
def new_macro(self):
fname = save_dialog(self, filter_string="Python File (*.py)")
if not fname:
return
with open(fname, 'w') as f:
contents = new_int_macro()
f.write(contents)
self.add_macro(fname)
@pyqtSlot()
def browse_macro(self):
fname = open_dialog(self, filter_string="Python File (*.py)")
if not fname:
return
self.add_macro(fname)
@pyqtSlot()
def remove_selected(self):
rows = self.macroListView.selectionModel().selectedRows()
if len(rows) == 0:
return
for idx in rows:
row = idx.row()
self.macroListModel.remove_macro(row)
return
class ActiveMacroModel(QAbstractTableModel):
err_window = None
requestOutput = pyqtSignal(HTTPRequest)
macroOutput = pyqtSignal(str)
def __init__(self, parent, client, *args, **kwargs):
QAbstractTableModel.__init__(self, *args, **kwargs)
self.client = client
self.parent = parent
self.headers = ["Path"]
self.macros = []
def _emit_all_data(self):
self.dataChanged.emit(self.createIndex(0, 0), self.createIndex(self.columnCount(None), self.rowCount(None)))
def headerData(self, section, orientation, role):
if role == Qt.DisplayRole and orientation == Qt.Horizontal:
return self.headers[section]
return QVariant()
def rowCount(self, parent):
return len(self.macros)
def columnCount(self, parent):
return len(self.headers)
def data(self, index, role):
if role == Qt.DisplayRole:
return self.macros[index.row()][0]
return QVariant()
def flags(self, index):
return Qt.ItemIsEnabled | Qt.ItemIsSelectable
def add_macro(self, path):
self.beginResetModel()
self._emit_all_data()
fileMacro = FileMacro(self.parent, filename=path)
fileMacro.macroOutput.connect(self.macroOutput)
fileMacro.macroError.connect(self.add_macro_exception)
fileMacro.requestOutput.connect(self.requestOutput)
fileMacro.macroComplete.connect(self.display_macro_complete)
self.macros.append((path, fileMacro))
self.endResetModel()
def run_macro(self, ind, reqs=None):
path, macro = self.macros[ind]
reqs = reqs or []
macro.execute(self.client, reqs)
def remove_macro(self, ind):
self.beginResetModel()
self._emit_all_data()
self.macros = self.macros[:ind] + self.macros[ind+1:]
self.endResetModel()
@pyqtSlot(str)
def add_macro_exception(self, estr):
if not self.err_window:
self.err_window = MacroErrWindow()
self.err_window.add_error(estr)
@pyqtSlot(str)
def display_macro_complete(self, msg):
display_info_box(msg, title="Macro complete")
class ActiveMacroWidget(QWidget):
# Provides an interface to send a set of requests to python scripts
def __init__(self, client, *args, **kwargs):
from .reqlist import ReqTableWidget, ReqBrowser
QWidget.__init__(self, *args, **kwargs)
self.client = client
self.setLayout(QVBoxLayout())
tab_widg = QTabWidget()
# Input
inputLayout = QVBoxLayout()
inputLayout.setContentsMargins(0, 0, 0, 0)
inputLayout.addWidget(QLabel("Input"))
inputLayout.setSpacing(8)
self.reqlist = ReqTableWidget(self.client)
butlayout = QHBoxLayout()
delButton = QPushButton("Remove")
clearButton = QPushButton("Clear")
importAllButton = QPushButton("Import Currently Filtered Requests")
delButton.clicked.connect(self.reqlist.delete_selected)
clearButton.clicked.connect(self.reqlist.clear)
importAllButton.clicked.connect(self.import_all_reqs)
butlayout.addWidget(delButton)
butlayout.addWidget(clearButton)
butlayout.addWidget(importAllButton)
butlayout.addStretch()
inputLayout.addWidget(self.reqlist)
inputLayout.addLayout(butlayout)
# Macro selection
listLayout = QVBoxLayout()
listLayout.addWidget(QLabel("Macros"))
listLayout.setContentsMargins(0, 0, 0, 0)
listLayout.setSpacing(8)
self.tableModel = ActiveMacroModel(self, self.client)
self.tableModel.macroOutput.connect(self.add_macro_output)
self.tableView = QTableView()
self.tableModel.requestOutput.connect(self.add_request_output)
self.tableView.setModel(self.tableModel)
self.tableView.verticalHeader().setSectionResizeMode(QHeaderView.ResizeToContents)
self.tableView.verticalHeader().hide()
self.tableView.horizontalHeader().setSectionResizeMode(QHeaderView.ResizeToContents)
self.tableView.horizontalHeader().setStretchLastSection(True)
self.tableView.horizontalHeader().hide()
self.tableView.setSelectionMode(QAbstractItemView.SingleSelection)
butlayout2 = QHBoxLayout()
newButton = QPushButton("New")
newButton.clicked.connect(self.new_macro)
addButton2 = QPushButton("Add...")
addButton2.clicked.connect(self.browse_macro)
delButton2 = QPushButton("Remove")
delButton2.clicked.connect(self.remove_selected)
runButton2 = QPushButton("Run")
runButton2.clicked.connect(self.run_selected_macro)
butlayout2.addWidget(newButton)
butlayout2.addWidget(addButton2)
butlayout2.addWidget(delButton2)
butlayout2.addWidget(runButton2)
butlayout2.addStretch()
listLayout.addWidget(self.tableView)
listLayout.addLayout(butlayout2)
# Output
outputLayout = QVBoxLayout()
outputLayout.setContentsMargins(0, 0, 0, 0)
outputLayout.setSpacing(8)
self.outreqlist = ReqBrowser(self.client, reload_reqs=False, filter_tab=False)
self.outreqlist.listWidg.allow_save = True
outbutlayout = QHBoxLayout()
delButton = QPushButton("Clear")
delButton.clicked.connect(self.clear_output)
outbutlayout.addWidget(delButton)
outbutlayout.addStretch()
outputLayout.addWidget(self.outreqlist)
outputLayout.addLayout(outbutlayout)
text_out_layout = QVBoxLayout()
text_out_layout.setContentsMargins(0, 0, 0, 0)
self.macro_text_out = QPlainTextEdit()
text_out_layout.addWidget(self.macro_text_out)
text_out_butlayout = QHBoxLayout()
clearBut = QPushButton("Clear")
clearBut.clicked.connect(self.clear_text_output)
text_out_butlayout.addWidget(clearBut)
text_out_butlayout.addStretch()
text_out_layout.addLayout(text_out_butlayout)
# Tabs
intab = QWidget()
intabLayout = QVBoxLayout()
intabLayout.setContentsMargins(0, 0, 0, 0)
intabLayout.addLayout(listLayout)
intabLayout.addLayout(inputLayout)
intab.setLayout(intabLayout)
tab_widg.addTab(intab, "Input")
reqOutputWidg = QWidget()
reqOutputWidg.setLayout(outputLayout)
tab_widg.addTab(reqOutputWidg, "Req. Output")
textOutputWidg = QWidget()
textOutputWidg.setLayout(text_out_layout)
tab_widg.addTab(textOutputWidg, "Text Output")
self.layout().addWidget(tab_widg)
@pyqtSlot(list)
def add_requests(self, reqs):
# Add requests to active macro input
for req in reqs:
self.reqlist.add_request(req)
@pyqtSlot()
def new_macro(self):
fname = save_dialog(self, filter_string="Python File (*.py)")
if not fname:
return
with open(fname, 'w') as f:
contents = new_active_macro()
f.write(contents)
self.tableModel.add_macro(fname)
@pyqtSlot()
def browse_macro(self):
fname = open_dialog(self, filter_string="Python File (*.py)")
if not fname:
return
self.tableModel.add_macro(fname)
@pyqtSlot()
def remove_selected(self):
rows = self.tableView.selectionModel().selectedRows()
if len(rows) == 0:
return
for idx in rows:
row = idx.row()
self.tableModel.remove_macro(row)
return
@pyqtSlot()
def run_selected_macro(self):
rows = self.tableView.selectionModel().selectedRows()
if len(rows) == 0:
return
for idx in rows:
row = idx.row()
reqs = self.reqlist.get_all_requests()
self.tableModel.run_macro(row, reqs)
return
@pyqtSlot(HTTPRequest)
def add_request_output(self, req):
self.outreqlist.listWidg.add_request(req)
@pyqtSlot()
def clear_output(self):
self.outreqlist.set_requests([])
@pyqtSlot()
def clear_text_output(self):
self.macro_text_out.setPlainText("")
@pyqtSlot(str)
def add_macro_output(self, s):
t = self.macro_text_out.toPlainText()
t += s
self.macro_text_out.setPlainText(t)
@pyqtSlot()
def import_all_reqs(self):
reqs = self.client.in_context_requests(headers_only=True)
self.add_requests(reqs)
class MacroErrWindow(QWidget):
def __init__(self, *args, **kwargs):
QObject.__init__(self, *args, **kwargs)
self.msg = ""
self.setLayout(QVBoxLayout())
self.msgwidg = QPlainTextEdit()
self.layout().addWidget(self.msgwidg)
def add_error(self, msg):
self.msg += msg + "\n\n"
self.msgwidg.setPlainText(self.msg)
self.show()
def closeEvent(self, event):
self.msgwidg.setPlainText("")
IntMacroListModel.err_window = None
ActiveMacroModel.err_window = None
def make_err_str(macro, e):
estr = "Exception in macro %s:\n" % macro.fname
estr += str(e) + '\n'
estr += str(traceback.format_exc())
return estr
class ArgWindow(QDialog):
def __init__(self, parent, argspec, cached=None):
QDialog.__init__(self, parent)
winLayout = QVBoxLayout()
formLayout = QFormLayout()
self.shownargs = []
self.canceled = False
argnames = set()
for spec in argspec:
name = None
argtype = None
argval = None
if isinstance(spec, str):
name = spec
argtype = "str"
else:
if len(spec) > 0:
name = spec[0]
if len(spec) > 1:
argtype = spec[1]
if len(spec) > 2:
argval = spec[2]
if not name:
continue
if not argtype:
continue
if name in argnames:
continue
widg = None
if argtype.lower() in ("str", "string"):
argtype = "str"
widg = QLineEdit()
if name in cached:
widg.setText(cached[name])
else:
return
formLayout.addRow(QLabel(name), widg)
self.shownargs.append(((name, argtype, argval), widg))
argnames.add(name)
butlayout = QHBoxLayout()
okbut = QPushButton("Ok")
okbut.clicked.connect(self.accept)
cancelbut = QPushButton("Cancel")
cancelbut.clicked.connect(self.reject)
self.rejected.connect(self._set_canceled)
butlayout.addWidget(okbut)
butlayout.addWidget(cancelbut)
butlayout.addStretch()
winLayout.addLayout(formLayout)
winLayout.addLayout(butlayout)
self.setLayout(winLayout)
@pyqtSlot()
def _set_canceled(self):
self.canceled = True
def get_args(self):
if self.canceled:
return None
retargs = {}
for shownarg in self.shownargs:
spec, widg = shownarg
name, argtype, typeargs = spec
if argtype == "str":
retargs[name] = widg.text()
return retargs
def get_macro_args(parent, argspec, cached=None):
if not isinstance(argspec, list):
return
argwin = ArgWindow(parent, argspec, cached=cached)
argwin.exec_()
return argwin.get_args()
def req_python_def(varname, req):
method = req.method
path = req.url.geturl()
pmajor = req.proto_major
pminor = req.proto_minor
headers = req.headers.dict().items()
dest_host = req.dest_host
dest_port = req.dest_port
if req.use_tls:
use_tls = "True"
else:
use_tls = "False"
body = ""
if len(req.body) > 0:
s = '"'
if b'\n' in req.body:
s = '"""'
for c in req.body:
if chr(c) in qtprintable:
body += chr(c)
else:
body += "\\x%02x" % c
body = "%s%s%s" % (s, body, s)
ret = ''
ret += '%s = HTTPRequest(' % varname
ret += 'proto_major=%d, proto_minor=%d,\n' % (pmajor, pminor)
ret += ' use_tls=%s, dest_host="%s", dest_port=%d,\n' % (use_tls, dest_host, dest_port)
ret += ' method="%s", path="%s", headers={\n' % (method, path)
for k, vs in headers:
qvs = []
for v in vs:
qvs.append('"%s"' % v)
vstr = "[" + ", ".join(qvs) + "]"
ret += ' "%s": %s,\n' % (k, vstr)
ret += ' },\n'
if len(body) > 0:
ret += ' body=%s\n' % body
ret += ")"
return ret
def create_macro_template(reqs):
ret = "from guppyproxy.proxy import HTTPRequest\n\n"
i = 0
for req in reqs:
ret += req_python_def("req%d"%i, req)
ret += "\n\n"
i += 1
ret += "def run_macro(client, args, reqs):\n"
if i == 0:
ret += " pass\n"
for ii in range(i):
ret += " client.submit(req%d)\n" % ii
ret += " client.output_req(req%d)\n\n" % ii
return ret
def new_active_macro():
return "def run_macro(client, args, reqs):\n # Macro code goes here\n pass"
def new_int_macro():
return """def mangle_request(client, args, req):
# modify request here
return req
def mangle_response(client, args, req, rsp):
# modify response here
return rsp
"""
================================================
FILE: guppyproxy/proxy.py
================================================
#!/usr/bin/env python3
import base64
import copy
import datetime
import json
import math
import re
import socket
import threading
from collections import namedtuple
from itertools import count
from urllib.parse import urlparse, ParseResult, parse_qs, urlencode
from subprocess import Popen, PIPE
from http import cookies as hcookies
from PyQt5.QtCore import QThread, QObject, pyqtSlot
class MessageError(Exception):
pass
class ProxyException(Exception):
pass
class InvalidQuery(Exception):
pass
class SocketClosed(Exception):
pass
class SockBuffer:
# I can't believe I have to implement this
def __init__(self, sock):
self.buf = [] # a list of chunks of strings
self.s = sock
self.closed = False
def close(self):
try:
self.s.shutdown(socket.SHUT_RDWR)
self.s.close()
except OSError:
# already closed
pass
finally:
self.closed = True
def _check_newline(self):
for chunk in self.buf:
if '\n' in chunk:
return True
return False
def readline(self):
# Receive until we get a newline, raise SocketClosed if socket is closed
while True:
try:
data = self.s.recv(256)
except OSError:
raise SocketClosed()
if not data:
raise SocketClosed()
self.buf.append(data)
if b'\n' in data:
break
# Combine chunks
allbytes = b''.join(self.buf)
head, tail = allbytes.split(b'\n', 1)
self.buf = [tail]
return head.decode()
def send(self, data):
try:
self.s.send(data)
except OSError:
raise SocketClosed()
class ProxyThread(QThread):
threads = {}
tiditer = count()
def __init__(self, target=None, args=tuple()):
global mainWidg
QThread.__init__(self)
self.f = target
self.args = args
self.tid = next(ProxyThread.tiditer)
ProxyThread.threads[self.tid] = self
self.finished.connect(clean_thread(self.tid))
def run(self):
self.f(*self.args)
def wait(self):
QThread.wait(self)
@classmethod
def waitall(cls):
ts = [(tid, thread) for tid, thread in cls.threads.items()]
for tid, thread in ts:
thread.wait()
def clean_thread(tid):
@pyqtSlot()
def clean():
del ProxyThread.threads[tid]
return clean
class Headers:
def __init__(self, headers=None):
self.headers = {}
if headers is not None:
if isinstance(headers, Headers):
for _, pairs in headers.headers.items():
for k, v in pairs:
self.add(k, v)
else:
for k, vs in headers.items():
for v in vs:
self.add(k, v)
def __contains__(self, hd):
for k, _ in self.headers.items():
if k.lower() == hd.lower():
return True
return False
def add(self, k, v):
try:
lst = self.headers[k.lower()]
lst.append((k, v))
except KeyError:
self.headers[k.lower()] = [(k, v)]
def set(self, k, v):
self.headers[k.lower()] = [(k, v)]
def get(self, k):
return self.headers[k.lower()][0][1]
def delete(self, k):
try:
del self.headers[k.lower()]
except KeyError:
pass
def pairs(self, key=None):
for _, kvs in self.headers.items():
for k, v in kvs:
if key is None or k.lower() == key.lower():
yield (k, v)
def dict(self):
retdict = {}
for _, kvs in self.headers.items():
for k, v in kvs:
if k in retdict:
retdict[k].append(v)
else:
retdict[k] = [v]
return retdict
class RequestContext:
def __init__(self, client, query=None):
self._current_query = []
self.client = client
if query is not None:
self._current_query = query
def _validate(self, query):
self.client.validate_query(query)
def set_query(self, query):
self._validate(query)
self._current_query = query
def apply_phrase(self, phrase):
self._validate([phrase])
self._current_query.append(phrase)
def pop_phrase(self):
if len(self._current_query) > 0:
self._current_query.pop()
def apply_filter(self, filt):
self._validate([[filt]])
self._current_query.append([filt])
@property
def query(self):
return copy.deepcopy(self._current_query)
class URL:
def __init__(self, url):
parsed = urlparse(url)
if url is not None:
parsed = urlparse(url)
self.scheme = parsed.scheme
self.netloc = parsed.netloc
self.path = parsed.path
self.params = parsed.params
self.query = parsed.query
self.fragment = parsed.fragment
else:
self.scheme = ""
self.netloc = ""
self.path = "/"
self.params = ""
self.query = ""
self.fragment = ""
def geturl(self, include_params=True):
params = self.params
query = self.query
fragment = self.fragment
if not include_params:
params = ""
query = ""
fragment = ""
r = ParseResult(scheme=self.scheme,
netloc=self.netloc,
path=self.path,
params=params,
query=query,
fragment=fragment)
return r.geturl()
def parameters(self):
try:
return parse_qs(self.query, keep_blank_values=True)
except Exception:
return {}
def param_iter(self):
for k, vs in self.parameters().items():
for v in vs:
yield k, v
def set_param(self, key, val):
params = self.parameters()
params[key] = [val]
self.query = urlencode(params)
def add_param(self, key, val):
params = self.parameters()
if key in params:
params[key].append(val)
else:
params[key] = [val]
self.query = urlencode(params)
def del_param(self, key):
params = self.parameters()
del params[key]
self.query = urlencode(params)
def set_params(self, params):
self.query = urlencode(params)
class InterceptMacro:
"""
A class representing a macro that modifies requests as they pass through the
proxy
"""
def __init__(self):
self.name = ''
self.intercept_requests = False
self.intercept_responses = False
self.intercept_ws = False
def __repr__(self):
return "<InterceptingMacro (%s)>" % self.name
def mangle_request(self, request):
return request
def mangle_response(self, request, response):
return response
def mangle_websocket(self, request, response, message):
return message
class HTTPRequest:
def __init__(self, method="GET", path="/", proto_major=1, proto_minor=1,
headers=None, body=bytes(), dest_host="", dest_port=80,
use_tls=False, time_start=None, time_end=None, db_id="",
tags=None, headers_only=False, storage_id=0):
# http info
self.method = method
self.url = URL(path)
self.proto_major = proto_major
self.proto_minor = proto_minor
self.headers = Headers(headers)
self.headers_only = headers_only
self._body = bytes()
if not headers_only:
self.body = body
# metadata
self.dest_host = dest_host
self.dest_port = dest_port
self.use_tls = use_tls
self.time_start = time_start
self.time_end = time_end
self.response = None
self.unmangled = None
self.ws_messages = []
self.db_id = db_id
self.storage_id = storage_id
if tags is not None:
self.tags = set(tags)
else:
self.tags = set()
@property
def body(self):
return self._body
@body.setter
def body(self, bs):
self.headers_only = False
if type(bs) is str:
self._body = bs.encode()
elif type(bs) is bytes:
self._body = bs
else:
raise Exception("invalid body type: {}".format(type(bs)))
self.headers.set("Content-Length", str(len(self._body)))
@property
def content_length(self):
if 'content-length' in self.headers:
return int(self.headers.get('content-length'))
return len(self.body)
def status_line(self):
sline = "{method} {path} HTTP/{proto_major}.{proto_minor}".format(
method=self.method, path=self.url.geturl(), proto_major=self.proto_major,
proto_minor=self.proto_minor).encode()
return sline
def headers_section(self):
message = self.status_line() + b"\r\n"
for k, v in self.headers.pairs():
message += "{}: {}\r\n".format(k, v).encode()
return message
def full_message(self):
message = self.headers_section()
message += b"\r\n"
message += self.body
return message
def parameters(self):
try:
return parse_qs(self.body.decode(), keep_blank_values=True)
except Exception:
return {}
def param_iter(self, ignore_content_type=False):
if not ignore_content_type:
if "content-type" not in self.headers:
return
if "www-form-urlencoded" not in self.headers.get("content-type").lower():
return
for k, vs in self.parameters().items():
for v in vs:
yield k, v
def set_param(self, key, val):
params = self.parameters()
params[key] = [val]
self.body = urlencode(params)
def add_param(self, key, val):
params = self.parameters()
if key in params:
params[key].append(val)
else:
params[key] = [val]
self.body = urlencode(params)
def del_param(self, key):
params = self.parameters()
del params[key]
self.body = urlencode(params)
def set_params(self, params):
self.body = urlencode(params)
def cookies(self):
try:
cookie = hcookies.BaseCookie()
cookie.load(self.headers.get("cookie"))
return cookie
except Exception as e:
return hcookies.BaseCookie()
def cookie_iter(self):
c = self.cookies()
for k in c:
yield k, c[k].value
def set_cookie(self, key, val):
c = self.cookies()
c[key] = val
self.set_cookies(c)
def del_cookie(self, key):
c = self.cookies()
del c[key]
self.set_cookies(c)
def set_cookies(self, c):
if isinstance(c, hcookies.BaseCookie):
# it's a basecookie
cookie_pairs = []
for k in c:
cookie_pairs.append('{}={}'.format(k, c[k].value))
header_str = '; '.join(cookie_pairs)
elif isinstance(c, HTTPRequest):
# it's a request we should copy cookies from
try:
header_str = c.headers.get("Cookie")
except KeyError:
header_str = ""
else:
# it's a dictionary
cookie_pairs = []
for k, v in c.items():
cookie_pairs.append('{}={}'.format(k, v))
header_str = '; '.join(cookie_pairs)
if header_str == '':
try:
self.headers.delete("Cookie")
except KeyError:
pass
else:
self.headers.set("Cookie", header_str)
def add_cookies(self, c):
new_cookies = self.cookies()
if isinstance(c, hcookies.BaseCookie):
for k in c:
new_cookies[k] = c[k].value
elif isinstance(c, HTTPRequest):
for k, v in c.cookie_iter():
new_cookies[k] = v
elif isinstance(c, HTTPResponse):
for k, v in c.cookie_iter():
new_cookies[k] = v
else:
for k, v in c.items():
new_cookies[k] = v
self.set_cookies(new_cookies)
def full_url(self):
return get_full_url(self)
def copy(self):
return HTTPRequest(
method=self.method,
path=self.url.geturl(),
proto_major=self.proto_major,
proto_minor=self.proto_minor,
headers=self.headers,
body=self.body,
dest_host=self.dest_host,
dest_port=self.dest_port,
use_tls=self.use_tls,
tags=copy.deepcopy(self.tags),
headers_only=self.headers_only,
)
class HTTPResponse:
def __init__(self, status_code=200, reason="OK", proto_major=1, proto_minor=1,
headers=None, body=bytes(), db_id="", headers_only=False, storage_id=0):
self.status_code = status_code
self.reason = reason
self.proto_major = proto_major
self.proto_minor = proto_minor
self.headers = Headers()
if headers is not None:
for k, vs in headers.items():
for v in vs:
self.headers.add(k, v)
self.headers_only = headers_only
self._body = bytes()
if not headers_only:
self.body = body
self.unmangled = None
self.db_id = db_id
self.storage = storage_id
@property
def body(self):
return self._body
@body.setter
def body(self, bs):
self.headers_only = False
if type(bs) is str:
self._body = bs.encode()
elif type(bs) is bytes:
self._body = bs
else:
raise Exception("invalid body type: {}".format(type(bs)))
self.headers.set("Content-Length", str(len(self._body)))
@property
def content_length(self):
if 'content-length' in self.headers:
return int(self.headers.get('content-length'))
return len(self.body)
def status_line(self):
sline = "HTTP/{proto_major}.{proto_minor} {status_code} {reason}".format(
proto_major=self.proto_major, proto_minor=self.proto_minor,
status_code=self.status_code, reason=self.reason).encode()
return sline
def headers_section(self):
message = self.status_line() + b"\r\n"
for k, v in self.headers.pairs():
message += "{}: {}\r\n".format(k, v).encode()
return message
def full_message(self):
message = self.headers_section()
message += b"\r\n"
message += self.body
return message
def cookies(self):
try:
cookie = hcookies.BaseCookie()
for _, v in self.headers.pairs('set-cookie'):
cookie.load(v)
return cookie
except Exception as e:
return hcookies.BaseCookie()
def cookie_iter(self):
c = self.cookies()
for k in c:
yield k, c[k].value
def set_cookie(self, key, val):
c = self.cookies()
c[key] = val
self.set_cookies(c)
def del_cookie(self, key):
c = self.cookies()
del c[key]
self.set_cookies(c)
def set_cookies(self, c):
self.headers.delete("set-cookie")
if isinstance(c, hcookies.BaseCookie):
cookies = c
else:
cookies = hcookies.BaseCookie()
for k, v in c.items():
cookies[k] = v
for _, m in c.items():
self.headers.add("Set-Cookie", m.OutputString())
def copy(self):
return HTTPResponse(
status_code=self.status_code,
reason=self.reason,
proto_major=self.proto_major,
proto_minor=self.proto_minor,
headers=self.headers.headers,
body=self.body,
headers_only=self.headers_only,
)
class WSMessage:
def __init__(self, is_binary=True, message=bytes(), to_server=True,
timestamp=None, db_id="", storage_id=0):
self.is_binary = is_binary
self.message = message
self.to_server = to_server
self.timestamp = timestamp or datetime.datetime(1970, 1, 1)
self.unmangled = None
self.db_id = db_id
self.storage = storage_id
def copy(self):
return WSMessage(
is_binary=self.is_binary,
message=self.message,
to_server=self.to_server,
)
ScopeResult = namedtuple("ScopeResult", ["is_custom", "filter"])
ListenerResult = namedtuple("ListenerResult", ["lid", "addr"])
GenPemCertsResult = namedtuple("GenPemCertsResult", ["key_pem", "cert_pem"])
SavedQuery = namedtuple("SavedQuery", ["name", "query"])
SavedStorage = namedtuple("SavedStorage", ["storage_id", "description"])
def messagingFunction(func):
def f(self, *args, **kwargs):
if self.is_interactive:
raise MessageError("cannot be called while other message is interactive")
if self.closed:
raise MessageError("connection is closed")
with self.message_lock:
return func(self, *args, **kwargs)
return f
class ProxyConnection:
next_id = 1
def __init__(self, kind="", addr=""):
self.connid = ProxyConnection.next_id
ProxyConnection.next_id += 1
self.sbuf = None
self.buf = bytes()
self.parent_client = None
self.debug = False
self.is_interactive = False
self.closed = True
self.message_lock = threading.Lock()
self.kind = None
self.addr = None
self.int_thread = None
if kind.lower() == "tcp":
tcpaddr, port = addr.rsplit(":", 1)
self.connect_tcp(tcpaddr, int(port))
elif kind.lower() == "unix":
self.connect_unix(addr)
def __enter__(self):
return self
def __exit__(self, exc_type, exc_value, traceback):
self.close()
def connect_tcp(self, addr, port):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((addr, port))
self.sbuf = SockBuffer(s)
self.closed = False
self.kind = "tcp"
self.addr = "{}:{}".format(addr, port)
def connect_unix(self, addr):
s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
s.connect(addr)
self.sbuf = SockBuffer(s)
self.closed = False
self.kind = "unix"
self.addr = addr
@property
def maddr(self):
if self.kind is not None:
return "{}:{}".format(self.kind, self.addr)
else:
return None
def close(self):
self.sbuf.close()
if self.parent_client is not None:
try:
self.parent_client.conns.remove(self)
except KeyError:
pass
self.closed = True
def read_message(self):
ln = self.sbuf.readline()
if self.debug:
print("<({}) {}".format(self.connid, ln))
j = json.loads(ln)
if ("Success" in j) and (j["Success"] is False):
if "Reason" in j:
raise MessageError(j["Reason"])
raise MessageError("unknown error")
return j
def submit_command(self, cmd):
ln = json.dumps(cmd).encode() + b"\n"
if self.debug:
print(">({}) {}".format(self.connid, ln.decode()[:-1]))
self.sbuf.send(ln)
def reqrsp_cmd(self, cmd):
self.submit_command(cmd)
ret = self.read_message()
if ret is None:
raise Exception()
return ret
###########
# Commands
@messagingFunction
def ping(self):
cmd = {"Command": "Ping"}
result = self.reqrsp_cmd(cmd)
return result["Ping"]
@messagingFunction
def submit(self, req, storage=0):
cmd = {
"Command": "Submit",
"Request": encode_req(req),
"Storage": 0,
}
if storage is not None:
cmd["Storage"] = storage
result = self.reqrsp_cmd(cmd)
if "SubmittedRequest" not in result:
raise MessageError("no request returned")
newreq = decode_req(result["SubmittedRequest"], storage=storage)
req.response = newreq.response
req.unmangled = newreq.unmangled
req.time_start = newreq.time_start
req.time_end = newreq.time_end
req.db_id = newreq.db_id
req.storage_id = storage
@messagingFunction
def save_new(self, req, storage):
cmd = {
"Command": "SaveNew",
"Request": encode_req(req),
"Storage": storage,
}
result = self.reqrsp_cmd(cmd)
req.db_id = result["DbId"]
req.storage_id = storage
return result["DbId"]
def _query_storage(self, q, storage, headers_only=False, max_results=0):
cmd = {
"Command": "StorageQuery",
"Query": q,
"HeadersOnly": headers_only,
"MaxResults": max_results,
"Storage": storage,
}
result = self.reqrsp_cmd(cmd)
reqs = []
unmangled = set()
for reqd in result["Results"]:
req = decode_req(reqd, headers_only=headers_only, storage=storage)
req.storage_id = storage
reqs.append(req)
if req.unmangled is not None:
unmangled.add(req.unmangled.db_id)
return [r for r in reqs if r.db_id not in unmangled]
@messagingFunction
def query_storage(self, q, storage, max_results=0, headers_only=False):
return self._query_storage(q, storage, headers_only=headers_only, max_results=max_results)
@messagingFunction
def req_by_id(self, reqid, storage, headers_only=False):
results = self._query_storage([[["dbid", "is", reqid]]], storage,
headers_only=headers_only, max_results=1)
if len(results) == 0:
raise MessageError("request with id {} does not exist".format(reqid))
return results[0]
@messagingFunction
def set_scope(self, filt):
cmd = {
"Command": "SetScope",
"Query": filt,
}
self.reqrsp_cmd(cmd)
@messagingFunction
def get_scope(self):
cmd = {
"Command": "ViewScope",
}
result = self.reqrsp_cmd(cmd)
ret = ScopeResult(result["IsCustom"], result["Query"])
return ret
@messagingFunction
def add_tag(self, reqid, tag, storage):
cmd = {
"Command": "AddTag",
"ReqId": reqid,
"Tag": tag,
"Storage": storage,
}
self.reqrsp_cmd(cmd)
@messagingFunction
def remove_tag(self, reqid, tag, storage):
cmd = {
"Command": "RemoveTag",
"ReqId": reqid,
"Tag": tag,
"Storage": storage,
}
self.reqrsp_cmd(cmd)
@messagingFunction
def clear_tag(self, reqid, storage):
cmd = {
"Command": "ClearTag",
"ReqId": reqid,
"Storage": storage,
}
self.reqrsp_cmd(cmd)
@messagingFunction
def all_saved_queries(self, storage):
cmd = {
"Command": "AllSavedQueries",
"Storage": storage,
}
results = self.reqrsp_cmd(cmd)
queries = []
for result in results["Queries"]:
queries.append(SavedQuery(name=result["Name"], query=result["Query"]))
return queries
@messagingFunction
def save_query(self, name, filt, storage):
cmd = {
"Command": "SaveQuery",
"Name": name,
"Query": filt,
"Storage": storage,
}
self.reqrsp_cmd(cmd)
@messagingFunction
def load_query(self, name, storage):
cmd = {
"Command": "LoadQuery",
"Name": name,
"Storage": storage,
}
result = self.reqrsp_cmd(cmd)
return result["Query"]
@messagingFunction
def delete_query(self, name, storage):
cmd = {
"Command": "DeleteQuery",
"Name": name,
"Storage": storage,
}
self.reqrsp_cmd(cmd)
@messagingFunction
def add_listener(self, addr, port, transparent=False, destHost="",
destPort=0, destUseTLS=False):
laddr = "{}:{}".format(addr, port)
cmd = {
"Command": "AddListener",
"Type": "tcp",
"Addr": laddr,
"TransparentMode": transparent,
"DestHost": destHost,
"DestPort": destPort,
"DestUseTLS": destUseTLS,
}
result = self.reqrsp_cmd(cmd)
lid = result["Id"]
return lid
@messagingFunction
def remove_listener(self, lid):
cmd = {
"Command": "RemoveListener",
"Id": lid,
}
self.reqrsp_cmd(cmd)
@messagingFunction
def get_listeners(self):
cmd = {
"Command": "GetListeners",
}
result = self.reqrsp_cmd(cmd)
results = []
for r in result["Results"]:
results.append((r["Id"], r["Addr"]))
return results
@messagingFunction
def load_certificates(self, cert_file, pkey_file):
cmd = {
"Command": "LoadCerts",
"KeyFile": pkey_file,
"CertificateFile": cert_file,
}
self.reqrsp_cmd(cmd)
@messagingFunction
def set_certificates(self, pkey_pem, cert_pem):
cmd = {
"Command": "SetCerts",
"KeyPEMData": pkey_pem,
"CertificatePEMData": cert_pem,
}
self.reqrsp_cmd(cmd)
@messagingFunction
def clear_certificates(self):
cmd = {
"Command": "ClearCerts",
}
self.reqrsp_cmd(cmd)
@messagingFunction
def generate_certificates(self, pkey_file, cert_file):
cmd = {
"Command": "GenCerts",
"KeyFile": pkey_file,
"CertFile": cert_file,
}
self.reqrsp_cmd(cmd)
@messagingFunction
def generate_pem_certificates(self):
cmd = {
"Command": "GenPEMCerts",
}
result = self.reqrsp_cmd(cmd)
ret = GenPemCertsResult(result["KeyPEMData"], result["CertificatePEMData"])
return ret
@messagingFunction
def validate_query(self, query):
cmd = {
"Command": "ValidateQuery",
"Query": query,
}
try:
self.reqrsp_cmd(cmd)
except MessageError as e:
raise InvalidQuery(str(e))
@messagingFunction
def check_request(self, query, req=None, storage_id=-1, db_id=""):
cmd = {
"Command": "checkrequest",
"Query": query,
}
if req:
cmd["Request"] = encode_req(req)
if db_id != "":
cmd["DbId"] = db_id
cmd["StorageId"] = storage_id
result = self.reqrsp_cmd(cmd)
return result["Result"]
@messagingFunction
def add_sqlite_storage(self, path, desc):
cmd = {
"Command": "AddSQLiteStorage",
"Path": path,
"Description": desc
}
result = self.reqrsp_cmd(cmd)
return result["StorageId"]
@messagingFunction
def add_in_memory_storage(self, desc):
cmd = {
"Command": "AddInMemoryStorage",
"Description": desc
}
result = self.reqrsp_cmd(cmd)
return result["StorageId"]
@messagingFunction
def close_storage(self, storage_id):
cmd = {
"Command": "CloseStorage",
"StorageId": storage_id,
}
self.reqrsp_cmd(cmd)
@messagingFunction
def set_proxy_storage(self, storage_id):
cmd = {
"Command": "SetProxyStorage",
"StorageId": storage_id,
}
self.reqrsp_cmd(cmd)
@messagingFunction
def list_storage(self):
cmd = {
"Command": "ListStorage",
}
result = self.reqrsp_cmd(cmd)
ret = []
for ss in result["Storages"]:
ret.append(SavedStorage(ss["Id"], ss["Description"]))
return ret
@messagingFunction
def set_proxy(self, use_proxy=False, proxy_host="", proxy_port=0, use_creds=False,
username="", password="", is_socks=False):
cmd = {
"Command": "SetProxy",
"UseProxy": use_proxy,
"ProxyHost": proxy_host,
"ProxyPort": proxy_port,
"ProxyIsSOCKS": is_socks,
"UseCredentials": use_creds,
"Username": username,
"Password": password,
}
self.reqrsp_cmd(cmd)
@messagingFunction
def intercept(self, macro):
# Run an intercepting macro until closed
# Start intercepting
self.is_interactive = True
cmd = {
"Command": "Intercept",
"InterceptRequests": macro.intercept_requests,
"InterceptResponses": macro.intercept_responses,
"InterceptWS": macro.intercept_ws,
}
try:
self.reqrsp_cmd(cmd)
except Exception as e:
self.is_interactive = False
raise e
def run_macro():
iditer = count()
threads = {}
while True:
try:
msg = self.read_message()
except MessageError as e:
return
except SocketClosed:
return
def mangle_and_respond(msg):
retCmd = None
if msg["Type"] == "httprequest":
req = decode_req(msg["Request"])
newReq = macro.mangle_request(req)
if newReq is None:
retCmd = {
"Id": msg["Id"],
"Dropped": True,
}
else:
newReq.unmangled = None
newReq.response = None
newReq.ws_messages = []
retCmd = {
"Id": msg["Id"],
"Dropped": False,
"Request": encode_req(newReq),
}
elif msg["Type"] == "httpresponse":
req = decode_req(msg["Request"])
rsp = decode_rsp(msg["Response"])
newRsp = macro.mangle_response(req, rsp)
if newRsp is None:
retCmd = {
"Id": msg["Id"],
"Dropped": True,
}
else:
newRsp.unmangled = None
retCmd = {
"Id": msg["Id"],
"Dropped": False,
"Response": encode_rsp(newRsp),
}
elif msg["Type"] == "wstoserver" or msg["Type"] == "wstoclient":
req = decode_req(msg["Request"])
rsp = decode_rsp(msg["Response"])
wsm = decode_ws(msg["WSMessage"])
newWsm = macro.mangle_websocket(req, rsp, wsm)
if newWsm is None:
retCmd = {
"Id": msg["Id"],
"Dropped": True,
}
else:
newWsm.unmangled = None
retCmd = {
"Id": msg["Id"],
"Dropped": False,
"WSMessage": encode_ws(newWsm),
}
else:
raise Exception("Unknown message type: " + msg["Type"])
if retCmd is not None:
try:
self.submit_command(retCmd)
except SocketClosed:
return
tid = next(iditer)
mangle_thread = ProxyThread(target=mangle_and_respond,
args=(msg,))
threads[tid] = mangle_thread
mangle_thread.start()
self.int_thread = ProxyThread(target=run_macro)
self.int_thread.start()
@messagingFunction
def watch_storage(self, storage_id=-1, headers_only=True):
# Generator that generates request, response, and wsmessages as they
# are stored by the proxy
cmd = {
"Command": "WatchStorage",
"StorageId": storage_id,
"HeadersOnly": headers_only,
}
try:
self.reqrsp_cmd(cmd)
except Exception as e:
self.is_interactive = False
raise e
while True:
msg = self.read_message()
if msg["Request"]:
msg["Request"] = decode_req(msg["Request"],
storage=msg["StorageId"],
headers_only=headers_only)
if msg["Response"]:
msg["Response"] = decode_rsp(msg["Response"],
storage=msg["StorageId"],
headers_only=headers_only)
if msg["WSMessage"]:
msg["WSMessage"] = decode_ws(msg["WSMessage"],
storage=msg["StorageId"],
headers_only=headers_only)
yield msg
@messagingFunction
def set_plugin_value(self, key, value, storage_id):
cmd = {
"Command": "SetPluginValue",
"Storage": storage_id,
"Key": key,
"Value": value,
}
self.reqrsp_cmd(cmd)
@messagingFunction
def get_plugin_value(self, key, storage_id):
cmd = {
"Command": "GetPluginValue",
"Storage": storage_id,
"Key": key,
}
result = self.reqrsp_cmd(cmd)
return result["Value"]
ActiveStorage = namedtuple("ActiveStorage", ["type", "storage_id", "prefix"])
def _serialize_storage(stype, prefix):
return "{}|{}".format(stype, prefix)
class ProxyClient:
def __init__(self, binary=None, debug=False, conn_addr=None):
self.binloc = binary
self.proxy_proc = None
self.ltype = None
self.laddr = None
self.debug = debug
self.conn_addr = conn_addr
self.conns = set()
self.msg_conn = None # conn for single req/rsp messages
self.context = RequestContext(self)
self.storage_by_id = {}
self.storage_by_prefix = {}
self.proxy_storage = None
self.inmem_storage = None
self.reqrsp_methods = {
"submit_command",
# "reqrsp_cmd",
"ping",
# "submit",
# "save_new",
# "query_storage",
# "req_by_id",
"set_scope",
"get_scope",
# "add_tag",
# "remove_tag",
# "clear_tag",
"all_saved_queries",
"save_query",
"load_query",
"delete_query",
"add_listener",
"remove_listener",
"get_listeners",
"load_certificates",
"set_certificates",
"clear_certificates",
"generate_certificates",
"generate_pem_certificates",
"validate_query",
# "check_request",
"list_storage",
# "add_sqlite_storage",
# "add_in_memory_storage",
# "close_storage",
# "set_proxy_storage",
"set_proxy",
# "set_plugin_value",
# "get_plugin_value",
}
def __enter__(self):
if self.conn_addr is not None:
self.msg_connect(self.conn_addr)
else:
self.execute_binary(binary=self.binloc, debug=self.debug)
return self
def __exit__(self, exc_type, exc_value, traceback):
self.close()
def __getattr__(self, name):
if name in self.reqrsp_methods:
return getattr(self.msg_conn, name)
raise NotImplementedError(name)
@property
def maddr(self):
if self.ltype is not None:
return "{}:{}".format(self.ltype, self.laddr)
else:
return None
def execute_binary(self, binary=None, debug=False, listen_addr=None):
self.binloc = binary
args = [self.binloc]
if listen_addr is not None:
args += ["--msglisten", listen_addr]
else:
args += ["--msgauto"]
if debug:
args += ["--dbg"]
self.proxy_proc = Popen(args, stdout=PIPE, stderr=PIPE)
# Wait for it to start and make connection
listenstr = self.proxy_proc.stdout.readline().rstrip()
self.msg_connect(listenstr.decode())
def msg_connect(self, addr):
self.ltype, self.laddr = addr.split(":", 1)
self.msg_conn = self.new_conn()
self._get_storage()
def close(self):
conns = list(self.conns)
for conn in conns:
conn.close()
if self.proxy_proc is not None:
self.proxy_proc.terminate()
def new_conn(self):
conn = ProxyConnection(kind=self.ltype, addr=self.laddr)
conn.parent_client = self
conn.debug = self.debug
self.conns.add(conn)
return conn
# functions involving storage
def _add_storage(self, storage, prefix):
self.storage_by_prefix[prefix] = storage
self.storage_by_id[storage.storage_id] = storage
def _clear_storage(self):
self.storage_by_prefix = {}
self.storage_by_id = {}
def _get_storage(self):
self._clear_storage()
storages = self.list_storage()
for s in storages:
stype, prefix = s.description.split("|")
storage = ActiveStorage(stype, s.storage_id, prefix)
self._add_storage(storage, prefix)
def parse_reqid(self, reqid):
if reqid[0].isalpha():
prefix = reqid[0]
realid = reqid[1:]
else:
prefix = ""
realid = reqid
# `u`, `s` are special cases for the unmangled version of req and rsp
if prefix == 'u':
req = self.req_by_id(realid)
if req.unmangled is None:
raise MessageError("request %s was not mangled" % reqid)
ureq = req.unmangled
return self.storage_by_id[ureq.storage_id], ureq.db_id
elif prefix == 's':
req = self.req_by_id(realid)
if req.response is None:
raise MessageError("response %s was not mangled" % reqid)
if req.response.unmangled is None:
raise MessageError("response %s was not mangled" % reqid)
return self.storage_by_id[req.storage_id], req.db_id
else:
storage = self.storage_by_prefix[prefix]
return storage, realid
def storage_iter(self):
for _, s in self.storage_by_id.items():
yield s
def _stg_or_def(self, storage):
if storage is None:
return self.proxy_storage
return storage
def is_in_context(self, req):
return self.check_request(self.context.query, req)
def in_context_requests(self, headers_only=False, max_results=0):
return self.query_storage(self.context.query,
headers_only=headers_only,
max_results=max_results)
def in_context_requests_async(self, slot, headers_only=False, max_results=0, *args, **kwargs):
return self.query_storage(slot,
self.context.query,
headers_only=headers_only,
max_results=max_results)
def in_context_requests_iter(self, headers_only=False, max_results=0):
results = self.query_storage(self.context.query,
headers_only=headers_only,
max_results=max_results)
ret = results
if max_results > 0 and len(results) > max_results:
ret = results[:max_results]
for reqh in ret:
req = self.req_by_id(reqh.db_id, storage_id=reqh.storage_id)
yield req
def get_reqid(self, req):
prefix = ""
if req.storage_id in self.storage_by_id:
s = self.storage_by_id[req.storage_id]
prefix = s.prefix
return "{}{}".format(prefix, req.db_id)
def load_by_reqheaders(self, req):
reqid = self.get_reqid(req)
return self.req_by_id(reqid)
# functions that don't just pass through to underlying conn
def add_sqlite_storage(self, path, prefix):
desc = _serialize_storage("sqlite", prefix)
sid = self.msg_conn.add_sqlite_storage(path, desc)
s = ActiveStorage(type="sqlite", storage_id=sid, prefix=prefix)
self._add_storage(s, prefix)
return s
def add_in_memory_storage(self, prefix):
desc = _serialize_storage("inmem", prefix)
sid = self.msg_conn.add_in_memory_storage(desc)
s = ActiveStorage(type="inmem", storage_id=sid, prefix=prefix)
self._add_storage(s, prefix)
return s
def close_storage(self, storage_id):
s = self.storage_by_id[storage_id]
self.msg_conn.close_storage(s.storage_id)
del self.storage_by_id[s.storage_id]
del self.storage_by_prefix[s.prefix]
def set_proxy_storage(self, storage_id):
s = self.storage_by_id[storage_id]
self.msg_conn.set_proxy_storage(s.storage_id)
self.proxy_storage = storage_id
def set_storage_prefix(self, storage_id, prefix):
if prefix in self.storage_by_prefix:
raise Exception("prefix already exists")
s = self.storage_by_id[storage_id]
del self.storage_by_prefix[s.prefix]
news = ActiveStorage(type=s.type, prefix=prefix, storage_id=s.storage_id)
self.storage_by_prefix[news.prefix] = news
self.storage_by_id[storage_id] = news
def save_new(self, req, inmem=False, storage=None):
if inmem:
storage = self.inmem_storage
else:
storage = self._stg_or_def(storage)
self.msg_conn.save_new(req, storage=storage)
def submit(self, req, save=False, inmem=False, storage=None):
if save:
storage = self._stg_or_def(storage)
if inmem:
storage = self.inmem_storage
self.msg_conn.submit(req, storage=storage)
def query_storage(self, q, max_results=0, headers_only=False, storage=None, conn=None):
results = []
conn = conn or self.msg_conn
if storage is None:
for s in self.storage_iter():
results += conn.query_storage(q, max_results=max_results,
headers_only=headers_only,
storage=s.storage_id)
else:
results += conn.query_storage(q, max_results=max_results,
headers_only=headers_only,
storage=storage)
def kfunc(req):
if req.time_start is None:
return datetime.datetime.utcfromtimestamp(0)
return req.time_start
results.sort(key=kfunc)
results = [r for r in reversed(results)]
return results
def query_storage_async(self, slot, *args, **kwargs):
def perform_query():
try:
with self.new_conn() as c:
r = self.query_storage(*args, conn=c, **kwargs)
slot.emit(r)
except Exception:
pass
ProxyThread(target=perform_query).start()
def req_by_id(self, reqid, storage_id=None, headers_only=False):
if storage_id is None:
storage, db_id = self.parse_reqid(reqid)
storage_id = storage.storage_id
else:
db_id = reqid
retreq = self.msg_conn.req_by_id(db_id, headers_only=headers_only,
storage=storage_id)
if reqid[0] == 's': # `u` is handled by parse_reqid
retreq.response = retreq.response.unmangled
return retreq
def check_request(self, query, req=None, reqid=""):
if req is not None:
return self.msg_conn.check_request(query, req=req)
else:
storage, db_id = self.parse_reqid(reqid)
storage_id = storage.storage_id
return self.msg_conn.check_request(query, storage_id=storage_id, db_id=db_id)
raise Exception("check_request requires either a request or reqid")
# for these and submit, might need storage stored on the request itself
def add_tag(self, reqid, tag, storage=None):
self.msg_conn.add_tag(reqid, tag, storage=self._stg_or_def(storage))
def remove_tag(self, reqid, tag, storage=None):
self.msg_conn.remove_tag(reqid, tag, storage=self._stg_or_def(storage))
def clear_tag(self, reqid, storage=None):
self.msg_conn.clear_tag(reqid, storage=self._stg_or_def(storage))
def all_saved_queries(self, storage=None):
self.msg_conn.all_saved_queries(storage=None)
def save_query(self, name, filt, storage=None):
self.msg_conn.save_query(name, filt, storage=self._stg_or_def(storage))
def load_query(self, name, storage=None):
self.msg_conn.load_query(name, storage=self._stg_or_def(storage))
def delete_query(self, name, storage=None):
self.msg_conn.delete_query(name, storage=self._stg_or_def(storage))
def set_plugin_value(self, key, value, storage=None):
self.msg_conn.set_plugin_value(key, value, self._stg_or_def(storage))
def get_plugin_value(self, key, storage=None):
return self.msg_conn.get_plugin_value(key, self._stg_or_def(storage))
def decode_req(result, headers_only=False, storage=0):
if "StartTime" in result and result["StartTime"] > 0:
time_start = time_from_nsecs(result["StartTime"])
else:
time_start = None
if "EndTime" in result and result["EndTime"] > 0:
time_end = time_from_nsecs(result["EndTime"])
else:
time_end = None
if "DbId" in result:
db_id = result["DbId"]
else:
db_id = ""
if "Tags" in result:
tags = result["Tags"]
else:
tags = ""
ret = HTTPRequest(
method=result["Method"],
path=result["Path"],
proto_major=result["ProtoMajor"],
proto_minor=result["ProtoMinor"],
headers=copy.deepcopy(result["Headers"]),
body=base64.b64decode(result["Body"]),
dest_host=result["DestHost"],
dest_port=result["DestPort"],
use_tls=result["UseTLS"],
time_start=time_start,
time_end=time_end,
tags=tags,
headers_only=headers_only,
db_id=db_id,
storage_id=storage)
if "Unmangled" in result:
ret.unmangled = decode_req(result["Unmangled"], headers_only=headers_only, storage=storage)
if "Response" in result:
ret.response = decode_rsp(result["Response"], headers_only=headers_only, storage=storage)
if "WSMessages" in result:
for wsm in result["WSMessages"]:
ret.ws_messages.append(decode_ws(wsm, storage=storage))
return ret
def decode_rsp(result, headers_only=False, storage=0):
ret = HTTPResponse(
status_code=result["StatusCode"],
reason=result["Reason"],
proto_major=result["ProtoMajor"],
proto_minor=result["ProtoMinor"],
headers=copy.deepcopy(result["Headers"]),
body=base64.b64decode(result["Body"]),
headers_only=headers_only,
storage_id=storage,
)
if "Unmangled" in result:
ret.unmangled = decode_rsp(result["Unmangled"], headers_only=headers_only, storage=storage)
return ret
def decode_ws(result, storage=0):
timestamp = None
db_id = ""
if "Timestamp" in result:
timestamp = time_from_nsecs(result["Timestamp"])
if "DbId" in result:
db_id = result["DbId"]
ret = WSMessage(
is_binary=result["IsBinary"],
message=base64.b64decode(result["Message"]),
to_server=result["ToServer"],
timestamp=timestamp,
db_id=db_id,
storage_id=storage,
)
if "Unmangled" in result:
ret.unmangled = decode_ws(result["Unmangled"], storage=storage)
return ret
def encode_req(req, int_rsp=False):
msg = {
"DestHost": req.dest_host,
"DestPort": req.dest_port,
"UseTLS": req.use_tls,
"Method": req.method,
"Path": req.url.geturl(),
"ProtoMajor": req.proto_major,
"ProtoMinor": req.proto_major,
"Headers": req.headers.dict(),
"Tags": list(req.tags),
"Body": base64.b64encode(copy.copy(req.body)).decode(),
}
if not int_rsp:
msg["StartTime"] = time_to_nsecs(req.time_start)
msg["EndTime"] = time_to_nsecs(req.time_end)
if req.unmangled is not None:
msg["Unmangled"] = encode_req(req.unmangled)
if req.response is not None:
msg["Response"] = encode_rsp(req.response)
msg["WSMessages"] = []
for wsm in req.ws_messages:
msg["WSMessages"].append(encode_ws(wsm))
return msg
def encode_rsp(rsp, int_rsp=False):
msg = {
"ProtoMajor": rsp.proto_major,
"ProtoMinor": rsp.proto_minor,
"StatusCode": rsp.status_code,
"Reason": rsp.reason,
"Headers": rsp.headers.dict(),
"Body": base64.b64encode(copy.copy(rsp.body)).decode(),
}
if not int_rsp:
if rsp.unmangled is not None:
msg["Unmangled"] = encode_rsp(rsp.unmangled)
return msg
def encode_ws(ws, int_rsp=False):
msg = {
"Message": base64.b64encode(ws.message).decode(),
"IsBinary": ws.is_binary,
"toServer": ws.to_server,
}
if not int_rsp:
if ws.unmangled is not None:
msg["Unmangled"] = encode_ws(ws.unmangled)
msg["Timestamp"] = time_to_nsecs(ws.timestamp)
msg["DbId"] = ws.db_id
return msg
def time_from_nsecs(nsecs):
secs = nsecs / 1000000000
t = datetime.datetime.utcfromtimestamp(secs)
return t
def time_to_nsecs(t):
if t is None:
return None
secs = (t - datetime.datetime(1970, 1, 1)).total_seconds()
return int(math.floor(secs * 1000000000))
RequestStatusLine = namedtuple("RequestStatusLine", ["method", "path", "proto_major", "proto_minor"])
ResponseStatusLine = namedtuple("ResponseStatusLine", ["proto_major", "proto_minor", "status_code", "reason"])
def parse_req_sline(sline):
if len(sline.split(b' ')) == 3:
verb, path, version = sline.split(b' ')
elif len(sline.split(b' ')) == 2:
verb, version = sline.split(b' ')
path = b''
else:
raise Exception("malformed statusline")
raw_version = version[5:] # strip HTTP/
pmajor, pminor = raw_version.split(b'.', 1)
return RequestStatusLine(verb.decode(), path.decode(), int(pmajor), int(pminor))
def parse_rsp_sline(sline):
if len(sline.split(b' ')) > 2:
version, status_code, reason = sline.split(b' ', 2)
else:
version, status_code = sline.split(b' ', 1)
reason = ''
raw_version = version[5:] # strip HTTP/
pmajor, pminor = raw_version.split(b'.', 1)
return ResponseStatusLine(int(pmajor), int(pminor), int(status_code), reason.decode())
def _parse_message(bs, sline_parser):
header_env, body = re.split(br"(?:\r\n|\n)(?:\r\n|\n)", bs, 1)
status_line, header_bytes = re.split(b"\r?\n", header_env, 1)
h = Headers()
for l in re.split(br"\r?\n", header_bytes):
k, v = l.split(b": ", 1)
if k.lower != 'content-length':
h.add(k.decode(), v.decode())
h.add("Content-Length", str(len(body)))
return (sline_parser(status_line), h, body)
def parse_request(bs, dest_host='', dest_port=80, use_tls=False):
req_sline, headers, body = _parse_message(bs, parse_req_sline)
req = HTTPRequest(
method=req_sline.method,
path=req_sline.path,
proto_major=req_sline.proto_major,
proto_minor=req_sline.proto_minor,
headers=headers.dict(),
body=body,
dest_host=dest_host,
dest_port=dest_port,
use_tls=use_tls)
return req
def parse_response(bs):
rsp_sline, headers, body = _parse_message(bs, parse_rsp_sline)
rsp = HTTPResponse(
status_code=rsp_sline.status_code,
reason=rsp_sline.reason,
proto_major=rsp_sline.proto_major,
proto_minor=rsp_sline.proto_minor,
headers=headers.dict(),
body=body)
return rsp
def get_full_url(req):
netloc = req.dest_host
if req.use_tls:
scheme = "https"
if req.dest_port != 443:
netloc = "%s:%d" % (req.dest_host, req.dest_port)
else:
scheme = "http"
if req.dest_port != 80:
netloc = "%s:%d" % (req.dest_host, req.dest_port)
rpath = req.url
u = URL("")
u.scheme = scheme
u.netloc = netloc
u.path = rpath.path
u.params = rpath.params
u.query = rpath.query
u.fragment = rpath.fragment
return u.geturl()
================================================
FILE: guppyproxy/repeater.py
================================================
from guppyproxy.util import display_error_box
from guppyproxy.reqview import ReqViewWidget
from PyQt5.QtWidgets import QWidget, QVBoxLayout, QHBoxLayout, QPushButton, QLineEdit, QCheckBox, QLabel, QSizePolicy, QToolButton
from PyQt5.QtCore import pyqtSlot
class RepeaterWidget(QWidget):
def __init__(self, client):
QWidget.__init__(self)
self.client = client
self.history = []
self.history_pos = 0
self.setLayout(QVBoxLayout())
self.layout().setSpacing(0)
self.layout().setContentsMargins(0, 0, 0, 0)
buttons = QHBoxLayout()
buttons.setContentsMargins(0, 0, 0, 0)
buttons.setSpacing(8)
submitButton = QPushButton("Submit")
submitButton.clicked.connect(self.submit)
self.dest_host_input = QLineEdit()
self.dest_port_input = QLineEdit()
self.dest_port_input.setMaxLength(5)
self.dest_port_input.setSizePolicy(QSizePolicy.Minimum, QSizePolicy.Preferred)
self.dest_usetls_input = QCheckBox()
self.back_button = QToolButton()
self.back_button.setText("<")
self.back_button.clicked.connect(self.back)
self.forward_button = QToolButton()
self.forward_button.setText(">")
self.forward_button.clicked.connect(self.forward)
buttons.addWidget(self.back_button)
buttons.addWidget(self.forward_button)
buttons.addWidget(submitButton)
buttons.addWidget(QLabel("Host:"))
buttons.addWidget(self.dest_host_input)
buttons.addWidget(QLabel("Port:"))
buttons.addWidget(self.dest_port_input)
buttons.addWidget(QLabel("Use TLS:"))
buttons.addWidget(self.dest_usetls_input)
buttons.addStretch()
self.reqview = ReqViewWidget(tag_tab=True)
self.reqview.set_read_only(False)
self.reqview.set_tags_read_only(False)
self.layout().addLayout(buttons)
self.layout().addWidget(self.reqview)
self.req = None
self.dest_host = ""
self.dest_port = 80
self.use_tls = False
self._update_buttons()
def _set_host(self, host):
self.dest_host_input.setText(host)
def _set_port(self, port):
if port is None or port <= 0:
self.dest_port_input.setText("")
else:
self.dest_port_input.setText(str(port))
def _set_usetls(self, usetls):
if usetls:
self.dest_usetls_input.setCheckState(2)
else:
self.dest_usetls_input.setCheckState(0)
def _set_dest_info(self, host, port, usetls):
self._set_host(host)
self._set_port(port)
self._set_usetls(usetls)
def _get_dest_info(self):
host = self.dest_host_input.text()
try:
port = int(self.dest_port_input.text())
except:
port = -1
if self.dest_usetls_input.checkState() == 0:
usetls = False
else:
usetls = True
return (host, port, usetls)
def set_request(self, req, update_history=True):
self._set_dest_info("", -1, False)
if update_history:
self.history.append(req)
self.history_pos = len(self.history)-1
self._update_buttons()
if req:
self.req = req
self.req.tags = set(["repeater"])
self._set_dest_info(req.dest_host, req.dest_port, req.use_tls)
self.reqview.set_request(self.req)
@pyqtSlot(set)
def update_req_tags(self, tags):
if self.req:
self.req.tags = tags
@pyqtSlot()
def submit(self):
try:
req = self.reqview.get_request()
if not req:
display_error_box("Could not parse request")
return
except:
display_error_box("Could not parse request")
return
req.tags.add("repeater")
host, port, usetls = self._get_dest_info()
if port is None:
display_error_box("Invalid port")
return
req.dest_host = host
req.dest_port = port
req.dest_usetls = usetls
try:
self.client.submit(req, save=True)
self.req = req
self.set_request(req)
except Exception as e:
errmsg = "Error submitting request:\n%s" % str(e)
display_error_box(errmsg)
return
@pyqtSlot()
def back(self):
if self.history_pos > 0:
self.history_pos -= 1
self.set_request(self.history[self.history_pos], update_history=False)
self._update_buttons()
@pyqtSlot()
def forward(self):
if self.history_pos < len(self.history)-1:
self.history_pos += 1
self.set_request(self.history[self.history_pos], update_history=False)
self._update_buttons()
def _update_buttons(self):
self.forward_button.setEnabled(True)
self.back_button.setEnabled(True)
if len(self.history) == 0 or self.history_pos == len(self.history)-1:
self.forward_button.setEnabled(False)
if self.history_pos == 0:
self.back_button.setEnabled(False)
================================================
FILE: guppyproxy/reqlist.py
================================================
import threading
import shlex
from guppyproxy.util import max_len_str, query_to_str, display_error_box, display_info_box, display_req_context, display_multi_req_context, hostport, method_color, sc_color, DisableUpdates, host_color
from guppyproxy.proxy import HTTPRequest, RequestContext, InvalidQuery, SocketClosed, time_to_nsecs, ProxyThread
from guppyproxy.reqview import ReqViewWidget
from guppyproxy.reqtree import ReqTreeView
from PyQt5.QtWidgets import QWidget, QTableWidget, QTableWidgetItem, QGridLayout, QHeaderView, QAbstractItemView, QVBoxLayout, QHBoxLayout, QComboBox, QTabWidget, QPushButton, QLineEdit, QStackedLayout, QToolButton, QCheckBox, QLabel, QTableView, QMenu
from PyQt5.QtCore import pyqtSlot, pyqtSignal, QObject, QVariant, Qt, QAbstractTableModel, QModelIndex, QItemSelection, QSortFilterProxyModel
from itertools import groupby, count
def get_field_entry():
dropdown = QComboBox()
dropdown.addItem("Anywhere", "all")
dropdown.addItem("Req. Body", "reqbody")
dropdown.addItem("Rsp. Body", "rspbody")
dropdown.addItem("Any Body", "body")
# dropdown.addItem("WSMessage", "wsmessage")
dropdown.addItem("Req. Header", "reqheader")
dropdown.addItem("Rsp. Header", "rspheader")
dropdown.addItem("Any Header", "header")
dropdown.addItem("Method", "method")
dropdown.addItem("Host", "host")
dropdown.addItem("Path", "path")
dropdown.addItem("URL", "url")
dropdown.addItem("Status", "statuscode")
dropdown.addItem("Tag", "tag")
dropdown.addItem("Any Param", "param")
dropdown.addItem("URL Param", "urlparam")
dropdown.addItem("Post Param", "postparam")
dropdown.addItem("Rsp. Cookie", "rspcookie")
dropdown.addItem("Req. Cookie", "reqcookie")
dropdown.addItem("Any Cookie", "cookie")
# dropdown.addItem("After", "")
# dropdown.addItem("Before", "")
# dropdown.addItem("TimeRange", "")
# dropdown.addItem("Id", "")
return dropdown
def get_string_cmp_entry():
dropdown = QComboBox()
dropdown.addItem("cnt.", "contains")
dropdown.addItem("cnt. (rgx)", "containsregexp")
dropdown.addItem("is", "is")
dropdown.addItem("len. >", "lengt")
dropdown.addItem("len. <", "lenlt")
dropdown.addItem("len. =", "leneq")
return dropdown
class StringCmpWidget(QWidget):
returnPressed = pyqtSignal()
def __init__(self, *args, **kwargs):
QWidget.__init__(self, *args, **kwargs)
layout = QHBoxLayout()
self.cmp_entry = get_string_cmp_entry()
self.text_entry = QLineEdit()
self.text_entry.returnPressed.connect(self.returnPressed)
layout.addWidget(self.cmp_entry)
layout.addWidget(self.text_entry)
self.setLayout(layout)
self.layout().setContentsMargins(0, 0, 0, 0)
def get_value(self):
str_cmp = self.cmp_entry.itemData(self.cmp_entry.currentIndex())
str_val = self.text_entry.text()
return [str_cmp, str_val]
def reset(self):
self.cmp_entry.setCurrentIndex(0)
self.text_entry.setText("")
def dt_sort_key(r):
if r.time_start:
return time_to_nsecs(r.time_start)
return 0
class StringKVWidget(QWidget):
returnPressed = pyqtSignal()
def __init__(self, *args, **kwargs):
QWidget.__init__(self, *args, **kwargs)
self.str2_shown = False
self.str1 = StringCmpWidget()
self.str2 = StringCmpWidget()
self.str1.returnPressed.connect(self.returnPressed)
self.str2.returnPressed.connect(self.returnPressed)
self.toggle_button = QToolButton()
self.toggle_button.setText("+")
self.toggle_button.clicked.connect(self._show_hide_str2)
layout = QHBoxLayout()
layout.addWidget(self.str1)
layout.addWidget(self.str2)
layout.addWidget(self.toggle_button)
self.str2.setVisible(self.str2_shown)
self.setLayout(layout)
self.layout().setContentsMargins(0, 0, 0, 0)
@pyqtSlot()
def _show_hide_str2(self):
if self.str2_shown:
self.toggle_button.setText("+")
self.str2_shown = False
else:
self.toggle_button.setText("-")
self.str2_shown = True
self.str2.setVisible(self.str2_shown)
def get_value(self):
retval = self.str1.get_value()
if self.str2_shown:
retval += self.str2.get_value()
return retval
def reset(self):
self.str1.reset()
self.str2.reset()
class DropdownFilterEntry(QWidget):
# a widget that lets you enter filters using ezpz dropdowns/text boxes
filterEntered = pyqtSignal(list)
def __init__(self, *args, **kwargs):
QWidget.__init__(self, *args, **kwargs)
layout = QHBoxLayout()
confirm = QToolButton()
confirm.setText("OK")
confirm.setToolTip("Apply the entered filter")
self.field_entry = get_field_entry()
# stack containing widgets for string, k/v, date, daterange
self.str_cmp_entry = StringCmpWidget()
self.kv_cmp_entry = StringKVWidget()
self.inv_entry = QCheckBox("inv")
# date
# daterange
self.entry_layout = QStackedLayout()
self.entry_layout.setContentsMargins(0, 0, 0, 0)
self.current_entry = 0
self.entry_layout.addWidget(self.str_cmp_entry)
self.entry_layout.addWidget(self.kv_cmp_entry)
# add date # 2
# add daterange # 3
confirm.clicked.connect(self.confirm_entry)
self.str_cmp_entry.returnPressed.connect(self.confirm_entry)
self.kv_cmp_entry.returnPressed.connect(self.confirm_entry)
self.field_entry.currentIndexChanged.connect(self._display_value_widget)
layout.addWidget(confirm)
layout.addWidget(self.inv_entry)
layout.addWidget(self.field_entry)
layout.addLayout(self.entry_layout)
self.setLayout(layout)
self.setContentsMargins(0, 0, 0, 0)
self._display_value_widget()
@pyqtSlot()
def _display_value_widget(self):
# show the correct value widget in the value stack layout
field = self.field_entry.itemData(self.field_entry.currentIndex())
self.current_entry = 0
if field in ("all", "reqbody", "rspbody", "body", "wsmessage", "method",
"host", "path", "url", "statuscode", "tag"):
self.current_entry = 0
elif field in ("reqheader", "rspheader", "header", "param", "urlparam"
"postparam", "rspcookie", "reqcookie", "cookie"):
self.current_entry = 1
# elif for date
# elif for daterange
self.entry_layout.setCurrentIndex(self.current_entry)
def get_value(self):
val = []
if self.inv_entry.isChecked():
val.append("inv")
field = self.field_entry.itemData(self.field_entry.currentIndex())
val.append(field)
if self.current_entry == 0:
val += self.str_cmp_entry.get_value()
elif self.current_entry == 1:
val += self.kv_cmp_entry.get_value()
# elif for date
# elif for daterange
return [val] # no support for OR
@pyqtSlot()
def confirm_entry(self):
phrases = self.get_value()
self.filterEntered.emit(phrases)
self.str_cmp_entry.reset()
self.kv_cmp_entry.reset()
# reset date
# reset date range
class TextFilterEntry(QWidget):
# a text box that can be used to enter filters
filterEntered = pyqtSignal(list)
def __init__(self, *args, **kwargs):
QWidget.__init__(self, *args, **kwargs)
layout = QHBoxLayout()
self.textEntry = QLineEdit()
self.textEntry.returnPressed.connect(self.confirm_entry)
self.textEntry.setToolTip("Enter the filter here and press return to apply it")
layout.addWidget(self.textEntry)
self.setLayout(layout)
self.layout().setContentsMargins(0, 0, 0, 0)
@pyqtSlot()
def confirm_entry(self):
args = shlex.split(self.textEntry.text())
phrases = [list(group) for k, group in groupby(args, lambda x: x == "OR") if not k]
self.filterEntered.emit(phrases)
self.textEntry.setText("")
class FilterEntry(QWidget):
# a widget that lets you switch between filter entries
filterEntered = pyqtSignal(list)
def __init__(self, *args, **kwargs):
QWidget.__init__(self, *args, **kwargs)
self.current_entry = 0
self.max_entries = 2
self.text_entry = TextFilterEntry()
dropdown_entry = DropdownFilterEntry()
self.text_entry.filterEntered.connect(self.filterEntered)
dropdown_entry.filterEntered.connect(self.filterEntered)
self.entry_layout = QStackedLayout()
self.entry_layout.addWidget(dropdown_entry)
self.entry_layout.addWidget(self.text_entry)
swap_button = QToolButton()
swap_button.setText(">")
swap_button.setToolTip("Switch between dropdown and text entry")
swap_button.clicked.connect(self.next_entry)
hlayout = QHBoxLayout()
hlayout.addWidget(swap_button)
hlayout.addLayout(self.entry_layout)
self.setLayout(hlayout)
self.layout().setContentsMargins(0, 0, 0, 0)
self.layout().setSpacing(0)
@pyqtSlot()
def next_entry(self):
self.current_entry += 1
self.current_entry = self.current_entry % self.max_entries
self.entry_layout.setCurrentIndex(self.current_entry)
def set_entry(self, entry):
self.current_entry = entry
self.current_entry = self.current_entry % self.max_entries
self.entry_layout.setCurrentIndex(self.current_entry)
class FilterListWidget(QTableWidget):
# list part of the filter tab
def __init__(self, *args, **kwargs):
self.client = kwargs.pop("client")
QTableWidget.__init__(self, *args, **kwargs)
self.context = RequestContext(self.client)
# Set up table
self.setColumnCount(1)
self.horizontalHeader().hide()
self.horizontalHeader().setSectionResizeMode(QHeaderView.Stretch)
self.verticalHeader().hide()
self.verticalHeader().setSectionResizeMode(QHeaderView.ResizeToContents)
#self.setSelectionMode(QAbstractItemView.NoSelection)
#self.setEditTriggers(QAbstractItemView.NoEditTriggers)
def append_fstr(self, fstr):
args = shlex.split(fstr)
phrase = [list(group) for k, group in groupby(args, lambda x: x == "OR") if not k]
self.context.apply_phrase(phrase)
self._append_fstr_row(fstr)
def set_query(self, query):
self.context.set_query(query)
self.redraw_table()
def pop_phrase(self):
self.context.pop_phrase()
self.redraw_table()
def clear_phrases(self):
self.context.set_query([])
self.redraw_table()
def _append_fstr_row(self, fstr):
row = self.rowCount()
self.insertRow(row)
self.setItem(row, 0, QTableWidgetItem(fstr))
def redraw_table(self):
self.setRowCount(0)
query = self.context.query
for p in query:
condstrs = [' '.join(l) for l in p]
fstr = ' OR '.join(condstrs)
self._append_fstr_row(fstr)
def get_query(self):
return self.context.query
class FilterEditor(QWidget):
# a widget containing a list of filters and the ability to edit the filters in the list
filtersEdited = pyqtSignal(list)
builtin_filters = (
('No Images', ['inv', 'path', 'containsregexp', r'(\.png$|\.jpg$|\.jpeg$|\.gif$|\.ico$|\.bmp$|\.svg$)']),
('No JavaScript/CSS/Fonts', ['inv', 'path', 'containsregexp', r'(\.js$|\.css$|\.woff$)']),
)
def __init__(self, *args, **kwargs):
self.client = kwargs.pop("client")
QWidget.__init__(self, *args, **kwargs)
layout = QVBoxLayout()
# Manage bar
manage_bar = QHBoxLayout()
pop_button = QPushButton("Pop")
pop_button.setToolTip("Remove the most recently applied filter")
clear_button = QPushButton("Clear")
clear_button.setToolTip("Remove all active filters")
scope_reset_button = QPushButton("Scope")
scope_reset_button.setToolTip("Set the active filters to the current scope")
scope_save_button = QPushButton("Save Scope")
scope_save_button.setToolTip("Set the scope to the current filters. Any messages that don't match the active filters will be ignored by the proxy.")
self.builtin_combo = QComboBox()
self.builtin_combo.addItem("Apply a built-in filter", None)
for desc, filt in FilterEditor.builtin_filters:
self.builtin_combo.addItem(desc, filt)
self.builtin_combo.currentIndexChanged.connect(self._apply_builtin_filter)
manage_bar.addWidget(clear_button)
manage_bar.addWidget(pop_button)
manage_bar.addWidget(scope_reset_button)
manage_bar.addWidget(scope_save_button)
manage_bar.addWidget(self.builtin_combo)
manage_bar.addStretch()
mbar_widget = QWidget()
mbar_widget.setLayout(manage_bar)
pop_button.clicked.connect(self.pop_phrase)
clear_button.clicked.connect(self.clear_phrases)
scope_reset_button.clicked.connect(self.reset_to_scope)
scope_save_button.clicked.connect(self.save_scope)
# Filter list
self.filter_list = FilterListWidget(client=self.client)
# Filter entry
self.entry = FilterEntry()
self.entry.setMaximumHeight(self.entry.sizeHint().height())
self.entry.filterEntered.connect(self.apply_phrase)
layout.addWidget(mbar_widget)
layout.addWidget(self.filter_list)
layout.addWidget(self.entry)
self.setLayout(layout)
self.layout().setSpacing(0)
self.layout().setContentsMargins(0, 0, 0, 0)
@pyqtSlot()
def save_scope(self):
query = self.filter_list.get_query()
self.client.set_scope(query)
display_info_box("Scope updated")
@pyqtSlot()
def reset_to_scope(self):
query = self.client.get_scope().filter
self.filter_list.set_query(query)
self.filtersEdited.emit(self.filter_list.get_query())
@pyqtSlot()
def clear_phrases(self):
self.filter_list.clear_phrases()
self.filtersEdited.emit(self.filter_list.get_query())
@pyqtSlot()
def pop_phrase(self):
self.filter_list.pop_phrase()
self.filtersEdited.emit(self.filter_list.get_query())
@pyqtSlot(list)
def apply_phrase(self, phrase):
fstr = query_to_str([phrase])
try:
self.filter_list.append_fstr(fstr)
except InvalidQuery as e:
display_error_box("Could not add filter:\n\n%s" % e)
return
self.filtersEdited.emit(self.filter_list.get_query())
@pyqtSlot(int)
def _apply_builtin_filter(self, ind):
phrase = self.builtin_combo.itemData(ind)
if phrase:
self.apply_phrase([phrase])
self.builtin_combo.setCurrentIndex(0)
def set_is_text(self, is_text):
if is_text:
self.entry.set_entry(1)
else:
self.entry.set_entry(0)
class ReqListModel(QAbstractTableModel):
requestsLoading = pyqtSignal()
requestsLoaded = pyqtSignal()
HD_ID = 0
HD_VERB = 1
HD_HOST = 2
HD_PATH = 3
HD_SCODE = 4
HD_REQLEN = 5
HD_RSPLEN = 6
HD_TIME = 7
HD_TAGS = 8
HD_MNGL = 9
def __init__(self, client, *args, **kwargs):
QAbstractTableModel.__init__(self, *args, **kwargs)
self.client = client
self.header_order = [
self.HD_ID,
self.HD_VERB,
self.HD_HOST,
self.HD_PATH,
self.HD_SCODE,
self.HD_REQLEN,
self.HD_RSPLEN,
self.HD_TIME,
self.HD_TAGS,
self.HD_MNGL,
]
self.table_headers = {
self.HD_ID: "ID",
self.HD_VERB: "Method",
self.HD_HOST: "Host",
self.HD_PATH: "Path",
self.HD_SCODE: "S-Code",
self.HD_REQLEN: "Req Len",
self.HD_RSPLEN: "Rsp Len",
self.HD_TIME: "Time",
self.HD_TAGS: "Tags",
self.HD_MNGL: "Mngl",
}
self.reqs = []
self.sort_enabled = False
self.header_count = len(self.header_order)
self.reqs_loaded = 0
def headerData(self, section, orientation, role):
if role == Qt.DisplayRole and orientation == Qt.Horizontal:
hd = self.header_order[section]
return self.table_headers[hd]
return QVariant()
def rowCount(self, parent):
return self.reqs_loaded
def columnCount(self, parent):
return self.header_count
def _gen_req_row(self, req):
MAX_PATH_LEN = 60
MAX_TAG_LEN = 40
reqid = self.client.get_reqid(req)
method = req.method
host = hostport(req)
path = max_len_str(req.url.path, MAX_PATH_LEN)
reqlen = str(req.content_length)
tags = max_len_str(', '.join(sorted(req.tags)), MAX_TAG_LEN)
if req.response:
scode = str(req.response.status_code) + ' ' + req.response.reason
rsplen = str(req.response.content_length)
else:
scode = "--"
rsplen = "--"
if req.time_start and req.time_end:
time_delt = req.time_end - req.time_start
reqtime = ("%.2f" % time_delt.total_seconds())
else:
reqtime = "--"
if req.unmangled and req.response and req.response.unmangled:
manglestr = "q/s"
elif req.unmangled:
manglestr = "q"
elif req.response and req.response.unmangled:
manglestr = "s"
else:
manglestr = "N/A"
return (req, reqid, method, host, path, scode, reqlen, rsplen, reqtime, tags, manglestr)
def data(self, index, role):
if role == Qt.BackgroundColorRole:
req = self.reqs[index.row()][0]
if index.column() == 2:
return host_color(hostport(req))
elif index.column() == 4:
if req.response:
return sc_color(str(req.response.status_code))
elif index.column() == 1:
return method_color(req.method)
return QVariant()
elif role == Qt.DisplayRole:
rowdata = self.reqs[index.row()]
return rowdata[index.column()+1]
return QVariant()
def canFetchMore(self, parent):
if parent.isValid():
return False
return (self.reqs_loaded < len(self.reqs))
def fetchMore(self, parent):
if parent.isValid():
return
if self.reqs_loaded == len(self.reqs):
return
n_to_fetch = 50
if self.reqs_loaded + n_to_fetch > len(self.reqs):
n_to_fetch = len(self.reqs) - self.reqs_loaded
self.beginInsertRows(QModelIndex(), self.reqs_loaded, self.reqs_loaded + n_to_fetch)
self.reqs_loaded += n_to_fetch
self.endInsertRows()
def _sort_reqs(self):
def skey(rowdata):
return dt_sort_key(rowdata[0])
if self.sort_enabled:
self.reqs = sorted(self.reqs, key=skey, reverse=True)
def _req_ind(self, req=None, reqid=None):
if not reqid:
reqid = self.client.get_reqid(req)
for ind, rowdata in zip(count(), self.reqs):
req = rowdata[0]
if self.client.get_reqid(req) == reqid:
return ind
return -1
def _emit_all_data(self):
self.dataChanged.emit(self.createIndex(0, 0), self.createIndex(self.rowCount(None), self.columnCount(None)))
def _set_requests(self, reqs):
self.reqs = [self._gen_req_row(req) for req in reqs]
self.reqs_loaded = 0
def set_requests(self, reqs):
self.beginResetModel()
self._set_requests(reqs)
self._sort_reqs()
self._emit_all_data()
self.endResetModel()
def clear(self):
self.beginResetModel()
self.reqs = []
self.reqs_loaded = 0
self._emit_all_data()
self.endResetModel()
def add_request_head(self, req):
self.beginInsertRows(QModelIndex(), 0, 0)
self.reqs = [self._gen_req_row(req)] + self.reqs
self.reqs_loaded += 1
self.endInsertRows()
def add_request(self, req):
self.beginResetModel()
self.reqs.append(self._gen_req_row(req))
self.reqs_loaded = 0
self._sort_reqs()
self._emit_all_data()
self.endResetModel()
def add_requests(self, reqs):
self.beginResetModel()
for req in reqs:
self.reqs.append(self._gen_req_row(req))
self.reqs_loaded = 0
self._sort_reqs()
self._emit_all_data()
self.endResetModel()
def update_request(self, req):
ind = self._req_ind(req)
if ind < 0:
return
self.reqs[ind] = self._gen_req_row(req)
self.dataChanged.emit(self.createIndex(ind, 0), self.createIndex(ind, self.rowCount(None)))
def delete_request(self, req=None, reqid=None):
ind = self._req_ind(req, reqid)
if ind < 0:
return
self.beginRemoveRows(QModelIndex(), ind, ind)
self.reqs_loaded -= 1
self.reqs = self.reqs[:ind] + self.reqs[(ind+1):]
self.endRemoveRows()
def has_request(self, req=None, reqid=None):
if self._req_ind(req, reqid) < 0:
return False
return True
def get_requests(self):
return [row[0] for row in self.reqs]
def disable_sort(self):
self.sort_enabled = False
def enable_sort(self):
self.sort_enabled = True
self._sort_reqs()
def req_by_ind(self, ind):
return self.reqs[ind][0]
class ReqBrowser(QWidget):
# Widget containing request viewer, tabs to view list of reqs, filters, and (evevntually) site map
# automatically updated with requests as they're saved
def __init__(self, client, repeater_widget=None, macro_widget=None, reload_reqs=True, update=False, filter_tab=True, is_client_context=False):
QWidget.__init__(self)
self.client = client
self.filters = []
self.reload_reqs = reload_reqs
self.mylayout = QGridLayout()
self.mylayout.setSpacing(0)
self.mylayout.setContentsMargins(0, 0, 0, 0)
# reqtable updater
if update:
self.updater = ReqListUpdater(self.client)
else:
self.updater = None
# reqtable/search
self.listWidg = ReqTableWidget(client, repeater_widget=repeater_widget, macro_widget=macro_widget)
if self.updater:
self.updater.add_reqlist_widget(self.listWidg)
self.listWidg.requestsSelected.connect(self.update_viewer)
self.listLayout = QVBoxLayout()
self.listLayout.setContentsMargins(0, 0, 0, 0)
self.listLayout.setSpacing(0)
self.listButtonLayout = QHBoxLayout()
self.listButtonLayout.setContentsMargins(0, 0, 0, 0)
clearSelectionBut = QPushButton("Clear Selection")
clearSelectionBut.clicked.connect(self.listWidg.clear_selection)
self.listButtonLayout.addWidget(clearSelectionBut)
self.listButtonLayout.addStretch()
self.listLayout.addWidget(self.listWidg)
self.listLayout.addLayout(self.listButtonLayout)
# Filter widget
self.filterWidg = FilterEditor(client=self.client)
self.filterWidg.filtersEdited.connect(self.listWidg.set_filter)
if is_client_context:
self.filterWidg.filtersEdited.connect(self.set_client_context)
self.filterWidg.reset_to_scope()
# Tree widget
self.treeWidg = ReqTreeView()
# add tabs
self.listTabs = QTabWidget()
lwidg = QWidget()
lwidg.setLayout(self.listLayout)
self.listTabs.addTab(lwidg, "List")
self.tree_ind = self.listTabs.count()
self.listTabs.addTab(self.treeWidg, "Tree")
if filter_tab:
self.listTabs.addTab(self.filterWidg, "Filters")
self.listTabs.currentChanged.connect(self._tab_changed)
# reqview
self.reqview = ReqViewWidget(info_tab=True, param_tab=True, tag_tab=True)
self.reqview.set_tags_read_only(False)
self.reqview.tag_widg.tagsUpdated.connect(self._tags_updated)
self.listWidg.req_view_widget = self.reqview
self.mylayout.addWidget(self.reqview, 0, 0, 3, 1)
self.mylayout.addWidget(self.listTabs, 4, 0, 2, 1)
self.setLayout(self.mylayout)
def show_filters(self):
self.listTabs.setCurrentIndex(2)
def show_history(self):
self.listTabs.setCurrentIndex(0)
def show_tree(self):
self.listTabs.setCurrentIndex(1)
@pyqtSlot(list)
def set_client_context(self, query):
self.client.context.set_query(query)
@pyqtSlot()
def reset_to_scope(self):
self.filterWidg.reset_to_scope()
@pyqtSlot(list)
def update_viewer(self, reqs):
self.reqview.set_request(None)
if len(reqs) > 0:
if self.reload_reqs:
reqh = reqs[0]
req = self.client.req_by_id(reqh.db_id)
else:
req = reqs[0]
self.reqview.set_request(req)
@pyqtSlot(list)
def update_filters(self, query):
self.filters = query
@pyqtSlot(HTTPRequest)
def add_request_item(self, req):
self.listWidg.add_request_item(req)
self.treeWidg.add_request_item(req)
@pyqtSlot(list)
def set_requests(self, reqs):
self.listWidg.set_requests(reqs)
self.treeWidg.set_requests(reqs)
@pyqtSlot(int)
def _tab_changed(self, i):
if i == self.tree_ind:
self.treeWidg.set_requests(self.listWidg.get_requests())
@pyqtSlot(set)
def _tags_updated(self, tags):
req = self.reqview.req
req.tags = tags
if req.db_id:
reqid = self.client.get_reqid(req)
self.client.clear_tag(reqid)
for tag in tags:
self.client.add_tag(reqid, tag)
def set_filter_is_text(self, is_text):
self.filterWidg.set_is_text(is_text)
class ReqListUpdater(QObject):
newRequest = pyqtSignal(HTTPRequest)
requestUpdated = pyqtSignal(HTTPRequest)
requestDeleted = pyqtSignal(str)
def __init__(self, client):
QObject.__init__(self)
self.mtx = threading.Lock()
self.client = client
self.reqlist_widgets = []
self.t = ProxyThread(target=self.run_updater)
self.t.start()
def add_reqlist_widget(self, widget):
self.mtx.acquire()
try:
self.newRequest.connect(widget.add_request)
self.requestUpdated.connect(widget.update_request)
self.requestDeleted.connect(widget.delete_request)
self.reqlist_widgets.append(widget)
finally:
self.mtx.release()
def run_updater(self):
conn = self.client.new_conn()
try:
try:
for msg in conn.watch_storage():
self.mtx.acquire()
try:
if msg["Action"] == "NewRequest":
self.newRequest.emit(msg["Request"])
elif msg["Action"] == "RequestUpdated":
self.requestUpdated.emit(msg["Request"])
elif msg["Action"] == "RequestDeleted":
self.requestDeleted.emit(msg["MessageId"])
finally:
self.mtx.release()
except SocketClosed:
return
finally:
conn.close()
def stop(self):
self.conn.close()
class ReqTableWidget(QWidget):
requestsChanged = pyqtSignal(list)
requestsSelected = pyqtSignal(list)
def __init__(self, client, repeater_widget=None, macro_widget=None, *args, **kwargs):
QWidget.__init__(self, *args, **kwargs)
self.allow_save = False
self.client = client
self.repeater_widget = repeater_widget
self.macro_widget = macro_widget
self.query = []
self.req_view_widget = None
self.setLayout(QStackedLayout())
self.layout().setContentsMargins(0, 0, 0, 0)
self.tableModel = ReqListModel(self.client)
self.tableView = QTableView()
self.tableView.setModel(self.tableModel)
self.tableView.verticalHeader().setSectionResizeMode(QHeaderView.ResizeToContents)
self.tableView.horizontalHeader().setSectionResizeMode(QHeaderView.ResizeToContents)
self.tableView.verticalHeader().hide()
self.tableView.setSelectionBehavior(QAbstractItemView.SelectRows)
#self.tableView.setSelectionMode(QAbstractItemView.SingleSelection)
self.tableView.horizontalHeader().setStretchLastSection(True)
self.tableView.selectionModel().selectionChanged.connect(self.on_select_change)
self.tableModel.dataChanged.connect(self._paint_view)
self.tableModel.rowsInserted.connect(self._on_rows_inserted)
self.requestsChanged.connect(self.set_requests)
self.requestsSelected.connect(self._updated_selected_request)
self.selected_reqs = []
self.layout().addWidget(self.tableView)
self.layout().addWidget(QLabel("<b>Loading requests from data file...</b>"))
@pyqtSlot(HTTPRequest)
def add_request(self, req):
with DisableUpdates(self.tableView):
if req.db_id != "":
reqid = self.client.get_reqid(req)
if self.client.check_request(self.query, reqid=reqid):
self.tableModel.add_request_head(req)
if req.unmangled and req.unmangled.db_id != "" and self.tableModel.has_request(req.unmangled):
self.tableModel.delete_request(req.unmangled)
else:
if self.client.check_request(self.query, req=req):
self.tableModel.add_request_head(req)
@pyqtSlot()
def clear(self):
self.tableModel.clear()
def get_requests(self):
return self.tableModel.get_requests()
@pyqtSlot(list)
def set_requests(self, reqs, check_filter=False):
to_add = []
if not check_filter:
to_add = reqs
else:
for req in reqs:
if req.db_id != "":
reqid = self.client.get_reqid(req)
if self.client.check_request(self.query, reqid=reqid):
to_add.append(req)
else:
if self.client.check_request(self.query, req=req):
to_add.append(req)
with DisableUpdates(self.tableView):
self.clear()
self.tableModel.disable_sort()
self.tableModel.add_requests(to_add)
self.tableModel.enable_sort()
self.set_is_not_loading()
@pyqtSlot(HTTPRequest)
def update_request(self, req):
with DisableUpdates(self.tableView):
self.tableModel.update_request(req)
if req.db_id != "":
if req.unmangled and req.unmangled.db_id != "":
self.tableModel.delete_request(reqid=self.client.get_reqid(req.unmangled))
@pyqtSlot(str)
def delete_request(self, reqid):
with DisableUpdates(self.tableView):
self.tableModel.delete_request(reqid=reqid)
@pyqtSlot(list)
def set_filter(self, query):
self.query = query
self.set_is_loading()
self.client.query_storage_async(self.requestsChanged, self.query, headers_only=True)
@pyqtSlot(list)
def _updated_selected_request(self, reqs):
if len(reqs) > 0:
self.selected_reqs = reqs
else:
self.selected_reqs = []
@pyqtSlot(QModelIndex, int, int)
def _on_rows_inserted(self, parent, first, last):
rows = self.tableView.selectionModel().selectedRows()
if len(rows) > 0:
row = rows[0].row()
idx = self.tableModel.index(row, 0, QModelIndex())
self.tableView.scrollTo(idx)
@pyqtSlot(QItemSelection, QItemSelection)
def on_select_change(self, newSelection, oldSelection):
reqs = []
added = set()
for rowidx in self.tableView.selectionModel().selectedRows():
row = rowidx.row()
if row not in added:
reqs.append(self.tableModel.req_by_ind(row))
added.add(row)
self.requestsSelected.emit(reqs)
@pyqtSlot()
def clear_selection(self):
self.tableView.clearSelection()
def get_selected_request(self):
# load the full request
if len(self.selected_reqs) > 0:
return self.client.load_by_reqheaders(self.selected_reqs[0])
else:
return None
def get_selected_requests(self):
ret = []
for hreq in self.selected_reqs:
ret.append(self.client.load_by_reqheaders(hreq))
return ret
def get_all_requests(self):
return [self.client.req_by_id(self.client.get_reqid(req)) for req in self.tableModel.get_requests()]
def contextMenuEvent(self, event):
if len(self.selected_reqs) > 1:
reqs = self.get_selected_requests()
display_multi_req_context(self, self.client, reqs, event,
macro_widget=self.macro_widget,
save_option=self.allow_save)
elif len(self.selected_reqs) == 1:
req = self.get_selected_request()
display_req_context(self, self.client, req, event,
repeater_widget=self.repeater_widget,
req_view_widget=self.req_view_widget,
macro_widget=self.macro_widget,
save_option=self.allow_save)
def set_is_loading(self):
self.set_loading(True)
def set_is_not_loading(self):
self.set_loading(False)
def set_loading(self, is_loading):
with DisableUpdates(self.tableView):
if is_loading:
self.layout().setCurrentIndex(1)
else:
self.layout().setCurrentIndex(0)
@pyqtSlot(QModelIndex, QModelIndex)
def _paint_view(self, indA, indB):
self.tableView.repaint()
@pyqtSlot()
def delete_selected(self):
with DisableUpdates(self.tableView):
for req in self.selected_reqs:
self.tableModel.delete_request(req=req)
================================================
FILE: guppyproxy/reqtree.py
================================================
from guppyproxy.proxy import HTTPRequest
from PyQt5.QtWidgets import QWidget, QTreeView, QVBoxLayout
from PyQt5.QtGui import QStandardItem, QStandardItemModel
from PyQt5.QtCore import pyqtSlot, Qt
def _include_req(req):
if not req.response:
return False
if req.response.status_code == 404:
return False
return True
class PathNodeItem(QStandardItem):
def __init__(self, text, *args, **kwargs):
QStandardItem.__init__(self, *args, **kwargs)
self.text = text
self.children = {}
def add_child(self, text):
if text not in self.children:
newitem = PathNodeItem(text, text)
newitem.setFlags(newitem.flags() ^ Qt.ItemIsEditable)
self.children[text] = newitem
self.appendRow(newitem)
def get_child(self, text):
return self.children[text]
def add_child_path(self, texts):
if not texts:
return
childtext = texts[0]
self.add_child(childtext)
child = self.get_child(childtext)
child.add_child_path(texts[1:])
class ReqTreeView(QWidget):
def __init__(self):
QWidget.__init__(self)
self.setLayout(QVBoxLayout())
self.layout().setSpacing(0)
self.layout().setContentsMargins(0, 0, 0, 0)
self.nodes = {}
self.tree_view = QTreeView()
self.tree_view.header().close()
self.root = QStandardItemModel()
self.tree_view.setModel(self.root)
self.layout().addWidget(self.tree_view)
@pyqtSlot(HTTPRequest)
def add_request_item(self, req):
path_parts = req.url.geturl(False).split("/")
path_parts = path_parts[1:]
path_parts = ["/" + p for p in path_parts]
path_parts = [req.dest_host] + path_parts
if path_parts[0] not in self.nodes:
item = PathNodeItem(path_parts[0], path_parts[0])
item.setFlags(item.flags() ^ Qt.ItemIsEditable)
self.nodes[path_parts[0]] = item
self.root.appendRow(item)
else:
item = self.nodes[path_parts[0]]
item.add_child_path(path_parts[1:])
@pyqtSlot(list)
def set_requests(self, reqs):
self.clear()
for req in reqs:
if _include_req(req):
self.add_request_item(req)
self.tree_view.expandAll()
def clear(self):
self.nodes = {}
self.root = QStandardItemModel()
self.tree_view.setModel(self.root)
================================================
FILE: guppyproxy/reqview.py
================================================
import re
from guppyproxy.util import datetime_string, DisableUpdates
from guppyproxy.proxy import HTTPRequest, get_full_url, parse_request
from guppyproxy.hexteditor import ComboEditor
from PyQt5.QtWidgets import QWidget, QTableWidget, QTableWidgetItem, QGridLayout, QHeaderView, QAbstractItemView, QLineEdit, QTabWidget, QVBoxLayout, QToolButton, QHBoxLayout, QStackedLayout
from PyQt5.QtCore import pyqtSlot, pyqtSignal, Qt
from pygments.lexer import Lexer
from pygments.lexers import get_lexer_for_mimetype, TextLexer
from pygments.lexers.textfmts import HttpLexer
from pygments.util import ClassNotFound
from pygments.token import Token
class HybridHttpLexer(Lexer):
tl = TextLexer()
hl = HttpLexer()
def __init__(self, max_len=50000, *args, **kwargs):
self.max_len = max_len
Lexer.__init__(self, *args, **kwargs)
def get_tokens_unprocessed(self, text):
try:
split = re.split(r"(?:\r\n|\n)(?:\r\n|\n)", text, 1)
if len(split) == 2:
h = split[0]
body = split[1]
else:
h = split[0]
body = ''
except Exception as e:
for v in self.tl.get_tokens_unprocessed(text):
yield v
raise e
for token in self.hl.get_tokens_unprocessed(h):
yield token
if len(body) > 0:
if len(body) <= self.max_len or self.max_len < 0:
second_parser = None
if "Content-Type" in h:
try:
ct = re.search("Content-Type: (.*)", h)
if ct is not None:
hval = ct.groups()[0]
mime = hval.split(";")[0]
second_parser = get_lexer_for_mimetype(mime)
except ClassNotFound:
pass
if second_parser is None:
yield (len(h), Token.Text, text[len(h):])
else:
for index, tokentype, value in second_parser.get_tokens_unprocessed(text[len(h):]):
yield (index + len(h), tokentype, value)
else:
yield (len(h), Token.Text, text[len(h):])
class InfoWidget(QWidget):
def __init__(self, *args, **kwargs):
QWidget.__init__(self, *args, **kwargs)
self.request = None
self.setLayout(QVBoxLayout())
self.layout().setSpacing(0)
self.layout().setContentsMargins(0, 0, 0, 0)
self.infotable = QTableWidget()
self.infotable.setColumnCount(2)
self.infotable.verticalHeader().hide()
self.infotable.horizontalHeader().hide()
self.infotable.horizontalHeader().setSectionResizeMode(QHeaderView.ResizeToContents)
self.infotable.verticalHeader().setSectionResizeMode(QHeaderView.ResizeToContents)
self.infotable.horizontalHeader().setStretchLastSection(True)
gitextract_l9ewhzbc/ ├── .gitignore ├── LICENSE ├── README.md ├── guppyproxy/ │ ├── __init__.py │ ├── config.py │ ├── decoder.py │ ├── gui.py │ ├── gup.py │ ├── hexteditor.py │ ├── interceptor.py │ ├── macros.py │ ├── proxy.py │ ├── repeater.py │ ├── reqlist.py │ ├── reqtree.py │ ├── reqview.py │ ├── settings.py │ ├── shortcuts.py │ └── util.py ├── img/ │ └── shark.icns ├── install.sh ├── puppyrsc/ │ ├── NOTE.md │ ├── puppy.linux32 │ ├── puppy.linux64 │ └── puppy.osx └── setup.py
SYMBOL INDEX (646 symbols across 15 files)
FILE: guppyproxy/config.py
class ProxyConfig (line 12) | class ProxyConfig:
method __init__ (line 15) | def __init__(self):
method loads (line 19) | def loads(self, js):
method dumps (line 23) | def dumps(self):
method load (line 32) | def load(self, fname):
method _set_config (line 42) | def _set_config(self, config_info):
method _parse_listeners (line 50) | def _parse_listeners(self, listeners):
method set_listeners (line 75) | def set_listeners(self, listeners):
method listeners (line 79) | def listeners(self):
method listeners (line 83) | def listeners(self, val):
method proxy (line 87) | def proxy(self):
method proxy (line 92) | def proxy(self, val):
method use_proxy (line 96) | def use_proxy(self):
method proxy_host (line 105) | def proxy_host(self):
method proxy_port (line 113) | def proxy_port(self):
method proxy_username (line 121) | def proxy_username(self):
method proxy_password (line 129) | def proxy_password(self):
method use_proxy_creds (line 137) | def use_proxy_creds(self):
method is_socks_proxy (line 141) | def is_socks_proxy(self):
FILE: guppyproxy/decoder.py
class DecodeError (line 12) | class DecodeError(Exception):
function asciihex_encode_helper (line 15) | def asciihex_encode_helper(s):
function asciihex_decode_helper (line 19) | def asciihex_decode_helper(s):
function base64_decode_helper (line 30) | def base64_decode_helper(s):
function url_decode_helper (line 41) | def url_decode_helper(s):
function url_encode_helper (line 46) | def url_encode_helper(s):
function html_encode_helper (line 51) | def html_encode_helper(s):
function html_decode_helper (line 55) | def html_decode_helper(s):
function pp_json (line 59) | def pp_json(s):
function decode_jwt (line 63) | def decode_jwt(s):
function decode_unixtime (line 77) | def decode_unixtime(s):
class DecoderWidget (line 86) | class DecoderWidget(QWidget):
method __init__ (line 88) | def __init__(self):
class DecoderInput (line 99) | class DecoderInput(QWidget):
method __init__ (line 117) | def __init__(self, *args, **kwargs):
method encode (line 141) | def encode(self):
FILE: guppyproxy/gui.py
class GuppyWindow (line 14) | class GuppyWindow(QWidget):
method __init__ (line 19) | def __init__(self, client):
method show_hist_tab (line 69) | def show_hist_tab(self):
method show_repeater_tab (line 72) | def show_repeater_tab(self):
method show_interceptor_tab (line 75) | def show_interceptor_tab(self):
method show_decoder_tab (line 78) | def show_decoder_tab(self):
method show_active_macro_tab (line 81) | def show_active_macro_tab(self):
method show_int_macro_tab (line 85) | def show_int_macro_tab(self):
method resizeEvent (line 89) | def resizeEvent(self, event):
method _delayedUpdate (line 96) | def _delayedUpdate(self):
method close (line 100) | def close(self):
FILE: guppyproxy/gup.py
function load_certificates (line 13) | def load_certificates(client, path):
function generate_certificates (line 18) | def generate_certificates(client, path):
function main (line 29) | def main():
function start (line 96) | def start():
FILE: guppyproxy/hexteditor.py
class PrettyPrintWidget (line 16) | class PrettyPrintWidget(QWidget):
method __init__ (line 21) | def __init__(self, *args, **kwargs):
method guess_format (line 53) | def guess_format(self):
method _combo_changed (line 66) | def _combo_changed(self):
method set_view (line 72) | def set_view(self, view):
method clear_output (line 93) | def clear_output(self):
method set_bytes (line 97) | def set_bytes(self, bs):
method fill_json (line 107) | def fill_json(self):
method fill_htmlxml (line 120) | def fill_htmlxml(self):
method fill_highlighted (line 138) | def fill_highlighted(self):
class HextEditor (line 154) | class HextEditor(QWidget):
method __init__ (line 162) | def __init__(self, enable_pretty=True):
method focus_in_event (line 188) | def focus_in_event(self, e):
method focus_left_event (line 194) | def focus_left_event(self, e):
method setReadOnly (line 201) | def setReadOnly(self, ro):
method _insert_byte (line 204) | def _insert_byte(self, cursor, b):
method clear (line 216) | def clear(self):
method set_lexer (line 219) | def set_lexer(self, lexer):
method set_bytes (line 222) | def set_bytes(self, bs):
method set_bytes_highlighted (line 244) | def set_bytes_highlighted(self, bs, lexer=None):
method get_bytes (line 259) | def get_bytes(self):
method _get_bytes (line 264) | def _get_bytes(self):
method _split_by_printables (line 292) | def _split_by_printables(cls, bs):
class HexEditor (line 313) | class HexEditor(QWidget):
method __init__ (line 315) | def __init__(self):
method set_bytes (line 330) | def set_bytes(self, bs):
method get_bytes (line 334) | def get_bytes(self):
method setReadOnly (line 337) | def setReadOnly(self, ro):
method _redraw_strcol (line 341) | def _redraw_strcol(self, row):
method redraw_table (line 350) | def redraw_table(self, length=None):
method _format_hex (line 386) | def _format_hex(cls, n):
method _cell_changed (line 390) | def _cell_changed(self, row, col):
class ComboEditor (line 420) | class ComboEditor(QWidget):
method __init__ (line 421) | def __init__(self, pretty_tab=True, enable_pretty=True):
method _tab_changed (line 451) | def _tab_changed(self, i):
method set_bytes (line 475) | def set_bytes(self, bs):
method set_bytes_highlighted (line 487) | def set_bytes_highlighted(self, bs, lexer=None):
method get_bytes (line 495) | def get_bytes(self):
method setReadOnly (line 502) | def setReadOnly(self, ro):
FILE: guppyproxy/interceptor.py
class InterceptEvent (line 12) | class InterceptEvent:
method __init__ (line 14) | def __init__(self):
method wait (line 19) | def wait(self):
method set (line 23) | def set(self, message):
method cancel (line 27) | def cancel(self):
class InterceptedMessage (line 32) | class InterceptedMessage:
method __init__ (line 34) | def __init__(self, request=None, response=None, wsmessage=None):
class InterceptorMacro (line 49) | class InterceptorMacro(InterceptMacro, QObject):
method __init__ (line 57) | def __init__(self, int_widget):
method mangle_request (line 64) | def mangle_request(self, request):
method mangle_response (line 75) | def mangle_response(self, request, response):
method mangle_websocket (line 83) | def mangle_websocket(self, request, response, message):
class InterceptorWidget (line 88) | class InterceptorWidget(QWidget):
method __init__ (line 89) | def __init__(self, client):
method int_req_toggled (line 140) | def int_req_toggled(self, state):
method int_rsp_toggled (line 145) | def int_rsp_toggled(self, state):
method int_ws_toggled (line 150) | def int_ws_toggled(self, state):
method message_received (line 155) | def message_received(self, msg):
method set_edited_message (line 160) | def set_edited_message(self, msg):
method edit_next_message (line 169) | def edit_next_message(self):
method forward_message (line 180) | def forward_message(self):
method cancel_edit (line 203) | def cancel_edit(self):
method clear_edit_queue (line 209) | def clear_edit_queue(self):
method restart_intercept (line 217) | def restart_intercept(self):
method close (line 232) | def close(self):
FILE: guppyproxy/macros.py
class MacroException (line 22) | class MacroException(Exception):
class MacroClient (line 25) | class MacroClient(QObject):
method __init__ (line 34) | def __init__(self, client):
method check_dead (line 38) | def check_dead(self):
method submit (line 45) | def submit(self, req, save=False):
method save (line 52) | def save(self, req):
method output (line 60) | def output(self, s):
method output_req (line 67) | def output_req(self, req):
method new_request (line 74) | def new_request(self, method="GET", path="/", proto_major=1, proto_min...
class FileInterceptMacro (line 85) | class FileInterceptMacro(InterceptMacro, QObject):
method __init__ (line 91) | def __init__(self, parent, client, filename):
method __repr__ (line 105) | def __repr__(self):
method load (line 109) | def load(self, fname):
method prompt_args (line 144) | def prompt_args(self):
method init (line 160) | def init(self, args):
method mangle_request (line 169) | def mangle_request(self, request):
method mangle_response (line 177) | def mangle_response(self, request, response):
method mangle_websocket (line 185) | def mangle_websocket(self, request, response, message):
class FileMacro (line 193) | class FileMacro(QObject):
method __init__ (line 199) | def __init__(self, parent, filename='', resultSlot=None):
method load (line 207) | def load(self):
method execute (line 221) | def execute(self, client, reqs):
class MacroWidget (line 248) | class MacroWidget(QWidget):
method __init__ (line 251) | def __init__(self, client, *args, **kwargs):
method show_active (line 273) | def show_active(self):
method show_int (line 276) | def show_int(self):
method add_requests (line 279) | def add_requests(self, reqs):
class IntMacroListModel (line 283) | class IntMacroListModel(QAbstractTableModel):
method __init__ (line 286) | def __init__(self, parent, client, *args, **kwargs):
method _emit_all_data (line 295) | def _emit_all_data(self):
method headerData (line 298) | def headerData(self, section, orientation, role):
method rowCount (line 303) | def rowCount(self, parent):
method columnCount (line 306) | def columnCount(self, parent):
method data (line 309) | def data(self, index, role):
method flags (line 322) | def flags(self, index):
method setData (line 328) | def setData(self, index, value, role):
method add_macro_exception (line 340) | def add_macro_exception(self, estr):
method add_macro (line 345) | def add_macro(self, macro_path):
method remove_macro (line 354) | def remove_macro(self, ind):
method enable_macro (line 362) | def enable_macro(self, ind):
method disable_macro (line 389) | def disable_macro(self, ind):
class IntMacroWidget (line 401) | class IntMacroWidget(QWidget):
method __init__ (line 404) | def __init__(self, client, *args, **kwargs):
method add_macro (line 441) | def add_macro(self, fname):
method reload_macros (line 444) | def reload_macros(self):
method new_macro (line 448) | def new_macro(self):
method browse_macro (line 458) | def browse_macro(self):
method remove_selected (line 465) | def remove_selected(self):
class ActiveMacroModel (line 474) | class ActiveMacroModel(QAbstractTableModel):
method __init__ (line 479) | def __init__(self, parent, client, *args, **kwargs):
method _emit_all_data (line 486) | def _emit_all_data(self):
method headerData (line 489) | def headerData(self, section, orientation, role):
method rowCount (line 494) | def rowCount(self, parent):
method columnCount (line 497) | def columnCount(self, parent):
method data (line 500) | def data(self, index, role):
method flags (line 505) | def flags(self, index):
method add_macro (line 508) | def add_macro(self, path):
method run_macro (line 519) | def run_macro(self, ind, reqs=None):
method remove_macro (line 524) | def remove_macro(self, ind):
method add_macro_exception (line 531) | def add_macro_exception(self, estr):
method display_macro_complete (line 537) | def display_macro_complete(self, msg):
class ActiveMacroWidget (line 541) | class ActiveMacroWidget(QWidget):
method __init__ (line 544) | def __init__(self, client, *args, **kwargs):
method add_requests (line 650) | def add_requests(self, reqs):
method new_macro (line 656) | def new_macro(self):
method browse_macro (line 666) | def browse_macro(self):
method remove_selected (line 673) | def remove_selected(self):
method run_selected_macro (line 683) | def run_selected_macro(self):
method add_request_output (line 694) | def add_request_output(self, req):
method clear_output (line 698) | def clear_output(self):
method clear_text_output (line 702) | def clear_text_output(self):
method add_macro_output (line 706) | def add_macro_output(self, s):
method import_all_reqs (line 712) | def import_all_reqs(self):
class MacroErrWindow (line 717) | class MacroErrWindow(QWidget):
method __init__ (line 719) | def __init__(self, *args, **kwargs):
method add_error (line 726) | def add_error(self, msg):
method closeEvent (line 731) | def closeEvent(self, event):
function make_err_str (line 736) | def make_err_str(macro, e):
class ArgWindow (line 742) | class ArgWindow(QDialog):
method __init__ (line 743) | def __init__(self, parent, argspec, cached=None):
method _set_canceled (line 798) | def _set_canceled(self):
method get_args (line 801) | def get_args(self):
function get_macro_args (line 812) | def get_macro_args(parent, argspec, cached=None):
function req_python_def (line 820) | def req_python_def(varname, req):
function create_macro_template (line 861) | def create_macro_template(reqs):
function new_active_macro (line 876) | def new_active_macro():
function new_int_macro (line 879) | def new_int_macro():
FILE: guppyproxy/proxy.py
class MessageError (line 20) | class MessageError(Exception):
class ProxyException (line 24) | class ProxyException(Exception):
class InvalidQuery (line 28) | class InvalidQuery(Exception):
class SocketClosed (line 32) | class SocketClosed(Exception):
class SockBuffer (line 36) | class SockBuffer:
method __init__ (line 39) | def __init__(self, sock):
method close (line 44) | def close(self):
method _check_newline (line 54) | def _check_newline(self):
method readline (line 60) | def readline(self):
method send (line 79) | def send(self, data):
class ProxyThread (line 86) | class ProxyThread(QThread):
method __init__ (line 90) | def __init__(self, target=None, args=tuple()):
method run (line 99) | def run(self):
method wait (line 102) | def wait(self):
method waitall (line 106) | def waitall(cls):
function clean_thread (line 111) | def clean_thread(tid):
class Headers (line 117) | class Headers:
method __init__ (line 118) | def __init__(self, headers=None):
method __contains__ (line 130) | def __contains__(self, hd):
method add (line 136) | def add(self, k, v):
method set (line 143) | def set(self, k, v):
method get (line 146) | def get(self, k):
method delete (line 149) | def delete(self, k):
method pairs (line 155) | def pairs(self, key=None):
method dict (line 161) | def dict(self):
class RequestContext (line 172) | class RequestContext:
method __init__ (line 173) | def __init__(self, client, query=None):
method _validate (line 179) | def _validate(self, query):
method set_query (line 182) | def set_query(self, query):
method apply_phrase (line 186) | def apply_phrase(self, phrase):
method pop_phrase (line 190) | def pop_phrase(self):
method apply_filter (line 194) | def apply_filter(self, filt):
method query (line 199) | def query(self):
class URL (line 203) | class URL:
method __init__ (line 204) | def __init__(self, url):
method geturl (line 222) | def geturl(self, include_params=True):
method parameters (line 240) | def parameters(self):
method param_iter (line 246) | def param_iter(self):
method set_param (line 251) | def set_param(self, key, val):
method add_param (line 256) | def add_param(self, key, val):
method del_param (line 264) | def del_param(self, key):
method set_params (line 269) | def set_params(self, params):
class InterceptMacro (line 273) | class InterceptMacro:
method __init__ (line 279) | def __init__(self):
method __repr__ (line 285) | def __repr__(self):
method mangle_request (line 288) | def mangle_request(self, request):
method mangle_response (line 291) | def mangle_response(self, request, response):
method mangle_websocket (line 294) | def mangle_websocket(self, request, response, message):
class HTTPRequest (line 298) | class HTTPRequest:
method __init__ (line 299) | def __init__(self, method="GET", path="/", proto_major=1, proto_minor=1,
method body (line 335) | def body(self):
method body (line 339) | def body(self, bs):
method content_length (line 350) | def content_length(self):
method status_line (line 355) | def status_line(self):
method headers_section (line 361) | def headers_section(self):
method full_message (line 367) | def full_message(self):
method parameters (line 373) | def parameters(self):
method param_iter (line 379) | def param_iter(self, ignore_content_type=False):
method set_param (line 389) | def set_param(self, key, val):
method add_param (line 394) | def add_param(self, key, val):
method del_param (line 402) | def del_param(self, key):
method set_params (line 407) | def set_params(self, params):
method cookies (line 410) | def cookies(self):
method cookie_iter (line 418) | def cookie_iter(self):
method set_cookie (line 423) | def set_cookie(self, key, val):
method del_cookie (line 428) | def del_cookie(self, key):
method set_cookies (line 433) | def set_cookies(self, c):
method add_cookies (line 461) | def add_cookies(self, c):
method full_url (line 477) | def full_url(self):
method copy (line 480) | def copy(self):
class HTTPResponse (line 496) | class HTTPResponse:
method __init__ (line 497) | def __init__(self, status_code=200, reason="OK", proto_major=1, proto_...
method body (line 520) | def body(self):
method body (line 524) | def body(self, bs):
method content_length (line 535) | def content_length(self):
method status_line (line 540) | def status_line(self):
method headers_section (line 546) | def headers_section(self):
method full_message (line 552) | def full_message(self):
method cookies (line 558) | def cookies(self):
method cookie_iter (line 567) | def cookie_iter(self):
method set_cookie (line 572) | def set_cookie(self, key, val):
method del_cookie (line 577) | def del_cookie(self, key):
method set_cookies (line 582) | def set_cookies(self, c):
method copy (line 593) | def copy(self):
class WSMessage (line 605) | class WSMessage:
method __init__ (line 606) | def __init__(self, is_binary=True, message=bytes(), to_server=True,
method copy (line 617) | def copy(self):
function messagingFunction (line 632) | def messagingFunction(func):
class ProxyConnection (line 643) | class ProxyConnection:
method __init__ (line 646) | def __init__(self, kind="", addr=""):
method __enter__ (line 666) | def __enter__(self):
method __exit__ (line 669) | def __exit__(self, exc_type, exc_value, traceback):
method connect_tcp (line 672) | def connect_tcp(self, addr, port):
method connect_unix (line 680) | def connect_unix(self, addr):
method maddr (line 689) | def maddr(self):
method close (line 695) | def close(self):
method read_message (line 704) | def read_message(self):
method submit_command (line 715) | def submit_command(self, cmd):
method reqrsp_cmd (line 721) | def reqrsp_cmd(self, cmd):
method ping (line 732) | def ping(self):
method submit (line 738) | def submit(self, req, storage=0):
method save_new (line 759) | def save_new(self, req, storage):
method _query_storage (line 770) | def _query_storage(self, q, storage, headers_only=False, max_results=0):
method query_storage (line 790) | def query_storage(self, q, storage, max_results=0, headers_only=False):
method req_by_id (line 794) | def req_by_id(self, reqid, storage, headers_only=False):
method set_scope (line 802) | def set_scope(self, filt):
method get_scope (line 810) | def get_scope(self):
method add_tag (line 819) | def add_tag(self, reqid, tag, storage):
method remove_tag (line 829) | def remove_tag(self, reqid, tag, storage):
method clear_tag (line 839) | def clear_tag(self, reqid, storage):
method all_saved_queries (line 848) | def all_saved_queries(self, storage):
method save_query (line 860) | def save_query(self, name, filt, storage):
method load_query (line 870) | def load_query(self, name, storage):
method delete_query (line 880) | def delete_query(self, name, storage):
method add_listener (line 889) | def add_listener(self, addr, port, transparent=False, destHost="",
method remove_listener (line 907) | def remove_listener(self, lid):
method get_listeners (line 915) | def get_listeners(self):
method load_certificates (line 926) | def load_certificates(self, cert_file, pkey_file):
method set_certificates (line 935) | def set_certificates(self, pkey_pem, cert_pem):
method clear_certificates (line 944) | def clear_certificates(self):
method generate_certificates (line 951) | def generate_certificates(self, pkey_file, cert_file):
method generate_pem_certificates (line 960) | def generate_pem_certificates(self):
method validate_query (line 969) | def validate_query(self, query):
method check_request (line 980) | def check_request(self, query, req=None, storage_id=-1, db_id=""):
method add_sqlite_storage (line 994) | def add_sqlite_storage(self, path, desc):
method add_in_memory_storage (line 1004) | def add_in_memory_storage(self, desc):
method close_storage (line 1013) | def close_storage(self, storage_id):
method set_proxy_storage (line 1021) | def set_proxy_storage(self, storage_id):
method list_storage (line 1029) | def list_storage(self):
method set_proxy (line 1040) | def set_proxy(self, use_proxy=False, proxy_host="", proxy_port=0, use_...
method intercept (line 1055) | def intercept(self, macro):
method watch_storage (line 1159) | def watch_storage(self, storage_id=-1, headers_only=True):
method set_plugin_value (line 1190) | def set_plugin_value(self, key, value, storage_id):
method get_plugin_value (line 1200) | def get_plugin_value(self, key, storage_id):
function _serialize_storage (line 1213) | def _serialize_storage(stype, prefix):
class ProxyClient (line 1217) | class ProxyClient:
method __init__ (line 1218) | def __init__(self, binary=None, debug=False, conn_addr=None):
method __enter__ (line 1273) | def __enter__(self):
method __exit__ (line 1280) | def __exit__(self, exc_type, exc_value, traceback):
method __getattr__ (line 1283) | def __getattr__(self, name):
method maddr (line 1289) | def maddr(self):
method execute_binary (line 1295) | def execute_binary(self, binary=None, debug=False, listen_addr=None):
method msg_connect (line 1311) | def msg_connect(self, addr):
method close (line 1316) | def close(self):
method new_conn (line 1323) | def new_conn(self):
method _add_storage (line 1332) | def _add_storage(self, storage, prefix):
method _clear_storage (line 1336) | def _clear_storage(self):
method _get_storage (line 1340) | def _get_storage(self):
method parse_reqid (line 1348) | def parse_reqid(self, reqid):
method storage_iter (line 1373) | def storage_iter(self):
method _stg_or_def (line 1377) | def _stg_or_def(self, storage):
method is_in_context (line 1382) | def is_in_context(self, req):
method in_context_requests (line 1385) | def in_context_requests(self, headers_only=False, max_results=0):
method in_context_requests_async (line 1390) | def in_context_requests_async(self, slot, headers_only=False, max_resu...
method in_context_requests_iter (line 1396) | def in_context_requests_iter(self, headers_only=False, max_results=0):
method get_reqid (line 1407) | def get_reqid(self, req):
method load_by_reqheaders (line 1414) | def load_by_reqheaders(self, req):
method add_sqlite_storage (line 1420) | def add_sqlite_storage(self, path, prefix):
method add_in_memory_storage (line 1427) | def add_in_memory_storage(self, prefix):
method close_storage (line 1434) | def close_storage(self, storage_id):
method set_proxy_storage (line 1440) | def set_proxy_storage(self, storage_id):
method set_storage_prefix (line 1445) | def set_storage_prefix(self, storage_id, prefix):
method save_new (line 1454) | def save_new(self, req, inmem=False, storage=None):
method submit (line 1461) | def submit(self, req, save=False, inmem=False, storage=None):
method query_storage (line 1468) | def query_storage(self, q, max_results=0, headers_only=False, storage=...
method query_storage_async (line 1490) | def query_storage_async(self, slot, *args, **kwargs):
method req_by_id (line 1500) | def req_by_id(self, reqid, storage_id=None, headers_only=False):
method check_request (line 1514) | def check_request(self, query, req=None, reqid=""):
method add_tag (line 1524) | def add_tag(self, reqid, tag, storage=None):
method remove_tag (line 1527) | def remove_tag(self, reqid, tag, storage=None):
method clear_tag (line 1530) | def clear_tag(self, reqid, storage=None):
method all_saved_queries (line 1533) | def all_saved_queries(self, storage=None):
method save_query (line 1536) | def save_query(self, name, filt, storage=None):
method load_query (line 1539) | def load_query(self, name, storage=None):
method delete_query (line 1542) | def delete_query(self, name, storage=None):
method set_plugin_value (line 1545) | def set_plugin_value(self, key, value, storage=None):
method get_plugin_value (line 1548) | def get_plugin_value(self, key, storage=None):
function decode_req (line 1552) | def decode_req(result, headers_only=False, storage=0):
function decode_rsp (line 1600) | def decode_rsp(result, headers_only=False, storage=0):
function decode_ws (line 1617) | def decode_ws(result, storage=0):
function encode_req (line 1641) | def encode_req(req, int_rsp=False):
function encode_rsp (line 1668) | def encode_rsp(rsp, int_rsp=False):
function encode_ws (line 1684) | def encode_ws(ws, int_rsp=False):
function time_from_nsecs (line 1698) | def time_from_nsecs(nsecs):
function time_to_nsecs (line 1704) | def time_to_nsecs(t):
function parse_req_sline (line 1715) | def parse_req_sline(sline):
function parse_rsp_sline (line 1728) | def parse_rsp_sline(sline):
function _parse_message (line 1739) | def _parse_message(bs, sline_parser):
function parse_request (line 1751) | def parse_request(bs, dest_host='', dest_port=80, use_tls=False):
function parse_response (line 1766) | def parse_response(bs):
function get_full_url (line 1778) | def get_full_url(req):
FILE: guppyproxy/repeater.py
class RepeaterWidget (line 7) | class RepeaterWidget(QWidget):
method __init__ (line 9) | def __init__(self, client):
method _set_host (line 60) | def _set_host(self, host):
method _set_port (line 63) | def _set_port(self, port):
method _set_usetls (line 69) | def _set_usetls(self, usetls):
method _set_dest_info (line 75) | def _set_dest_info(self, host, port, usetls):
method _get_dest_info (line 80) | def _get_dest_info(self):
method set_request (line 92) | def set_request(self, req, update_history=True):
method update_req_tags (line 105) | def update_req_tags(self, tags):
method submit (line 110) | def submit(self):
method back (line 137) | def back(self):
method forward (line 144) | def forward(self):
method _update_buttons (line 150) | def _update_buttons(self):
FILE: guppyproxy/reqlist.py
function get_field_entry (line 12) | def get_field_entry():
function get_string_cmp_entry (line 45) | def get_string_cmp_entry():
class StringCmpWidget (line 56) | class StringCmpWidget(QWidget):
method __init__ (line 59) | def __init__(self, *args, **kwargs):
method get_value (line 70) | def get_value(self):
method reset (line 75) | def reset(self):
function dt_sort_key (line 80) | def dt_sort_key(r):
class StringKVWidget (line 86) | class StringKVWidget(QWidget):
method __init__ (line 89) | def __init__(self, *args, **kwargs):
method _show_hide_str2 (line 111) | def _show_hide_str2(self):
method get_value (line 120) | def get_value(self):
method reset (line 126) | def reset(self):
class DropdownFilterEntry (line 131) | class DropdownFilterEntry(QWidget):
method __init__ (line 135) | def __init__(self, *args, **kwargs):
method _display_value_widget (line 173) | def _display_value_widget(self):
method get_value (line 187) | def get_value(self):
method confirm_entry (line 202) | def confirm_entry(self):
class TextFilterEntry (line 211) | class TextFilterEntry(QWidget):
method __init__ (line 215) | def __init__(self, *args, **kwargs):
method confirm_entry (line 226) | def confirm_entry(self):
class FilterEntry (line 233) | class FilterEntry(QWidget):
method __init__ (line 237) | def __init__(self, *args, **kwargs):
method next_entry (line 264) | def next_entry(self):
method set_entry (line 269) | def set_entry(self, entry):
class FilterListWidget (line 275) | class FilterListWidget(QTableWidget):
method __init__ (line 277) | def __init__(self, *args, **kwargs):
method append_fstr (line 291) | def append_fstr(self, fstr):
method set_query (line 297) | def set_query(self, query):
method pop_phrase (line 301) | def pop_phrase(self):
method clear_phrases (line 305) | def clear_phrases(self):
method _append_fstr_row (line 309) | def _append_fstr_row(self, fstr):
method redraw_table (line 314) | def redraw_table(self):
method get_query (line 322) | def get_query(self):
class FilterEditor (line 326) | class FilterEditor(QWidget):
method __init__ (line 335) | def __init__(self, *args, **kwargs):
method save_scope (line 386) | def save_scope(self):
method reset_to_scope (line 392) | def reset_to_scope(self):
method clear_phrases (line 398) | def clear_phrases(self):
method pop_phrase (line 403) | def pop_phrase(self):
method apply_phrase (line 408) | def apply_phrase(self, phrase):
method _apply_builtin_filter (line 418) | def _apply_builtin_filter(self, ind):
method set_is_text (line 424) | def set_is_text(self, is_text):
class ReqListModel (line 431) | class ReqListModel(QAbstractTableModel):
method __init__ (line 446) | def __init__(self, client, *args, **kwargs):
method headerData (line 478) | def headerData(self, section, orientation, role):
method rowCount (line 484) | def rowCount(self, parent):
method columnCount (line 487) | def columnCount(self, parent):
method _gen_req_row (line 490) | def _gen_req_row(self, req):
method data (line 523) | def data(self, index, role):
method canFetchMore (line 539) | def canFetchMore(self, parent):
method fetchMore (line 544) | def fetchMore(self, parent):
method _sort_reqs (line 556) | def _sort_reqs(self):
method _req_ind (line 562) | def _req_ind(self, req=None, reqid=None):
method _emit_all_data (line 571) | def _emit_all_data(self):
method _set_requests (line 574) | def _set_requests(self, reqs):
method set_requests (line 578) | def set_requests(self, reqs):
method clear (line 585) | def clear(self):
method add_request_head (line 592) | def add_request_head(self, req):
method add_request (line 598) | def add_request(self, req):
method add_requests (line 606) | def add_requests(self, reqs):
method update_request (line 615) | def update_request(self, req):
method delete_request (line 622) | def delete_request(self, req=None, reqid=None):
method has_request (line 631) | def has_request(self, req=None, reqid=None):
method get_requests (line 636) | def get_requests(self):
method disable_sort (line 639) | def disable_sort(self):
method enable_sort (line 642) | def enable_sort(self):
method req_by_ind (line 646) | def req_by_ind(self, ind):
class ReqBrowser (line 650) | class ReqBrowser(QWidget):
method __init__ (line 653) | def __init__(self, client, repeater_widget=None, macro_widget=None, re...
method show_filters (line 718) | def show_filters(self):
method show_history (line 721) | def show_history(self):
method show_tree (line 724) | def show_tree(self):
method set_client_context (line 728) | def set_client_context(self, query):
method reset_to_scope (line 732) | def reset_to_scope(self):
method update_viewer (line 736) | def update_viewer(self, reqs):
method update_filters (line 747) | def update_filters(self, query):
method add_request_item (line 751) | def add_request_item(self, req):
method set_requests (line 756) | def set_requests(self, reqs):
method _tab_changed (line 761) | def _tab_changed(self, i):
method _tags_updated (line 766) | def _tags_updated(self, tags):
method set_filter_is_text (line 775) | def set_filter_is_text(self, is_text):
class ReqListUpdater (line 779) | class ReqListUpdater(QObject):
method __init__ (line 785) | def __init__(self, client):
method add_reqlist_widget (line 793) | def add_reqlist_widget(self, widget):
method run_updater (line 803) | def run_updater(self):
method stop (line 823) | def stop(self):
class ReqTableWidget (line 827) | class ReqTableWidget(QWidget):
method __init__ (line 831) | def __init__(self, client, repeater_widget=None, macro_widget=None, *a...
method add_request (line 867) | def add_request(self, req):
method clear (line 880) | def clear(self):
method get_requests (line 883) | def get_requests(self):
method set_requests (line 887) | def set_requests(self, reqs, check_filter=False):
method update_request (line 908) | def update_request(self, req):
method delete_request (line 916) | def delete_request(self, reqid):
method set_filter (line 921) | def set_filter(self, query):
method _updated_selected_request (line 927) | def _updated_selected_request(self, reqs):
method _on_rows_inserted (line 934) | def _on_rows_inserted(self, parent, first, last):
method on_select_change (line 942) | def on_select_change(self, newSelection, oldSelection):
method clear_selection (line 953) | def clear_selection(self):
method get_selected_request (line 956) | def get_selected_request(self):
method get_selected_requests (line 963) | def get_selected_requests(self):
method get_all_requests (line 969) | def get_all_requests(self):
method contextMenuEvent (line 972) | def contextMenuEvent(self, event):
method set_is_loading (line 986) | def set_is_loading(self):
method set_is_not_loading (line 989) | def set_is_not_loading(self):
method set_loading (line 992) | def set_loading(self, is_loading):
method _paint_view (line 1000) | def _paint_view(self, indA, indB):
method delete_selected (line 1004) | def delete_selected(self):
FILE: guppyproxy/reqtree.py
function _include_req (line 7) | def _include_req(req):
class PathNodeItem (line 15) | class PathNodeItem(QStandardItem):
method __init__ (line 17) | def __init__(self, text, *args, **kwargs):
method add_child (line 22) | def add_child(self, text):
method get_child (line 29) | def get_child(self, text):
method add_child_path (line 32) | def add_child_path(self, texts):
class ReqTreeView (line 41) | class ReqTreeView(QWidget):
method __init__ (line 42) | def __init__(self):
method add_request_item (line 57) | def add_request_item(self, req):
method set_requests (line 72) | def set_requests(self, reqs):
method clear (line 79) | def clear(self):
FILE: guppyproxy/reqview.py
class HybridHttpLexer (line 15) | class HybridHttpLexer(Lexer):
method __init__ (line 19) | def __init__(self, max_len=50000, *args, **kwargs):
method get_tokens_unprocessed (line 23) | def get_tokens_unprocessed(self, text):
class InfoWidget (line 61) | class InfoWidget(QWidget):
method __init__ (line 62) | def __init__(self, *args, **kwargs):
method _add_info (line 79) | def _add_info(self, k, v):
method set_request (line 87) | def set_request(self, req):
class ParamWidget (line 150) | class ParamWidget(QWidget):
method __init__ (line 151) | def __init__(self, *args, **kwargs):
method _add_info (line 174) | def _add_info(self, table, k, v):
method format_table (line 182) | def format_table(self, table):
method clear_tables (line 189) | def clear_tables(self):
method set_request (line 194) | def set_request(self, req):
class TagList (line 216) | class TagList(QTableWidget):
method __init__ (line 220) | def __init__(self, *args, **kwargs):
method add_tag (line 232) | def add_tag(self, tag):
method set_tags (line 237) | def set_tags(self, tags, emit=True):
method clear_tags (line 243) | def clear_tags(self):
method _append_str_row (line 248) | def _append_str_row(self, fstr):
method redraw_table (line 253) | def redraw_table(self):
method delete_selected (line 259) | def delete_selected(self):
method get_tags (line 269) | def get_tags(self):
class TagWidget (line 273) | class TagWidget(QWidget):
method __init__ (line 276) | def __init__(self, *args, **kwargs):
method add_tag (line 300) | def add_tag(self):
method set_read_only (line 309) | def set_read_only(self, readonly):
class ReqViewWidget (line 315) | class ReqViewWidget(QWidget):
method __init__ (line 318) | def __init__(self, info_tab=False, param_tab=False, tag_tab=False, *ar...
method set_read_only (line 371) | def set_read_only(self, ro):
method set_tags_read_only (line 374) | def set_tags_read_only(self, ro):
method get_request (line 378) | def get_request(self):
method set_request (line 392) | def set_request(self, req):
method update_editors (line 410) | def update_editors(self):
method show_message (line 419) | def show_message(self):
FILE: guppyproxy/settings.py
class ListenerList (line 10) | class ListenerList(QTableWidget):
method __init__ (line 14) | def __init__(self, *args, **kwargs):
method _add_listener (line 26) | def _add_listener(self, interface, port):
method add_listener (line 29) | def add_listener(self, interface, port):
method set_listeners (line 34) | def set_listeners(self, listeners):
method _append_row (line 41) | def _append_row(self, interface, port):
method redraw_table (line 46) | def redraw_table(self):
method delete_selected (line 52) | def delete_selected(self):
method clear (line 61) | def clear(self):
method get_listeners (line 66) | def get_listeners(self):
class ListenerWidget (line 70) | class ListenerWidget(QWidget):
method __init__ (line 73) | def __init__(self, *args, **kwargs):
method add_listener (line 106) | def add_listener(self):
method set_listeners (line 121) | def set_listeners(self, listeners):
class DatafileWidget (line 125) | class DatafileWidget(QWidget):
method __init__ (line 128) | def __init__(self, *args, **kwargs):
method _load_datafile (line 145) | def _load_datafile(self):
method new_datafile (line 150) | def new_datafile(self):
method open_datafile (line 161) | def open_datafile(self):
class ProxyInfoWidget (line 170) | class ProxyInfoWidget(QWidget):
method __init__ (line 173) | def __init__(self, *args, **kwargs):
method _login_cb_statechange (line 204) | def _login_cb_statechange(self, state):
method _enable_cb_statechange (line 211) | def _enable_cb_statechange(self, state):
method _set_enabled (line 217) | def _set_enabled(self, enabled):
method _set_login_enabled (line 228) | def _set_login_enabled(self, enabled):
method _fill_form (line 233) | def _fill_form(self, enabled, host, port, need_creds, username, passwo...
method _confirm_entry (line 254) | def _confirm_entry(self):
class SettingsWidget (line 278) | class SettingsWidget(QWidget):
method __init__ (line 281) | def __init__(self, client, *args, **kwargs):
method load_config (line 303) | def load_config(self):
method _load_datafile (line 326) | def _load_datafile(self, path):
method _listeners_updated (line 341) | def _listeners_updated(self, new_listeners):
method _set_proxy_settings (line 370) | def _set_proxy_settings(self, proxy_data):
method reload_listeners (line 382) | def reload_listeners(self):
method save_config (line 392) | def save_config(self):
FILE: guppyproxy/shortcuts.py
class GuppyShortcuts (line 6) | class GuppyShortcuts(QObject):
method __init__ (line 22) | def __init__(self, guppy_window):
method add_shortcut (line 93) | def add_shortcut(self, action, desc, func, key=None):
method set_key (line 100) | def set_key(self, action, key):
method get_desc (line 104) | def get_desc(self, action):
method nav_to_filter_text (line 108) | def nav_to_filter_text(self):
method nav_to_filter_dropdown (line 115) | def nav_to_filter_dropdown(self):
method nav_to_filter_pop (line 121) | def nav_to_filter_pop(self):
method nav_to_history (line 127) | def nav_to_history(self):
method nav_to_tree (line 133) | def nav_to_tree(self):
method nav_to_repeater (line 138) | def nav_to_repeater(self):
method nav_to_interceptor (line 142) | def nav_to_interceptor(self):
method nav_to_decoder (line 146) | def nav_to_decoder(self):
method nav_to_decoder_and_paste (line 150) | def nav_to_decoder_and_paste(self):
method open_datafile (line 156) | def open_datafile(self):
method new_datafile (line 160) | def new_datafile(self):
method nav_to_active_macros (line 164) | def nav_to_active_macros(self):
method nav_to_int_macros (line 168) | def nav_to_int_macros(self):
FILE: guppyproxy/util.py
class DisableUpdates (line 18) | class DisableUpdates:
method __init__ (line 19) | def __init__(self, *args):
method __enter__ (line 32) | def __enter__(self):
method __exit__ (line 35) | def __exit__(self, exc_type, exc_value, traceback):
function str_hash_code (line 44) | def str_hash_code(s):
function dbgline (line 56) | def dbgline():
function is_printable (line 62) | def is_printable(s):
function printable_data (line 69) | def printable_data(data, include_newline=True):
function max_len_str (line 82) | def max_len_str(s, ln):
function display_error_box (line 90) | def display_error_box(msg, title="Error"):
function display_info_box (line 99) | def display_info_box(msg, title="Message"):
function copy_to_clipboard (line 108) | def copy_to_clipboard(s):
function paste_clipboard (line 112) | def paste_clipboard():
function running_as_app (line 116) | def running_as_app():
function set_running_as_app (line 121) | def set_running_as_app(is_app):
function default_dialog_dir (line 126) | def default_dialog_dir():
function set_default_dialog_dir (line 137) | def set_default_dialog_dir(s):
function save_dialog (line 142) | def save_dialog(parent, filter_string="Any File (*)", caption="Save File...
function open_dialog (line 150) | def open_dialog(parent, filter_string="Any File (*)", default_dir=None):
function display_req_context (line 158) | def display_req_context(parent, client, req, event, repeater_widget=None...
function display_multi_req_context (line 251) | def display_multi_req_context(parent, client, reqs, event, macro_widget=...
function method_color (line 274) | def method_color(method):
function sc_color (line 286) | def sc_color(sc):
function host_color (line 301) | def host_color(hostport):
function str_color (line 304) | def str_color(s, lighten=0, seed=0):
function hostport (line 319) | def hostport(req):
function _sh_esc (line 329) | def _sh_esc(s):
function curl_command (line 335) | def curl_command(req):
function list_remove (line 355) | def list_remove(lst, inds):
function hexdump (line 359) | def hexdump(src, length=16):
function confirm (line 370) | def confirm(message, default='n'):
function utc2local (line 397) | def utc2local(utc):
function datetime_string (line 403) | def datetime_string(dt):
function query_to_str (line 409) | def query_to_str(query):
function textedit_highlight (line 420) | def textedit_highlight(text, lexer):
Condensed preview — 26 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (261K chars).
[
{
"path": ".gitignore",
"chars": 116,
"preview": "*.pyc\ndata.db\n.coverage\n.cache\ntests/.cache\n.DS_Store\nTAGS\nconfig.json\nbuild/*\n*.egg-info/*\n.#*\n*notes*\n*.org\nstart\n"
},
{
"path": "LICENSE",
"chars": 1065,
"preview": "MIT License\n\nCopyright (c) 2018 Rob Glew\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\no"
},
{
"path": "README.md",
"chars": 24224,
"preview": "# Guppy Proxy\n\nThe Guppy Proxy is an intercepting proxy for performing web application security testing. Its features ar"
},
{
"path": "guppyproxy/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "guppyproxy/config.py",
"chars": 4018,
"preview": "import copy\nimport json\n\ndefault_config = \"\"\"{\n \"listeners\": [\n {\"iface\": \"127.0.0.1\", \"port\": 8080}\n ],\n "
},
{
"path": "guppyproxy/decoder.py",
"chars": 4348,
"preview": "import html\nimport base64\nimport urllib\nimport json\n\nfrom guppyproxy.util import display_error_box\nfrom guppyproxy.hexte"
},
{
"path": "guppyproxy/gui.py",
"chars": 3819,
"preview": "import random\n\nfrom guppyproxy.reqlist import ReqBrowser, ReqListModel\nfrom guppyproxy.repeater import RepeaterWidget\nfr"
},
{
"path": "guppyproxy/gup.py",
"chars": 3779,
"preview": "import argparse\nimport sys\nimport os\n\nfrom PyQt5.QtWidgets import QApplication\nfrom PyQt5.QtCore import Qt\nfrom guppypro"
},
{
"path": "guppyproxy/hexteditor.py",
"chars": 17852,
"preview": "import base64\nfrom guppyproxy.util import printable_data, qtprintable, textedit_highlight, DisableUpdates\nfrom guppyprox"
},
{
"path": "guppyproxy/interceptor.py",
"chars": 7316,
"preview": "from guppyproxy.util import display_error_box\nfrom guppyproxy.proxy import InterceptMacro, parse_request, parse_response"
},
{
"path": "guppyproxy/macros.py",
"chars": 30601,
"preview": "import glob\nimport imp\nimport os\nimport random\nimport re\nimport stat\nimport sys\nimport traceback\n\nfrom guppyproxy.proxy "
},
{
"path": "guppyproxy/proxy.py",
"chars": 54826,
"preview": "#!/usr/bin/env python3\n\nimport base64\nimport copy\nimport datetime\nimport json\nimport math\nimport re\nimport socket\nimport"
},
{
"path": "guppyproxy/repeater.py",
"chars": 5268,
"preview": "from guppyproxy.util import display_error_box\nfrom guppyproxy.reqview import ReqViewWidget\nfrom PyQt5.QtWidgets import Q"
},
{
"path": "guppyproxy/reqlist.py",
"chars": 35286,
"preview": "import threading\nimport shlex\n\nfrom guppyproxy.util import max_len_str, query_to_str, display_error_box, display_info_bo"
},
{
"path": "guppyproxy/reqtree.py",
"chars": 2486,
"preview": "from guppyproxy.proxy import HTTPRequest\nfrom PyQt5.QtWidgets import QWidget, QTreeView, QVBoxLayout\nfrom PyQt5.QtGui im"
},
{
"path": "guppyproxy/reqview.py",
"chars": 14711,
"preview": "import re\n\nfrom guppyproxy.util import datetime_string, DisableUpdates\nfrom guppyproxy.proxy import HTTPRequest, get_ful"
},
{
"path": "guppyproxy/settings.py",
"chars": 14564,
"preview": "from guppyproxy.util import list_remove, display_error_box, set_default_dialog_dir, default_dialog_dir, save_dialog, ope"
},
{
"path": "guppyproxy/shortcuts.py",
"chars": 5911,
"preview": "from guppyproxy.util import display_info_box, paste_clipboard\nfrom PyQt5.QtCore import pyqtSlot, QObject, Qt\nfrom PyQt5."
},
{
"path": "guppyproxy/util.py",
"chars": 13013,
"preview": "import os\nimport string\nimport time\nimport datetime\nimport random\nfrom guppyproxy.proxy import get_full_url, Headers\nfro"
},
{
"path": "install.sh",
"chars": 4944,
"preview": "#!/bin/bash\n\nprompt_yn() {\n read -p \"$1 (yN) \" yn;\n case $yn in\n [Yy]* ) return 0;;\n * ) return 1;;\n"
},
{
"path": "puppyrsc/NOTE.md",
"chars": 354,
"preview": "These binaries are pre-built versions of puppy (<https://github.com/roglew/puppy>) which is used to proxy HTTP requests "
},
{
"path": "setup.py",
"chars": 1323,
"preview": "#!/usr/bin/env python\n\nimport pkgutil\nfrom setuptools import setup, find_packages\ntry:\n import py2app\nexcept ImportEr"
}
]
// ... and 4 more files (download for full content)
About this extraction
This page contains the full source code of the roglew/guppy-proxy GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 26 files (30.2 MB), approximately 57.9k tokens, and a symbol index with 646 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.