## Contents
* [Requirements](#requirements)
* [Installation](#installation)
* [Features](#features)
* [Configuration](#configuration)
* [Authentication/Authorisation](#authenticationauthorisation)
* [Request/Response bodies](#requestresponse-bodies)
* [Meta-Profiling](#meta-profiling)
* [Recording a fraction of requests](#recording-a-fraction-of-requests)
* [Limiting request/response data](#limiting-requestresponse-data)
* [Clearing logged data](#clearing-logged-data)
* [Contributing](#contributing)
* [Development Environment](#development-environment)
## Requirements
Silk has been tested with:
* Django: 4.2, 5.1, 5.2, 6.0
* Python: 3.10, 3.11, 3.12, 3.13, 3.14
## Installation
Via pip into a `virtualenv`:
```bash
pip install django-silk
```
To including optional formatting of python snippets:
```bash
pip install django-silk[formatting]
```
In `settings.py` add the following:
```python
MIDDLEWARE = [
...
'silk.middleware.SilkyMiddleware',
...
]
TEMPLATES = [{
...
'OPTIONS': {
'context_processors': [
...
'django.template.context_processors.request',
],
},
}]
INSTALLED_APPS = (
...
'silk'
)
```
**Note:** The order of middleware is sensitive. If any middleware placed before `silk.middleware.SilkyMiddleware` returns a response without invoking its `get_response`, the `SilkyMiddleware` won’t run. To avoid this, ensure that middleware preceding `SilkyMiddleware` does not bypass or return a response without calling its `get_response`. For further details, check out the [Django documentation](https://docs.djangoproject.com/en/dev/topics/http/middleware/#middleware-order-and-layering).
**Note:** If you are using `django.middleware.gzip.GZipMiddleware`, place that **before** `silk.middleware.SilkyMiddleware`, otherwise you will get an encoding error.
If you want to use custom middleware, for example you developed the subclass of `silk.middleware.SilkyMiddleware`, so you can use this combination of settings:
```python
# Specify the path where is the custom middleware placed
SILKY_MIDDLEWARE_CLASS = 'path.to.your.middleware.MyCustomSilkyMiddleware'
# Use this variable in list of middleware
MIDDLEWARE = [
...
SILKY_MIDDLEWARE_CLASS,
...
]
```
To enable access to the user interface add the following to your `urls.py`:
```python
urlpatterns += [path('silk/', include('silk.urls', namespace='silk'))]
```
before running migrate:
```bash
python manage.py migrate
python manage.py collectstatic
```
Silk will automatically begin interception of requests and you can proceed to add profiling
if required. The UI can be reached at `/silk/`
### Alternative Installation
Via [github tags](https://github.com/jazzband/django-silk/releases):
```bash
pip install git+https://github.com/jazzband/django-silk.git@
It records things like:
* Time taken
* Num. queries
* Time spent on queries
* Request/Response headers
* Request/Response bodies
and so on.
Further details on each request are also available by clicking the relevant request:
### SQL Inspection
Silk also intercepts SQL queries that are generated by each request. We can get a summary on things like
the tables involved, number of joins and execution time (the table can be sorted by clicking on a column header):
Before diving into the stack trace to figure out where this request is coming from:
### Profiling
Turn on the SILKY_PYTHON_PROFILER setting to use Python's built-in `cProfile` profiler. Each request will be separately profiled and the profiler's output will be available on the request's Profiling page in the Silk UI. Note that as of Python 3.12, `cProfile` cannot run concurrently so [django-silk under Python 3.12 and later will not profile if another profile is running](https://github.com/jazzband/django-silk/pull/692) (even its own profiler in another thread).
```python
SILKY_PYTHON_PROFILER = True
```
If you would like to also generate a binary `.prof` file set the following:
```python
SILKY_PYTHON_PROFILER_BINARY = True
```
When enabled, a graph visualisation generated using [gprof2dot](https://github.com/jrfonseca/gprof2dot) and [viz.js](https://github.com/almende/vis) is shown in the profile detail page:
A custom storage class can be used for the saved generated binary `.prof` files:
```python
# For Django >= 4.2 and Django-Silk >= 5.1.0:
# See https://docs.djangoproject.com/en/5.0/ref/settings/#std-setting-STORAGES
STORAGES = {
'SILKY_STORAGE': {
'BACKEND': 'path.to.StorageClass',
},
# ...
}
# For Django < 4.2 or Django-Silk < 5.1.0
SILKY_STORAGE_CLASS = 'path.to.StorageClass'
```
The default storage class is `silk.storage.ProfilerResultStorage`, and when using that you can specify a path of your choosing. You must ensure the specified directory exists.
```python
# If this is not set, MEDIA_ROOT will be used.
SILKY_PYTHON_PROFILER_RESULT_PATH = '/path/to/profiles/'
```
A download button will become available with a binary `.prof` file for every request. This file can be used for further analysis using [snakeviz](https://github.com/jiffyclub/snakeviz) or other cProfile tools
To retrieve which endpoint generates a specific profile file it is possible to add a stub of the request path in the file name with the following:
```python
SILKY_PYTHON_PROFILER_EXTENDED_FILE_NAME = True
```
Silk can also be used to profile specific blocks of code/functions. It provides a decorator and a context
manager for this purpose.
For example:
```python
from silk.profiling.profiler import silk_profile
@silk_profile(name='View Blog Post')
def post(request, post_id):
p = Post.objects.get(pk=post_id)
return render(request, 'post.html', {
'post': p
})
```
Whenever a blog post is viewed we get an entry within the Silk UI:
Silk profiling not only provides execution time, but also collects SQL queries executed within the block in the same fashion as with requests:
#### Decorator
The silk decorator can be applied to both functions and methods
```python
from silk.profiling.profiler import silk_profile
# Profile a view function
@silk_profile(name='View Blog Post')
def post(request, post_id):
p = Post.objects.get(pk=post_id)
return render(request, 'post.html', {
'post': p
})
# Profile a method in a view class
class MyView(View):
@silk_profile(name='View Blog Post')
def get(self, request):
p = Post.objects.get(pk=post_id)
return render(request, 'post.html', {
'post': p
})
```
#### Context Manager
Using a context manager means we can add additional context to the name which can be useful for
narrowing down slowness to particular database records.
```python
def post(request, post_id):
with silk_profile(name='View Blog Post #%d' % self.pk):
p = Post.objects.get(pk=post_id)
return render(request, 'post.html', {
'post': p
})
```
#### Dynamic Profiling
One of Silk's more interesting features is dynamic profiling. If for example we wanted to profile a function in a dependency to which we only have read-only access (e.g. system python libraries owned by root) we can add the following to `settings.py` to apply a decorator at runtime:
```python
SILKY_DYNAMIC_PROFILING = [{
'module': 'path.to.module',
'function': 'MyClass.bar'
}]
```
which is roughly equivalent to:
```python
class MyClass:
@silk_profile()
def bar(self):
pass
```
The below summarizes the possibilities:
```python
"""
Dynamic function decorator
"""
SILKY_DYNAMIC_PROFILING = [{
'module': 'path.to.module',
'function': 'foo'
}]
# ... is roughly equivalent to
@silk_profile()
def foo():
pass
"""
Dynamic method decorator
"""
SILKY_DYNAMIC_PROFILING = [{
'module': 'path.to.module',
'function': 'MyClass.bar'
}]
# ... is roughly equivalent to
class MyClass:
@silk_profile()
def bar(self):
pass
"""
Dynamic code block profiling
"""
SILKY_DYNAMIC_PROFILING = [{
'module': 'path.to.module',
'function': 'foo',
# Line numbers are relative to the function as opposed to the file in which it resides
'start_line': 1,
'end_line': 2,
'name': 'Slow Foo'
}]
# ... is roughly equivalent to
def foo():
with silk_profile(name='Slow Foo'):
print (1)
print (2)
print(3)
print(4)
```
Note that dynamic profiling behaves in a similar fashion to that of the python mock framework in that
we modify the function in-place e.g:
```python
""" my.module """
from another.module import foo
# ...do some stuff
foo()
# ...do some other stuff
```
,we would profile `foo` by dynamically decorating `my.module.foo` as opposed to `another.module.foo`:
```python
SILKY_DYNAMIC_PROFILING = [{
'module': 'my.module',
'function': 'foo'
}]
```
If we were to apply the dynamic profile to the functions source module `another.module.foo` **after**
it has already been imported, no profiling would be triggered.
#### Custom Logic for Profiling
Sometimes you may want to dynamically control when the profiler runs. You can write your own logic for when to enable the profiler. To do this add the following to your `settings.py`:
This setting is mutually exclusive with SILKY_PYTHON_PROFILER and will be used over it if present. It will work with SILKY_DYNAMIC_PROFILING.
```python
def my_custom_logic(request):
return 'profile_requests' in request.session
SILKY_PYTHON_PROFILER_FUNC = my_custom_logic # profile only session has recording enabled.
```
You can also use a `lambda`.
```python
# profile only session has recording enabled.
SILKY_PYTHON_PROFILER_FUNC = lambda request: 'profile_requests' in request.session
```
### Code Generation
Silk currently generates two bits of code per request:
Both are intended for use in replaying the request. The curl command can be used to replay via command-line and the python code can be used within a Django unit test or simply as a standalone script.
## Configuration
### Authentication/Authorisation
By default anybody can access the Silk user interface by heading to `/silk/`. To enable your Django
auth backend place the following in `settings.py`:
```python
SILKY_AUTHENTICATION = True # User must login
SILKY_AUTHORISATION = True # User must have permissions
```
If `SILKY_AUTHORISATION` is `True`, by default Silk will only authorise users with `is_staff` attribute set to `True`.
You can customise this using the following in `settings.py`:
```python
def my_custom_perms(user):
return user.is_allowed_to_use_silk
SILKY_PERMISSIONS = my_custom_perms
```
You can also use a `lambda`.
```python
SILKY_PERMISSIONS = lambda user: user.is_superuser
```
### Request/Response bodies
By default, Silk will save down the request and response bodies for each request for future viewing
no matter how large. If Silk is used in production under heavy volume with large bodies this can have
a huge impact on space/time performance. This behaviour can be configured with the following options:
```python
SILKY_MAX_REQUEST_BODY_SIZE = -1 # Silk takes anything <0 as no limit
SILKY_MAX_RESPONSE_BODY_SIZE = 1024 # If response body>1024 bytes, ignore
```
### Meta-Profiling
Sometimes it is useful to be able to see what effect Silk is having on the request/response time. To do this add
the following to your `settings.py`:
```python
SILKY_META = True
```
Silk will then record how long it takes to save everything down to the database at the end of each
request:
Note that in the above screenshot, this means that the request took 29ms (22ms from Django and 7ms from Silk)
### Recording a Fraction of Requests
On high-load sites it may be helpful to only record a fraction of the requests that are made. To do this add the following to your `settings.py`:
Note: This setting is mutually exclusive with SILKY_INTERCEPT_FUNC.
```python
SILKY_INTERCEPT_PERCENT = 50 # log only 50% of requests
```
#### Custom Logic for Recording Requests
On high-load sites it may also be helpful to write your own logic for when to intercept requests. To do this add the following to your `settings.py`:
Note: This setting is mutually exclusive with SILKY_INTERCEPT_PERCENT.
```python
def my_custom_logic(request):
return 'record_requests' in request.session
SILKY_INTERCEPT_FUNC = my_custom_logic # log only session has recording enabled.
```
You can also use a `lambda`.
```python
# log only session has recording enabled.
SILKY_INTERCEPT_FUNC = lambda request: 'record_requests' in request.session
```
### Limiting request/response data
To make sure silky garbage collects old request/response data, a config var can be set to limit the number of request/response rows it stores.
```python
SILKY_MAX_RECORDED_REQUESTS = 10**4
```
The garbage collection is only run on a percentage of requests to reduce overhead. It can be adjusted with this config:
```python
SILKY_MAX_RECORDED_REQUESTS_CHECK_PERCENT = 10
```
In case you want decouple silk's garbage collection from your webserver's request processing, set SILKY_MAX_RECORDED_REQUESTS_CHECK_PERCENT=0 and trigger it manually, e.g. in a cron job:
```bash
python manage.py silk_request_garbage_collect
```
### Enable query analysis
To enable query analysis when supported by the dbms a config var can be set in order to execute queries with the analyze features.
```python
SILKY_ANALYZE_QUERIES = True
```
**Warning:** This setting may cause the database to execute the same query twice, depending on the backend. For instance, `EXPLAIN ANALYZE` in Postgres will [actually execute the query](https://www.postgresql.org/docs/current/sql-explain.html), which may result in unexpected data updates. Set this to True with caution.
To pass additional params for profiling when supported by the dbms (e.g. VERBOSE, FORMAT JSON), you can do this in the following manner.
```python
SILKY_EXPLAIN_FLAGS = {'format':'JSON', 'costs': True}
```
### Masking sensitive data on request body
By default, Silk is filtering values that contains the following keys (they are case insensitive)
```python
SILKY_SENSITIVE_KEYS = {'username', 'api', 'token', 'key', 'secret', 'password', 'signature'}
```
But sometimes, you might want to have your own sensitive keywords, then above configuration can be modified
```python
SILKY_SENSITIVE_KEYS = {'custom-password'}
```
### Clearing logged data
A management command will wipe out all logged data:
```bash
python manage.py silk_clear_request_log
```
## Contributing
[](https://jazzband.co/)
This is a [Jazzband](https://jazzband.co/) project. By contributing you agree to abide by the [Contributor Code of Conduct](https://jazzband.co/about/conduct) and follow the [guidelines](https://jazzband.co/about/guidelines).
### Development Environment
Silk features a project named `project` that can be used for `silk` development. It has the `silk` code symlinked so
you can work on the sample `project` and on the `silk` package at the same time.
In order to setup local development you should first install all the dependencies for the test `project`. From the
root of the `project` directory:
```bash
pip install -r requirements.txt
```
You will also need to install `silk`'s dependencies. From the root of the git repository:
```bash
pip install -e .
```
At this point your virtual environment should have everything it needs to run both the sample `project` and
`silk` successfully.
Before running, you must set the `DB_ENGINE` and `DB_NAME` environment variables:
```bash
export DB_ENGINE=sqlite3
export DB_NAME=db.sqlite3
```
For other combinations, check [`tox.ini`](./tox.ini).
Now from the root of the sample `project` apply the migrations
```bash
python manage.py migrate
```
Now from the root of the sample `project` directory start the django server
```bash
python manage.py runserver
```
#### Running the tests
```bash
cd project
python manage.py test
```
Happy profiling!
================================================
FILE: docs/Makefile
================================================
# Makefile for Sphinx documentation
# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
PAPER =
BUILDDIR = _build
# User-friendly check for sphinx-build
ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1)
$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/)
endif
# Internal variables.
PAPEROPT_a4 = -D latex_paper_size=a4
PAPEROPT_letter = -D latex_paper_size=letter
ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
# the i18n builder cannot share the environment and doctrees with the others
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext
help:
@echo "Please use \`make Use this app for testing and playing around with Silk. Displays a Blind creation form.
================================================ FILE: project/example_app/templates/example_app/index.html ================================================Use this app for testing and playing around with Silk. Displays a range of Blinds. Use admin to add them.
| Photo | Name | Child safe? |
|---|---|---|
| {% if blind.photo %} |
{{ blind.name }} | {% if blind.child_safe %}Yes{% else %}No{% endif %} |
Your username and password didn't match. Please try again.
{% endif %} ================================================ FILE: project/example_app/tests.py ================================================ # Create your tests here. ================================================ FILE: project/example_app/urls.py ================================================ from django.urls import path from . import views app_name = 'example_app' urlpatterns = [ path(route='', view=views.index, name='index'), path(route='create', view=views.ExampleCreateView.as_view(), name='create'), ] ================================================ FILE: project/example_app/views.py ================================================ from time import sleep # Create your views here. from django.shortcuts import render from django.urls import reverse_lazy from django.views.generic import CreateView from example_app import models from silk.profiling.profiler import silk_profile def index(request): @silk_profile() def do_something_long(): sleep(1.345) with silk_profile(name='Why do this take so long?'): do_something_long() return render(request, 'example_app/index.html', {'blinds': models.Blind.objects.all()}) class ExampleCreateView(CreateView): model = models.Blind fields = ['name'] success_url = reverse_lazy('example_app:index') ================================================ FILE: project/manage.py ================================================ #!/usr/bin/env python """Define the Django Silk management entry.""" import os import sys if __name__ == "__main__": os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project.settings") from django.core.management import execute_from_command_line execute_from_command_line(sys.argv) ================================================ FILE: project/project/__init__.py ================================================ __author__ = 'mtford' ================================================ FILE: project/project/settings.py ================================================ import os BASE_DIR = os.path.dirname(os.path.dirname(__file__)) SECRET_KEY = 'ey5!m&h-uj6c7dzp@(o1%96okkq4!&bjja%oi*v3r=2t(!$7os' DEBUG = True DEBUG_PROPAGATE_EXCEPTIONS = True ALLOWED_HOSTS = [] INSTALLED_APPS = ( 'django.contrib.staticfiles', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.messages', 'django.contrib.sessions', 'silk', 'example_app' ) ROOT_URLCONF = 'project.urls' DEFAULT_AUTO_FIELD = 'django.db.models.AutoField' MIDDLEWARE = [ 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'silk.middleware.SilkyMiddleware' ] WSGI_APPLICATION = 'wsgi.application' DB_ENGINE = os.environ.get("DB_ENGINE", "postgresql") DATABASES = { "default": { "ENGINE": f"django.db.backends.{DB_ENGINE}", "NAME": os.environ.get("DB_NAME", "postgres"), "USER": os.environ.get("DB_USER", 'postgres'), "PASSWORD": os.environ.get("DB_PASSWORD", "postgres"), "HOST": os.environ.get("DB_HOST", "127.0.0.1"), "PORT": os.environ.get("DB_PORT", 5432), "ATOMIC_REQUESTS": True }, } LANGUAGE_CODE = 'en-us' TIME_ZONE = 'UTC' USE_I18N = True USE_TZ = True LOGGING = { 'version': 1, 'formatters': { 'mosayc': { 'format': '%(asctime)-15s %(levelname)-7s %(message)s [%(funcName)s (%(filename)s:%(lineno)s)]', } }, 'handlers': { 'console': { 'level': 'DEBUG', 'class': 'logging.StreamHandler', 'formatter': 'mosayc' } }, 'loggers': { 'silk': { 'handlers': ['console'], 'level': 'DEBUG' } }, } STATIC_URL = '/static/' STATICFILES_FINDERS = ( 'django.contrib.staticfiles.finders.FileSystemFinder', 'django.contrib.staticfiles.finders.AppDirectoriesFinder', ) TEMP_DIR = os.path.join(BASE_DIR, "tmp") STATIC_ROOT = os.path.join(TEMP_DIR, "static") if not os.path.exists(STATIC_ROOT): os.makedirs(STATIC_ROOT) MEDIA_ROOT = BASE_DIR + '/media/' MEDIA_URL = '/media/' if not os.path.exists(MEDIA_ROOT): os.mkdir(MEDIA_ROOT) TEMPLATES = [ { 'BACKEND': 'django.template.backends.django.DjangoTemplates', 'DIRS': [], 'APP_DIRS': True, 'OPTIONS': { 'context_processors': [ 'django.template.context_processors.request', ], }, }, ] LOGIN_URL = '/login/' LOGIN_REDIRECT_URL = '/' SILKY_META = True SILKY_PYTHON_PROFILER = True SILKY_PYTHON_PROFILER_BINARY = True # Do not garbage collect for tests SILKY_MAX_RECORDED_REQUESTS_CHECK_PERCENT = 0 # SILKY_AUTHENTICATION = True # SILKY_AUTHORISATION = True ================================================ FILE: project/project/urls.py ================================================ from django.conf import settings from django.conf.urls.static import static from django.contrib import admin from django.contrib.auth import views from django.urls import include, path urlpatterns = [ path( route='silk/', view=include('silk.urls', namespace='silk'), ), path( route='example_app/', view=include('example_app.urls', namespace='example_app'), ), path(route='admin/', view=admin.site.urls), ] urlpatterns += [ path( route='login/', view=views.LoginView.as_view( template_name='example_app/login.html' ), name='login', ), ] urlpatterns += static(settings.STATIC_URL, document_root=settings.STATIC_ROOT) + \ static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT) ================================================ FILE: project/tests/__init__.py ================================================ from . import * # noqa: F401, F403 ================================================ FILE: project/tests/data/__init__.py ================================================ __author__ = 'mtford' ================================================ FILE: project/tests/data/dynamic.py ================================================ def foo(): print('1') print('2') print('3') def foo2(): print('1') print('2') print('3') ================================================ FILE: project/tests/factories.py ================================================ import factory import factory.fuzzy from example_app.models import Blind from silk.models import Request, Response, SQLQuery HTTP_METHODS = ['GET', 'POST', 'PUT', 'PATCH', 'HEAD', 'OPTIONS'] STATUS_CODES = [200, 201, 300, 301, 302, 401, 403, 404] class SQLQueryFactory(factory.django.DjangoModelFactory): query = factory.Sequence(lambda num: 'SELECT foo FROM bar WHERE foo=%s' % num) traceback = factory.Sequence(lambda num: 'Traceback #%s' % num) class Meta: model = SQLQuery class RequestMinFactory(factory.django.DjangoModelFactory): path = factory.Faker('uri_path') method = factory.fuzzy.FuzzyChoice(HTTP_METHODS) class Meta: model = Request class ResponseFactory(factory.django.DjangoModelFactory): request = factory.SubFactory(RequestMinFactory) status_code = factory.fuzzy.FuzzyChoice(STATUS_CODES) class Meta: model = Response class BlindFactory(factory.django.DjangoModelFactory): name = factory.Faker('pystr', min_chars=5, max_chars=10) child_safe = factory.Faker('pybool') photo = factory.django.ImageField() class Meta: model = Blind ================================================ FILE: project/tests/test_app_config.py ================================================ from django.apps import apps as proj_apps from django.test import TestCase from silk.apps import SilkAppConfig class TestAppConfig(TestCase): """ Test if correct AppConfig class is loaded by Django. """ def test_app_config_loaded(self): silk_app_config = proj_apps.get_app_config("silk") self.assertIsInstance(silk_app_config, SilkAppConfig) ================================================ FILE: project/tests/test_code.py ================================================ from collections import namedtuple from django.test import TestCase from silk.views.code import _code, _code_context, _code_context_from_request FILE_PATH = __file__ LINE_NUM = 5 END_LINE_NUM = 10 with open(__file__) as f: ACTUAL_LINES = [line + '\n' for line in f.read().split('\n')] class CodeTestCase(TestCase): def assertActualLineEqual(self, actual_line, end_line_num=None): expected_actual_line = ACTUAL_LINES[LINE_NUM - 1:end_line_num or LINE_NUM] self.assertEqual(actual_line, expected_actual_line) def assertCodeEqual(self, code): expected_code = [line.strip('\n') for line in ACTUAL_LINES[0:LINE_NUM + 10]] + [''] self.assertEqual(code, expected_code) def test_code(self): for end_line_num in None, END_LINE_NUM: actual_line, code = _code(FILE_PATH, LINE_NUM, end_line_num) self.assertActualLineEqual(actual_line, end_line_num) self.assertCodeEqual(code) def test_code_context(self): for end_line_num in None, END_LINE_NUM: for prefix in '', 'salchicha_': context = _code_context(FILE_PATH, LINE_NUM, end_line_num, prefix) self.assertActualLineEqual(context[prefix + 'actual_line'], end_line_num) self.assertCodeEqual(context[prefix + 'code']) self.assertEqual(context[prefix + 'file_path'], FILE_PATH) self.assertEqual(context[prefix + 'line_num'], LINE_NUM) def test_code_context_from_request(self): for end_line_num in None, END_LINE_NUM: for prefix in '', 'salchicha_': request = namedtuple('Request', 'GET')(dict(file_path=FILE_PATH, line_num=LINE_NUM)) context = _code_context_from_request(request, end_line_num, prefix) self.assertActualLineEqual(context[prefix + 'actual_line'], end_line_num) self.assertCodeEqual(context[prefix + 'code']) self.assertEqual(context[prefix + 'file_path'], FILE_PATH) self.assertEqual(context[prefix + 'line_num'], LINE_NUM) ================================================ FILE: project/tests/test_code_gen_curl.py ================================================ import shlex from unittest import TestCase from silk.code_generation.curl import curl_cmd class TestCodeGenCurl(TestCase): def test_post_json(self): result = curl_cmd( url="https://example.org/alpha/beta", method="POST", body={"gamma": "delta"}, content_type="application/json", ) result_words = shlex.split(result) self.assertEqual(result_words, [ 'curl', '-X', 'POST', '-H', 'content-type: application/json', '-d', '{"gamma": "delta"}', 'https://example.org/alpha/beta' ]) ================================================ FILE: project/tests/test_code_gen_django.py ================================================ import textwrap from unittest import TestCase from silk.code_generation.django_test_client import gen class TestCodeGenDjango(TestCase): def test_post(self): result = gen( path="/alpha/beta", method="POST", data={"gamma": "delta", "epsilon": "zeta"}, content_type="application/x-www-form-urlencoded", ) self.assertEqual(result, textwrap.dedent("""\ from django.test import Client c = Client() response = c.post(path='/alpha/beta', data={'gamma': 'delta', 'epsilon': 'zeta'}, content_type='application/x-www-form-urlencoded') """)) ================================================ FILE: project/tests/test_collector.py ================================================ import cProfile import os.path import sys from django.test import TestCase from tests.util import DictStorage from silk.collector import DataCollector from silk.config import SilkyConfig from .factories import RequestMinFactory class TestCollector(TestCase): def test_singleton(self): a = DataCollector() b = DataCollector() c = DataCollector() self.assertTrue(a == b == c) def test_query_registration(self): mock_query = {} DataCollector().register_query(mock_query) self.assertIn(mock_query, list(DataCollector().queries.values())) def test_clear(self): self.test_query_registration() DataCollector().clear() self.assertFalse(DataCollector().queries) def test_finalise(self): request = RequestMinFactory() DataCollector().configure(request) with self.subTest("Default file-based storage"): DataCollector().finalise() file = DataCollector().request.prof_file self.assertIsNotNone(file) with file.storage.open(file.name) as f: content = f.read() self.assertTrue(content) # Some storages, such as S3Boto3Storage, don't support local file system path. # Simulate this behaviour using DictStorage. with self.subTest("Pathless storage"): request.prof_file.storage = DictStorage() DataCollector().finalise() file = DataCollector().request.prof_file self.assertIsNotNone(file) with file.storage.open(file.name) as f: content = f.read() self.assertTrue(content) self.assertGreater(len(content), 0) def test_configure_exception(self): other_profiler = cProfile.Profile() other_profiler.enable() collector = DataCollector() collector.configure() other_profiler.disable() if sys.version_info >= (3, 12): self.assertEqual(collector.local.pythonprofiler, None) else: self.assertIsNotNone(collector.local.pythonprofiler) collector.stop_python_profiler() def test_profile_file_name_with_disabled_extended_file_name(self): SilkyConfig().SILKY_PYTHON_PROFILER_EXTENDED_FILE_NAME = False request_path = 'normal/uri/' resulting_prefix = self._get_prof_file_name(request_path) self.assertEqual(resulting_prefix, '') def test_profile_file_name_with_enabled_extended_file_name(self): SilkyConfig().SILKY_PYTHON_PROFILER_EXTENDED_FILE_NAME = True request_path = 'normal/uri/' resulting_prefix = self._get_prof_file_name(request_path) self.assertEqual(resulting_prefix, 'normal_uri_') def test_profile_file_name_with_path_traversal_and_special_char(self): SilkyConfig().SILKY_PYTHON_PROFILER_EXTENDED_FILE_NAME = True request_path = 'spÉciàl/.././大/uri/@É/' resulting_prefix = self._get_prof_file_name(request_path) self.assertEqual(resulting_prefix, 'special_uri_e_') def test_profile_file_name_with_long_path(self): SilkyConfig().SILKY_PYTHON_PROFILER_EXTENDED_FILE_NAME = True request_path = 'long/path/' + 'a' * 100 resulting_prefix = self._get_prof_file_name(request_path) # the path is limited to 50 char plus the last `_` self.assertEqual(len(resulting_prefix), 51) @classmethod def _get_prof_file_name(cls, request_path: str) -> str: request = RequestMinFactory() request.path = request_path DataCollector().configure(request) DataCollector().finalise() file_path = DataCollector().request.prof_file.name filename = os.path.basename(file_path) return filename.replace(f"{request.id}.prof", "") ================================================ FILE: project/tests/test_command_garbage_collect.py ================================================ from django.core import management from django.test import TestCase from silk import models from silk.config import SilkyConfig from .factories import RequestMinFactory class TestViewClearDB(TestCase): def test_garbage_collect_command(self): SilkyConfig().SILKY_MAX_RECORDED_REQUESTS = 2 RequestMinFactory.create_batch(3) self.assertEqual(models.Request.objects.count(), 3) management.call_command("silk_request_garbage_collect") self.assertEqual(models.Request.objects.count(), 2) management.call_command("silk_request_garbage_collect", max_requests=1) self.assertEqual(models.Request.objects.count(), 1) management.call_command( "silk_request_garbage_collect", max_requests=0, verbosity=2 ) self.assertEqual(models.Request.objects.count(), 0) ================================================ FILE: project/tests/test_compat.py ================================================ import json from unittest.mock import Mock from django.test import TestCase from silk.model_factory import ResponseModelFactory DJANGO_META_CONTENT_TYPE = 'CONTENT_TYPE' HTTP_CONTENT_TYPE = 'content-type' class TestByteStringCompatForResponse(TestCase): def test_bytes_compat(self): """ Test ResponseModelFactory formats json with bytes content """ mock = Mock() mock.headers = {HTTP_CONTENT_TYPE: 'application/json;'} d = {'k': 'v'} mock.content = bytes(json.dumps(d), 'utf-8') mock.get = mock.headers.get factory = ResponseModelFactory(mock) body, content = factory.body() self.assertDictEqual(json.loads(body), d) ================================================ FILE: project/tests/test_config_auth.py ================================================ from django.contrib.auth.models import User from django.test import TestCase from django.urls import NoReverseMatch, reverse from silk.config import SilkyConfig, default_permissions from silk.middleware import silky_reverse class TestAuth(TestCase): def test_authentication(self): SilkyConfig().SILKY_AUTHENTICATION = True response = self.client.get(silky_reverse('requests')) self.assertEqual(response.status_code, 302) url = response.url try: # If we run tests within the django_silk project, a login url is available from example_app self.assertIn(reverse('login'), url) except NoReverseMatch: # Otherwise the Django default login url is used, in which case we can test for that instead self.assertIn('http://testserver/login/', url) def test_default_authorisation(self): SilkyConfig().SILKY_AUTHENTICATION = True SilkyConfig().SILKY_AUTHORISATION = True SilkyConfig().SILKY_PERMISSIONS = default_permissions username_and_password = 'bob' # bob is an imbecile and uses the same pass as his username user = User.objects.create(username=username_and_password) user.set_password(username_and_password) user.save() self.client.login(username=username_and_password, password=username_and_password) response = self.client.get(silky_reverse('requests')) self.assertEqual(response.status_code, 403) user.is_staff = True user.save() response = self.client.get(silky_reverse('requests')) self.assertEqual(response.status_code, 200) def test_custom_authorisation(self): SilkyConfig().SILKY_AUTHENTICATION = True SilkyConfig().SILKY_AUTHORISATION = True def custom_authorisation(user): return user.username.startswith('mike') SilkyConfig().SILKY_PERMISSIONS = custom_authorisation username_and_password = 'bob' # bob is an imbecile and uses the same pass as his username user = User.objects.create(username=username_and_password) user.set_password(username_and_password) user.save() self.client.login(username=username_and_password, password=username_and_password) response = self.client.get(silky_reverse('requests')) self.assertEqual(response.status_code, 403) user.username = 'mike2' user.save() response = self.client.get(silky_reverse('requests')) self.assertEqual(response.status_code, 200) ================================================ FILE: project/tests/test_config_long_urls.py ================================================ from unittest.mock import Mock from django.test import TestCase from silk.model_factory import RequestModelFactory class TestLongRequestUrl(TestCase): def test_no_long_url(self): url = '1234567890' * 19 # 190-character URL mock_request = Mock() mock_request.headers = {'content-type': 'text/plain'} mock_request.GET = {} mock_request.path = url mock_request.method = 'get' request_model = RequestModelFactory(mock_request).construct_request_model() self.assertEqual(request_model.path, url) def test_long_url(self): url = '1234567890' * 200 # 2000-character URL mock_request = Mock() mock_request.headers = {'content-type': 'text/plain'} mock_request.GET = {} mock_request.method = 'get' mock_request.path = url request_model = RequestModelFactory(mock_request).construct_request_model() self.assertEqual(request_model.path, f'{url[:94]}...{url[1907:]}') self.assertEqual(len(request_model.path), 190) ================================================ FILE: project/tests/test_config_max_body_size.py ================================================ from unittest.mock import Mock from django.test import TestCase from django.urls import reverse from silk.collector import DataCollector from silk.config import SilkyConfig from silk.model_factory import RequestModelFactory, ResponseModelFactory from silk.models import Request class TestMaxBodySizeRequest(TestCase): def test_no_max_request(self): SilkyConfig().SILKY_MAX_REQUEST_BODY_SIZE = -1 mock_request = Mock() mock_request.headers = {'content-type': 'text/plain'} mock_request.GET = {} mock_request.path = reverse('silk:requests') mock_request.method = 'get' mock_request.body = b'a' * 1000 # 1000 bytes? request_model = RequestModelFactory(mock_request).construct_request_model() self.assertTrue(request_model.raw_body) def test_max_request(self): SilkyConfig().SILKY_MAX_REQUEST_BODY_SIZE = 10 # 10kb mock_request = Mock() mock_request.headers = {'content-type': 'text/plain'} mock_request.GET = {} mock_request.method = 'get' mock_request.body = b'a' * 1024 * 100 # 100kb mock_request.path = reverse('silk:requests') request_model = RequestModelFactory(mock_request).construct_request_model() self.assertFalse(request_model.raw_body) class TestMaxBodySizeResponse(TestCase): def setUp(self): DataCollector().request = Request.objects.create() def test_no_max_response(self): SilkyConfig().SILKY_MAX_RESPONSE_BODY_SIZE = -1 mock_response = Mock() mock_response.headers = {'content-type': 'text/plain'} mock_response.content = b'a' * 1000 # 1000 bytes? mock_response.status_code = 200 mock_response.get = mock_response.headers.get response_model = ResponseModelFactory(mock_response).construct_response_model() self.assertTrue(response_model.raw_body) def test_max_response(self): SilkyConfig().SILKY_MAX_RESPONSE_BODY_SIZE = 10 # 10kb mock_response = Mock() mock_response.headers = {'content-type': 'text/plain'} mock_response.content = b'a' * 1024 * 100 # 100kb mock_response.status_code = 200 mock_response.get = mock_response.headers.get response_model = ResponseModelFactory(mock_response).construct_response_model() self.assertFalse(response_model.raw_body) ================================================ FILE: project/tests/test_config_meta.py ================================================ from unittest.mock import NonCallableMock from django.test import TestCase from silk.collector import DataCollector from silk.config import SilkyConfig from silk.middleware import SilkyMiddleware from silk.models import Request from .util import delete_all_models def fake_get_response(): def fake_response(): return 'hello world' return fake_response class TestConfigMeta(TestCase): def _mock_response(self): response = NonCallableMock() response.headers = {} response.status_code = 200 response.queries = [] response.get = response.headers.get response.content = '' return response def _execute_request(self): delete_all_models(Request) DataCollector().configure(Request.objects.create()) response = self._mock_response() SilkyMiddleware(fake_get_response)._process_response('', response) self.assertTrue(response.status_code == 200) objs = Request.objects.all() self.assertEqual(objs.count(), 1) r = objs[0] return r def test_enabled(self): SilkyConfig().SILKY_META = True r = self._execute_request() self.assertTrue(r.meta_time is not None or r.meta_num_queries is not None or r.meta_time_spent_queries is not None) def test_disabled(self): SilkyConfig().SILKY_META = False r = self._execute_request() self.assertFalse(r.meta_time) ================================================ FILE: project/tests/test_db.py ================================================ """ Test profiling of DB queries without mocking, to catch possible incompatibility """ from django.shortcuts import reverse from django.test import Client, TestCase from silk.collector import DataCollector from silk.config import SilkyConfig from silk.models import Request from silk.profiling.profiler import silk_profile from .factories import BlindFactory class TestDbQueries(TestCase): @classmethod def setUpClass(cls): super().setUpClass() BlindFactory.create_batch(size=5) SilkyConfig().SILKY_META = False def test_profile_request_to_db(self): DataCollector().configure(Request(reverse('example_app:index'))) with silk_profile(name='test_profile'): resp = self.client.get(reverse('example_app:index')) DataCollector().profiles.values() assert len(resp.context['blinds']) == 5 def test_profile_request_to_db_with_constraints(self): DataCollector().configure(Request(reverse('example_app:create'))) resp = self.client.post(reverse('example_app:create'), {'name': 'Foo'}) self.assertEqual(resp.status_code, 302) class TestAnalyzeQueries(TestCase): @classmethod def setUpClass(cls): super().setUpClass() BlindFactory.create_batch(size=5) SilkyConfig().SILKY_META = False SilkyConfig().SILKY_ANALYZE_QUERIES = True @classmethod def tearDownClass(cls): super().tearDownClass() SilkyConfig().SILKLY_ANALYZE_QUERIES = False def test_analyze_queries(self): DataCollector().configure(Request(reverse('example_app:index'))) client = Client() with silk_profile(name='test_profile'): resp = client.get(reverse('example_app:index')) DataCollector().profiles.values() assert len(resp.context['blinds']) == 5 class TestAnalyzeQueriesExplainParams(TestAnalyzeQueries): @classmethod def setUpClass(cls): super().setUpClass() SilkyConfig().SILKY_EXPLAIN_FLAGS = {'verbose': True} @classmethod def tearDownClass(cls): super().tearDownClass() SilkyConfig().SILKY_EXPLAIN_FLAGS = None ================================================ FILE: project/tests/test_dynamic_profiling.py ================================================ from unittest.mock import patch from django.test import TestCase import silk from silk.profiling.dynamic import ( _get_module, _get_parent_module, profile_function_or_method, ) from .test_lib.assertion import dict_contains from .util import mock_data_collector class TestGetModule(TestCase): """test for _get_module""" def test_singular(self): module = _get_module('silk') self.assertEqual(module.__class__.__name__, 'module') self.assertEqual('silk', module.__name__) self.assertTrue(hasattr(module, 'models')) def test_dot(self): module = _get_module('silk.models') self.assertEqual(module.__class__.__name__, 'module') self.assertEqual('silk.models', module.__name__) self.assertTrue(hasattr(module, 'SQLQuery')) class TestGetParentModule(TestCase): """test for silk.tools._get_parent_module""" def test_singular(self): parent = _get_parent_module(silk) self.assertIsInstance(parent, dict) def test_dot(self): import silk.utils parent = _get_parent_module(silk.utils) self.assertEqual(parent, silk) class MyClass: def foo(self): pass def foo(): pass def source_file_name(): file_name = __file__ if file_name[-1] == 'c': file_name = file_name[:-1] return file_name class TestProfileFunction(TestCase): def test_method_as_str(self): # noinspection PyShadowingNames def foo(_): pass # noinspection PyUnresolvedReferences with patch.object(MyClass, 'foo', foo): profile_function_or_method('tests.test_dynamic_profiling', 'MyClass.foo', 'test') dc = mock_data_collector() with patch('silk.profiling.profiler.DataCollector', return_value=dc) as mock_DataCollector: MyClass().foo() self.assertEqual(mock_DataCollector.return_value.register_profile.call_count, 1) call_args = mock_DataCollector.return_value.register_profile.call_args[0][0] self.assertTrue(dict_contains({ 'func_name': foo.__name__, 'dynamic': True, 'file_path': source_file_name(), 'name': 'test', 'line_num': foo.__code__.co_firstlineno }, call_args)) def test_func_as_str(self): name = foo.__name__ line_num = foo.__code__.co_firstlineno profile_function_or_method('tests.test_dynamic_profiling', 'foo', 'test') dc = mock_data_collector() with patch('silk.profiling.profiler.DataCollector', return_value=dc) as mock_DataCollector: foo() self.assertEqual(mock_DataCollector.return_value.register_profile.call_count, 1) call_args = mock_DataCollector.return_value.register_profile.call_args[0][0] self.assertTrue(dict_contains({ 'func_name': name, 'dynamic': True, 'file_path': source_file_name(), 'name': 'test', 'line_num': line_num }, call_args)) ================================================ FILE: project/tests/test_encoding.py ================================================ import json from unittest.mock import Mock from django.test import TestCase from silk.model_factory import RequestModelFactory, ResponseModelFactory HTTP_CONTENT_TYPE = 'content-type' class TestEncodingForRequests(TestCase): """ Check that the RequestModelFactory deals with encodings correctly via charset """ def test_utf_plain(self): mock_request = Mock() mock_request.headers = {HTTP_CONTENT_TYPE: 'text/plain; charset=UTF-8'} mock_request.body = '语' mock_request.get = mock_request.headers.get factory = RequestModelFactory(mock_request) body, raw_body = factory.body() self.assertFalse(body) self.assertEqual(raw_body, mock_request.body) def test_plain(self): mock_request = Mock() mock_request.headers = {HTTP_CONTENT_TYPE: 'text/plain'} mock_request.body = 'sdfsdf' mock_request.get = mock_request.headers.get factory = RequestModelFactory(mock_request) body, raw_body = factory.body() self.assertFalse(body) self.assertEqual(raw_body, mock_request.body) def test_utf_json_not_encoded(self): mock_request = Mock() mock_request.headers = {HTTP_CONTENT_TYPE: 'application/json; charset=UTF-8'} d = {'x': '语'} mock_request.body = json.dumps(d) mock_request.get = mock_request.headers.get factory = RequestModelFactory(mock_request) body, raw_body = factory.body() self.assertDictEqual(json.loads(body), d) self.assertEqual(raw_body, mock_request.body) def test_utf_json_encoded(self): mock_request = Mock() mock_request.headers = {HTTP_CONTENT_TYPE: 'application/json; charset=UTF-8'} d = {'x': '语'} mock_request.body = json.dumps(d).encode('UTF-8') mock_request.get = mock_request.headers.get factory = RequestModelFactory(mock_request) body, raw_body = factory.body() self.assertDictEqual(json.loads(body), d) self.assertEqual(raw_body, mock_request.body.decode('UTF-8')) def test_utf_json_encoded_no_charset(self): """default to UTF-8""" mock_request = Mock() mock_request.headers = {HTTP_CONTENT_TYPE: 'application/json'} d = {'x': '语'} mock_request.body = json.dumps(d).encode('UTF-8') mock_request.get = mock_request.headers.get factory = RequestModelFactory(mock_request) body, raw_body = factory.body() self.assertDictEqual(json.loads(body), d) self.assertEqual(raw_body, mock_request.body.decode('UTF-8')) def test_invalid_encoding_json(self): mock_request = Mock() mock_request.headers = {HTTP_CONTENT_TYPE: 'application/json; charset=asdas-8'} d = {'x': '语'} mock_request.body = json.dumps(d).encode('UTF-8') mock_request.get = mock_request.headers.get factory = RequestModelFactory(mock_request) body, raw_body = factory.body() self.assertDictEqual(json.loads(body), d) self.assertEqual(raw_body, raw_body) class TestEncodingForResponse(TestCase): """ Check that the ResponseModelFactory deals with encodings correctly via charset """ def test_utf_plain(self): mock = Mock() mock.headers = {HTTP_CONTENT_TYPE: 'text/plain; charset=UTF-8'} mock.content = '语' mock.get = mock.headers.get factory = ResponseModelFactory(mock) body, content = factory.body() self.assertFalse(body) self.assertEqual(content, mock.content) def test_plain(self): mock = Mock() mock.headers = {HTTP_CONTENT_TYPE: 'text/plain'} mock.content = 'sdfsdf' mock.get = mock.headers.get factory = ResponseModelFactory(mock) body, content = factory.body() self.assertFalse(body) self.assertEqual(content, mock.content) def test_utf_json_not_encoded(self): mock = Mock() mock.headers = {HTTP_CONTENT_TYPE: 'application/json; charset=UTF-8'} d = {'x': '语'} mock.content = json.dumps(d) mock.get = mock.headers.get factory = ResponseModelFactory(mock) body, content = factory.body() self.assertDictEqual(json.loads(body), d) self.assertEqual(content, mock.content) def test_utf_json_encoded(self): mock = Mock() mock.headers = {HTTP_CONTENT_TYPE: 'application/json; charset=UTF-8'} d = {'x': '语'} mock.content = json.dumps(d) mock.get = mock.headers.get factory = ResponseModelFactory(mock) body, content = factory.body() self.assertDictEqual(json.loads(body), d) self.assertEqual(content, mock.content) def test_utf_json_encoded_no_charset(self): """default to UTF-8""" mock = Mock() mock.headers = {HTTP_CONTENT_TYPE: 'application/json'} d = {'x': '语'} mock.content = json.dumps(d) mock.get = mock.headers.get factory = ResponseModelFactory(mock) body, content = factory.body() self.assertDictEqual(json.loads(body), d) self.assertEqual(content, mock.content) def test_invalid_encoding_json(self): mock = Mock() mock.headers = {HTTP_CONTENT_TYPE: 'application/json; charset=asdas-8'} d = {'x': '语'} mock.content = json.dumps(d) mock.get = mock.headers.get factory = ResponseModelFactory(mock) body, content = factory.body() self.assertDictEqual(json.loads(body), d) self.assertEqual(mock.content, content) ================================================ FILE: project/tests/test_end_points.py ================================================ import random from django.db.models import Count, F from django.test import TestCase from django.urls import reverse from silk import models from silk.config import SilkyConfig from silk.middleware import silky_reverse from .test_lib.mock_suite import MockSuite class TestEndPoints(TestCase): """ Hit all the endpoints to check everything actually renders/no error 500s etc. Each test will ensure that an object with something to display is chosen to be rendered e.g. a request/profile that has queries """ @classmethod def setUpClass(cls): super().setUpClass() # We're not testing auth here. SilkyConfig().SILKY_AUTHORISATION = False SilkyConfig().SILKY_AUTHENTICATION = False mock_suite = MockSuite() for _ in range(0, 100): mock_suite.mock_request() def test_summary(self): response = self.client.get(silky_reverse('summary')) self.assertTrue(response.status_code == 200) def test_requests(self): response = self.client.get(silky_reverse('requests')) self.assertTrue(response.status_code == 200) def test_request_detail_on_get_request(self): request_id = random.choice( models.Request.objects.filter(method='GET').values_list('id', flat=True), ) response = self.client.get(silky_reverse('request_detail', kwargs={ 'request_id': request_id })) self.assertEqual(response.status_code, 200) def test_request_detail_on_post_request(self): request_id = random.choice( models.Request.objects.filter(method='POST').values_list('id', flat=True), ) response = self.client.get(silky_reverse('request_detail', kwargs={ 'request_id': request_id })) self.assertEqual(response.status_code, 200) def test_request_sql(self): request_query_data = random.choice( models.SQLQuery.objects .values('request_id') .filter(request_id__isnull=False) ) request_id = request_query_data['request_id'] base_url = silky_reverse('request_sql', kwargs={'request_id': request_id}) response = self.client.get(base_url) self.assertTrue(response.status_code == 200) # Test with valid page size response = self.client.get(base_url + "?per_page=100") self.assertTrue(response.status_code == 200) # Test with invalid page size response = self.client.get(base_url + "?per_page=notanumber") self.assertTrue(response.status_code == 200) def test_request_sql_detail(self): kwargs = random.choice( models.SQLQuery.objects .annotate(sql_id=F('id')) .values('sql_id', 'request_id') .filter(request_id__isnull=False) ) response = self.client.get(silky_reverse('request_sql_detail', kwargs=kwargs)) self.assertTrue(response.status_code == 200) def test_raw(self): request_query_data = random.choice( models.Request.objects .values('id') .filter(body__isnull=False) ) request_id = request_query_data['id'] url = reverse('silk:raw', kwargs={ 'request_id': request_id }) + '?typ=request&subtyp=processed' response = self.client.get(url) code = response.status_code self.assertTrue(code == 200) def test_request_profiling(self): request_id = random.choice( models.Profile.objects .values('request_id') .filter(request_id__isnull=False) ) response = self.client.get(silky_reverse('request_profiling', kwargs=request_id)) self.assertTrue(response.status_code == 200) def test_request_profile_detail(self): kwargs = random.choice( models.Profile.objects .annotate(profile_id=F('id')) .values('profile_id', 'request_id') .filter(request_id__isnull=False) ) response = self.client.get(silky_reverse('request_profile_detail', kwargs=kwargs)) self.assertTrue(response.status_code == 200) def test_request_and_profile_sql(self): kwargs = random.choice( models.Profile.objects .annotate(num=Count('queries'), profile_id=F('id')) .values('profile_id', 'request_id') .filter(request_id__isnull=False, num__gt=0) ) response = self.client.get(silky_reverse('request_and_profile_sql', kwargs=kwargs)) self.assertTrue(response.status_code == 200) def test_request_and_profile_sql_detail(self): random_profile = random.choice( models.Profile.objects .annotate(num=Count('queries'), profile_id=F('id')) .values('profile_id', 'request_id') .filter(request_id__isnull=False, num__gt=0) ) random_sql_query = random.choice( models.SQLQuery.objects .annotate(sql_id=F('id')) .values('sql_id') .filter(profiles__id=random_profile['profile_id']) ) kwargs = {} kwargs.update(random_profile) kwargs.update(random_sql_query) response = self.client.get(silky_reverse('request_and_profile_sql_detail', kwargs=kwargs)) self.assertTrue(response.status_code == 200) def test_profile_detail(self): profile_query_data = random.choice(models.Profile.objects.values('id')) profile_id = profile_query_data['id'] response = self.client.get(silky_reverse('profile_detail', kwargs={ 'profile_id': profile_id })) self.assertTrue(response.status_code == 200) def test_profile_sql(self): profile_query_data = random.choice( models.Profile.objects .annotate(num=Count('queries')) .values('id') .filter(num__gt=0) ) profile_id = profile_query_data['id'] response = self.client.get(silky_reverse('profile_sql', kwargs={'profile_id': profile_id})) self.assertTrue(response.status_code == 200) def test_profile_sql_detail(self): profile_query_data = random.choice( models.Profile.objects .annotate(num=Count('queries')) .values('id') .filter(num__gt=0) ) profile_id = profile_query_data['id'] sql_id = random.choice(models.SQLQuery.objects.filter(profiles=profile_id)).pk response = self.client.get(silky_reverse('profile_sql_detail', kwargs={'profile_id': profile_id, 'sql_id': sql_id})) self.assertTrue(response.status_code == 200) def test_profiling(self): response = self.client.get(silky_reverse('profiling')) self.assertTrue(response.status_code == 200) ================================================ FILE: project/tests/test_execute_sql.py ================================================ from unittest.mock import Mock, NonCallableMagicMock, NonCallableMock, patch from django.test import TestCase from django.utils.encoding import force_str from silk.collector import DataCollector from silk.models import Request, SQLQuery from silk.sql import execute_sql from .util import delete_all_models _simple_mock_query_sql = 'SELECT * FROM table_name WHERE column1 = %s' _simple_mock_query_params = ('asdf',) _non_unicode_binary_mock_query_params = (b'\x0a\x00\x00\xff',) _unicode_binary_mock_query_params = ('🫠'.encode(),) def mock_sql(mock_query_params): mock_sql_query = Mock(spec_set=['_execute_sql', 'query', 'as_sql', 'connection']) mock_sql_query._execute_sql = Mock() mock_sql_query.query = NonCallableMock(spec_set=['model']) mock_sql_query.query.model = Mock() mock_sql_query.as_sql = Mock(return_value=(_simple_mock_query_sql, mock_query_params)) mock_sql_query.connection = NonCallableMock( spec_set=['cursor', 'features', 'ops'], cursor=Mock( spec_set=['__call__'], return_value=NonCallableMagicMock(spec_set=['__enter__', '__exit__', 'execute']) ), features=NonCallableMock( spec_set=['supports_explaining_query_execution'], supports_explaining_query_execution=True ), ops=NonCallableMock(spec_set=['explain_query_prefix'], explain_query_prefix=Mock(return_value='')), ) return mock_sql_query, mock_query_params class BaseTestCase(TestCase): def tearDown(self): DataCollector().stop_python_profiler() def call_execute_sql(self, request, mock_query_params): DataCollector().configure(request=request) delete_all_models(SQLQuery) self.query_string = _simple_mock_query_sql self.mock_sql, self.query_params = mock_sql(mock_query_params) self.kwargs = { 'one': 1, 'two': 2 } self.args = [1, 2] execute_sql(self.mock_sql, *self.args, **self.kwargs) class TestCallNoRequest(BaseTestCase): def setUp(self): super().setUp() self.call_execute_sql(None, _simple_mock_query_params) def test_called(self): self.mock_sql._execute_sql.assert_called_once_with(*self.args, **self.kwargs) def test_count(self): self.assertEqual(0, len(DataCollector().queries)) class TestCallRequest(BaseTestCase): def test_query_simple(self): self.call_execute_sql(Request(), _simple_mock_query_params) self.mock_sql._execute_sql.assert_called_once_with(*self.args, **self.kwargs) self.assertEqual(1, len(DataCollector().queries)) query = list(DataCollector().queries.values())[0] expected = self.query_string % tuple(force_str(param) for param in self.query_params) self.assertEqual(query['query'], expected) def test_query_unicode(self): self.call_execute_sql(Request(), _unicode_binary_mock_query_params) self.mock_sql._execute_sql.assert_called_once_with(*self.args, **self.kwargs) self.assertEqual(1, len(DataCollector().queries)) query = list(DataCollector().queries.values())[0] expected = self.query_string % tuple(force_str(param) for param in self.query_params) self.assertEqual(query['query'], expected) def test_query_non_unicode(self): self.call_execute_sql(Request(), _non_unicode_binary_mock_query_params) self.mock_sql._execute_sql.assert_called_once_with(*self.args, **self.kwargs) self.assertEqual(0, len(DataCollector().queries)) class TestCallSilky(BaseTestCase): def test_no_effect(self): DataCollector().configure() sql, _ = mock_sql(_simple_mock_query_params) sql.query.model = NonCallableMagicMock(spec_set=['__module__']) sql.query.model.__module__ = 'silk.models' # No SQLQuery models should be created for silk requests for obvious reasons with patch('silk.sql.DataCollector', return_value=Mock()) as mock_DataCollector: execute_sql(sql) self.assertFalse(mock_DataCollector().register_query.call_count) class TestCollectorInteraction(BaseTestCase): def _query(self): try: query = list(DataCollector().queries.values())[0] except IndexError: self.fail('No queries created') return query def test_request(self): DataCollector().configure(request=Request.objects.create(path='/path/to/somewhere')) sql, _ = mock_sql(_simple_mock_query_params) execute_sql(sql) query = self._query() self.assertEqual(query['request'], DataCollector().request) def test_registration(self): DataCollector().configure(request=Request.objects.create(path='/path/to/somewhere')) sql, _ = mock_sql(_simple_mock_query_params) execute_sql(sql) query = self._query() self.assertIn(query, DataCollector().queries.values()) def test_explain_simple(self): DataCollector().configure(request=Request.objects.create(path='/path/to/somewhere')) sql, params = mock_sql(_simple_mock_query_params) prefix = "EXPLAIN" mock_cursor = sql.connection.cursor.return_value.__enter__.return_value sql.connection.ops.explain_query_prefix.return_value = prefix execute_sql(sql) self.assertNotIn(prefix, params) mock_cursor.execute.assert_called_once_with(f"{prefix} {_simple_mock_query_sql}", params) def test_explain_unicode(self): DataCollector().configure(request=Request.objects.create(path='/path/to/somewhere')) sql, params = mock_sql(_unicode_binary_mock_query_params) prefix = "EXPLAIN" mock_cursor = sql.connection.cursor.return_value.__enter__.return_value sql.connection.ops.explain_query_prefix.return_value = prefix execute_sql(sql) self.assertNotIn(prefix, params) mock_cursor.execute.assert_called_once_with(f"{prefix} {_simple_mock_query_sql}", params) def test_explain_non_unicode(self): DataCollector().configure(request=Request.objects.create(path='/path/to/somewhere')) sql, params = mock_sql(_non_unicode_binary_mock_query_params) prefix = "EXPLAIN" mock_cursor = sql.connection.cursor.return_value.__enter__.return_value sql.connection.ops.explain_query_prefix.return_value = prefix execute_sql(sql) self.assertNotIn(prefix, params) self.assertFalse(mock_cursor.execute.called) ================================================ FILE: project/tests/test_filters.py ================================================ import calendar import random from datetime import datetime, timedelta, timezone from itertools import groupby from math import floor from django.test import TestCase from django.utils import timezone as django_timezone from silk import models from silk.request_filters import ( AfterDateFilter, BeforeDateFilter, FunctionNameFilter, MethodFilter, NameFilter, NumQueriesFilter, OverallTimeFilter, PathFilter, SecondsFilter, StatusCodeFilter, TimeSpentOnQueriesFilter, ViewNameFilter, ) from .test_lib.mock_suite import MockSuite from .util import delete_all_models mock_suite = MockSuite() class TestRequestFilters(TestCase): @classmethod def setUpClass(cls): super().setUpClass() def _time_stamp(self, dt): return calendar.timegm(dt.utctimetuple()) def test_seconds_filter(self): requests = [mock_suite.mock_request() for _ in range(0, 10)] n = 0 for r in requests: r.start_time = django_timezone.now() - timedelta(seconds=n) r.save() n += 1 requests = models.Request.objects.filter(SecondsFilter(5)) for r in requests: dt = r.start_time seconds = self._time_stamp(django_timezone.now()) - self._time_stamp(dt) self.assertTrue(seconds < 6) # 6 to give a bit of leeway in case takes too long def test_view_name_filter(self): requests = [mock_suite.mock_request() for _ in range(0, 10)] r = random.choice(requests) view_name = r.view_name requests = models.Request.objects.filter(ViewNameFilter(view_name)) for r in requests: self.assertTrue(r.view_name == view_name) def test_path_filter(self): requests = [mock_suite.mock_request() for _ in range(0, 10)] r = random.choice(requests) path = r.path requests = models.Request.objects.filter(PathFilter(path)) for r in requests: self.assertTrue(r.path == path) def test_num_queries_filter(self): requests = [mock_suite.mock_request() for _ in range(0, 10)] counts = sorted(x.queries.count() for x in requests) c = counts[int(floor(len(counts) / 2))] num_queries_filter = NumQueriesFilter(c) query_set = models.Request.objects.all() query_set = num_queries_filter.contribute_to_query_set(query_set) filtered = query_set.filter(num_queries_filter) for f in filtered: self.assertGreaterEqual(f.queries.count(), c) def test_time_spent_queries_filter(self): requests = [mock_suite.mock_request() for _ in range(0, 10)] time_taken = sorted(sum(q.time_taken for q in x.queries.all()) for x in requests) c = time_taken[int(floor(len(time_taken) / 2))] time_taken_filter = TimeSpentOnQueriesFilter(c) query_set = models.Request.objects.all() query_set = time_taken_filter.contribute_to_query_set(query_set) filtered = query_set.filter(time_taken_filter) for f in filtered: self.assertGreaterEqual(sum(q.time_taken for q in f.queries.all()), c) def test_time_spent_filter(self): requests = [mock_suite.mock_request() for _ in range(0, 10)] time_taken = sorted(x.time_taken for x in requests) c = time_taken[int(floor(len(time_taken) / 2))] time_taken_filter = OverallTimeFilter(c) query_set = models.Request.objects.all() query_set = time_taken_filter.contribute_to_query_set(query_set) filtered = query_set.filter(time_taken_filter) for f in filtered: self.assertGreaterEqual(round(f.time_taken), round(c)) def test_status_code_filter(self): requests = [mock_suite.mock_request() for _ in range(0, 50)] requests = sorted(requests, key=lambda x: x.response.status_code) by_status_code = groupby(requests, key=lambda x: x.response.status_code) for status_code, expected in by_status_code: status_code_filter = StatusCodeFilter(status_code) query_set = models.Request.objects.all() query_set = status_code_filter.contribute_to_query_set(query_set) filtered = query_set.filter(status_code_filter) self.assertEqual(len(list(expected)), filtered.count()) def test_method_filter(self): requests = [mock_suite.mock_request() for _ in range(0, 50)] requests = sorted(requests, key=lambda x: x.method) by_method = groupby(requests, key=lambda x: x.method) for method, expected in by_method: method_filter = MethodFilter(method) query_set = models.Request.objects.all() query_set = method_filter.contribute_to_query_set(query_set) filtered = query_set.filter(method_filter) self.assertEqual(len(list(expected)), filtered.count()) class TestRequestAfterDateFilter(TestCase): def assertFilter(self, dt, f): requests = models.Request.objects.filter(f) for r in requests: self.assertTrue(r.start_time > dt) @classmethod def setUpClass(cls): super().setUpClass() cls.requests = [mock_suite.mock_request() for _ in range(0, 10)] def test_after_date_filter(self): r = random.choice(self.requests) dt = r.start_time f = AfterDateFilter(dt) self.assertFilter(dt, f) def test_after_date_filter_str(self): r = random.choice(self.requests) dt = r.start_time fmt = AfterDateFilter.fmt dt_str = dt.strftime(fmt) date_filter = AfterDateFilter f = date_filter(dt_str) new_dt = datetime.strptime(dt_str, fmt) new_dt = django_timezone.make_aware(new_dt, timezone.utc) self.assertFilter(new_dt, f) class TestRequestBeforeDateFilter(TestCase): def assertFilter(self, dt, f): requests = models.Request.objects.filter(f) for r in requests: self.assertTrue(r.start_time < dt) @classmethod def setUpClass(cls): super().setUpClass() cls.requests = [mock_suite.mock_request() for _ in range(0, 10)] def test_before_date_filter(self): r = random.choice(self.requests) dt = r.start_time f = BeforeDateFilter(dt) self.assertFilter(dt, f) def test_before_date_filter_str(self): r = random.choice(self.requests) dt = r.start_time fmt = BeforeDateFilter.fmt dt_str = dt.strftime(fmt) date_filter = BeforeDateFilter f = date_filter(dt_str) new_dt = datetime.strptime(dt_str, fmt) new_dt = django_timezone.make_aware(new_dt, timezone.utc) self.assertFilter(new_dt, f) class TestProfileFilters(TestCase): def setUp(self): delete_all_models(models.Profile) def test_name_filter(self): profiles = mock_suite.mock_profiles(n=10) p = random.choice(profiles) name = p.name requests = models.Profile.objects.filter(NameFilter(name)) for p in requests: self.assertTrue(p.name == name) def test_function_name_filter(self): profiles = mock_suite.mock_profiles(n=10) p = random.choice(profiles) func_name = p.func_name requests = models.Profile.objects.filter(FunctionNameFilter(func_name)) for p in requests: self.assertTrue(p.func_name == func_name) def test_num_queries_filter(self): profiles = mock_suite.mock_profiles(n=10) counts = sorted(x.queries.count() for x in profiles) c = counts[int(floor(len(counts) / 2))] num_queries_filter = NumQueriesFilter(c) query_set = models.Profile.objects.all() query_set = num_queries_filter.contribute_to_query_set(query_set) filtered = query_set.filter(num_queries_filter) for f in filtered: self.assertGreaterEqual(f.queries.count(), c) def test_time_spent_queries_filter(self): profiles = mock_suite.mock_profiles(n=10) time_taken = sorted(sum(q.time_taken for q in x.queries.all()) for x in profiles) c = time_taken[int(floor(len(time_taken) / 2))] time_taken_filter = TimeSpentOnQueriesFilter(c) query_set = models.Profile.objects.all() query_set = time_taken_filter.contribute_to_query_set(query_set) filtered = query_set.filter(time_taken_filter) for f in filtered: self.assertGreaterEqual(sum(q.time_taken for q in f.queries.all()), c) def test_time_spent_filter(self): profiles = [mock_suite.mock_request() for _ in range(0, 10)] time_taken = sorted(x.time_taken for x in profiles) c = time_taken[int(floor(len(time_taken) / 2))] time_taken_filter = OverallTimeFilter(c) query_set = models.Profile.objects.all() query_set = time_taken_filter.contribute_to_query_set(query_set) filtered = query_set.filter(time_taken_filter) for f in filtered: self.assertGreaterEqual(f.time_taken, c) ================================================ FILE: project/tests/test_lib/__init__.py ================================================ __author__ = 'mtford' ================================================ FILE: project/tests/test_lib/assertion.py ================================================ def dict_contains(child_dict, parent_dict): for key, value in child_dict.items(): if key not in parent_dict: return False if parent_dict[key] != value: return False return True ================================================ FILE: project/tests/test_lib/mock_suite.py ================================================ import json import os import random import traceback from datetime import timedelta from django.core import management from django.utils import timezone from silk import models from silk.models import Profile, SQLQuery class MockSuite: """ Provides some fake data to play around with. Also useful for testing """ methods = ['GET', 'POST', 'PUT', 'PATCH', 'HEAD', 'OPTIONS'] path_components = ['path', 'to', 'somewhere', 'around', 'here', 'bobs', 'your', 'uncle'] status_codes = [200, 201, 300, 301, 302, 403, 404, 500] profile_names = ['slow_bit_of_code', 'terrible_dependency', 'what_on_earth_is_this_code_doing'] file_path = [os.path.realpath(__file__)] func_names = ['', '', '', 'foo', 'bar'] view_names = ['app:blah', 'index', 'root', 'xxx:xyx'] sql_queries = [''' SELECT Book.title AS Title, COUNT(*) AS Authors FROM Book JOIN Book_author ON Book.isbn = Book_author.isbn GROUP BY Book.title; ''', ''' SELECT * FROM table ''', ''' SELECT * FROM Book WHERE price > 100.00 ORDER BY title; ''', ''' SELECT title, COUNT(*) AS Authors FROM Book NATURAL JOIN Book_author GROUP BY title; ''', ''' SELECT A.Col1, A.Col2, B.Col1,B.Col2 FROM (SELECT RealTableZ.Col1, RealTableY.Col2, RealTableY.ID AS ID FROM RealTableZ LEFT OUTER JOIN RealTableY ON RealTableZ.ForeignKeyY=RealTableY.ID WHERE RealTableY.Col11>14 ) AS B INNER JOIN A ON A.ForeignKeyY=B.ID '''] response_content_types = ['text/html', 'application/json', 'text/css'] response_content = { 'text/html': [''], 'text/css': ['#blah {font-weight: bold}'], 'application/json': ['[1, 2, 3]'] } request_content_types = ['application/json'] request_content = { 'application/json': ['{"blah": 5}'] } def _random_method(self): return random.choice(self.methods) def _random_path(self): num_components = random.randint(1, 5) return '/' + '/'.join(random.sample(self.path_components, num_components)) + '/' def _random_query(self): return random.choice(self.sql_queries) def mock_sql_queries(self, request=None, profile=None, n=1, as_dict=False): start_time, end_time = self._random_time() queries = [] for _ in range(0, n): tb = ''.join(reversed(traceback.format_stack())) d = { 'query': self._random_query(), 'start_time': start_time, 'end_time': end_time, 'request': request, 'traceback': tb } if as_dict: queries.append(d) else: query = SQLQuery.objects.create(**d) queries.append(query) if profile: if as_dict: for q in queries: profile['queries'].append(q) else: profile.queries.set(queries) return queries def mock_profile(self, request=None): start_time, end_time = self._random_time() dynamic = random.choice([True, False]) profile = Profile.objects.create(start_time=start_time, end_time=end_time, request=request, name=random.choice(self.profile_names), file_path=random.choice(self.file_path), line_num=3, func_name=random.choice(self.func_names), dynamic=dynamic, end_line_num=6 if dynamic else None, exception_raised=random.choice([True, False]) ) self.mock_sql_queries(profile=profile, n=random.randint(0, 10)) return profile def mock_profiles(self, request=None, n=1): profiles = [] for _ in range(0, n): profile = self.mock_profile(request) profiles.append(profile) return profiles def _random_time(self): start_time = timezone.now() duration = timedelta(milliseconds=random.randint(0, 3000)) end_time = start_time + duration return start_time, end_time def mock_request(self): start_time, end_time = self._random_time() num_sql_queries = random.randint(0, 20) request_content_type = random.choice(self.request_content_types) request_body = random.choice(self.request_content[request_content_type]) time_taken = end_time - start_time time_taken = time_taken.total_seconds() request = models.Request.objects.create(method=self._random_method(), path=self._random_path(), num_sql_queries=num_sql_queries, start_time=start_time, end_time=end_time, view_name=random.choice(self.view_names), time_taken=time_taken, encoded_headers=json.dumps({'content-type': request_content_type}), body=request_body) response_content_type = random.choice(self.response_content_types) response_body = random.choice(self.response_content[response_content_type]) models.Response.objects.create(request=request, status_code=random.choice(self.status_codes), encoded_headers=json.dumps({'content-type': response_content_type}), body=response_body) self.mock_sql_queries(request=request, n=num_sql_queries) self.mock_profiles(request, random.randint(0, 2)) return request if __name__ == '__main__': management.call_command('flush', interactive=False) requests = [MockSuite().mock_request() for _ in range(0, 100)] ================================================ FILE: project/tests/test_models.py ================================================ import datetime import uuid from django.core.management import call_command from django.test import TestCase, override_settings from freezegun import freeze_time from silk import models from silk.config import SilkyConfig from silk.storage import ProfilerResultStorage from .factories import RequestMinFactory, ResponseFactory, SQLQueryFactory # TODO test atomicity # http://stackoverflow.com/questions/13397038/uuid-max-character-length # UUID_MAX_LENGTH = 36 # TODO move to separate file test and collection it self class CaseInsensitiveDictionaryTest: pass class RequestTest(TestCase): def setUp(self): self.obj = RequestMinFactory.create() self.max_percent = SilkyConfig().SILKY_MAX_RECORDED_REQUESTS_CHECK_PERCENT self.max_requests = SilkyConfig().SILKY_MAX_RECORDED_REQUESTS def tearDown(self): SilkyConfig().SILKY_MAX_RECORDED_REQUESTS_CHECK_PERCENT = self.max_percent SilkyConfig().SILKY_MAX_RECORDED_REQUESTS = self.max_requests def test_uuid_is_primary_key(self): self.assertIsInstance(self.obj.id, uuid.UUID) @freeze_time('2016-01-01 12:00:00') def test_start_time_field_default(self): obj = RequestMinFactory.create() self.assertEqual(obj.start_time, datetime.datetime(2016, 1, 1, 12, 0, 0, tzinfo=datetime.timezone.utc)) def test_total_meta_time_if_have_no_meta_and_queries_time(self): self.assertEqual(self.obj.total_meta_time, 0) def test_total_meta_time_if_have_meta_time_spent_queries(self): obj = RequestMinFactory.create(meta_time_spent_queries=10.5) self.assertEqual(obj.total_meta_time, 10.5) def test_total_meta_time_if_meta_time(self): obj = RequestMinFactory.create(meta_time=3.3) self.assertEqual(obj.total_meta_time, 3.3) def test_total_meta_if_self_have_it_meta_and_queries_time(self): obj = RequestMinFactory.create(meta_time=3.3, meta_time_spent_queries=10.5) self.assertEqual(obj.total_meta_time, 13.8) def test_time_spent_on_sql_queries_if_has_no_related_SQLQueries(self): self.assertEqual(self.obj.time_spent_on_sql_queries, 0) def test_time_spent_on_sql_queries_if_has_related_SQLQueries_with_no_time_taken(self): query = SQLQueryFactory() self.obj.queries.add(query) self.assertEqual(query.time_taken, None) # No exception should be raised, and the result should be 0.0 self.assertEqual(self.obj.time_spent_on_sql_queries, 0.0) def test_time_spent_on_sql_queries_if_has_related_SQLQueries_and_time_taken(self): query1 = SQLQueryFactory(time_taken=3.5) query2 = SQLQueryFactory(time_taken=1.5) RequestMinFactory().queries.add(query1, query2) self.assertEqual(self.obj.time_spent_on_sql_queries, 0) def test_headers_if_has_no_encoded_headers(self): self.assertIsInstance(self.obj.headers, models.CaseInsensitiveDictionary) self.assertFalse(self.obj.headers) def test_headers_if_has_encoded_headers(self): self.obj.encoded_headers = '{"some-header": "some_data"}' self.assertIsInstance(self.obj.headers, models.CaseInsensitiveDictionary) self.assertDictEqual(self.obj.headers, {'some-header': 'some_data'}) def test_content_type_if_no_headers(self): self.assertEqual(self.obj.content_type, None) def test_content_type_if_no_specific_content_type(self): self.obj.encoded_headers = '{"foo": "some_data"}' self.assertEqual(self.obj.content_type, None) def test_content_type_if_header_have_content_type(self): self.obj.encoded_headers = '{"content-type": "some_data"}' self.assertEqual(self.obj.content_type, "some_data") def test_garbage_collect(self): self.assertTrue(models.Request.objects.filter(id=self.obj.id).exists()) SilkyConfig().SILKY_MAX_RECORDED_REQUESTS_CHECK_PERCENT = 100 SilkyConfig().SILKY_MAX_RECORDED_REQUESTS = 0 models.Request.garbage_collect() self.assertFalse(models.Request.objects.filter(id=self.obj.id).exists()) def test_probabilistic_garbage_collect(self): self.assertTrue(models.Request.objects.filter(id=self.obj.id).exists()) SilkyConfig().SILKY_MAX_RECORDED_REQUESTS_CHECK_PERCENT = 0 SilkyConfig().SILKY_MAX_RECORDED_REQUESTS = 0 models.Request.garbage_collect() self.assertTrue(models.Request.objects.filter(id=self.obj.id).exists()) def test_force_garbage_collect(self): self.assertTrue(models.Request.objects.filter(id=self.obj.id).exists()) SilkyConfig().SILKY_MAX_RECORDED_REQUESTS_CHECK_PERCENT = 0 SilkyConfig().SILKY_MAX_RECORDED_REQUESTS = 0 models.Request.garbage_collect(force=True) self.assertFalse(models.Request.objects.filter(id=self.obj.id).exists()) def test_greedy_garbage_collect(self): for x in range(3): obj = models.Request(path='/', method='get') obj.save() self.assertEqual(models.Request.objects.count(), 4) SilkyConfig().SILKY_MAX_RECORDED_REQUESTS_CHECK_PERCENT = 50 SilkyConfig().SILKY_MAX_RECORDED_REQUESTS = 3 models.Request.garbage_collect(force=True) self.assertGreater(models.Request.objects.count(), 0) def test_save_if_have_no_raw_body(self): obj = models.Request(path='/some/path/', method='get') self.assertEqual(obj.raw_body, '') obj.save() self.assertEqual(obj.raw_body, '') def test_save_if_have_raw_body(self): obj = models.Request(path='/some/path/', method='get', raw_body='some text') obj.save() self.assertEqual(obj.raw_body, 'some text') def test_save_if_have_no_body(self): obj = models.Request(path='/some/path/', method='get') self.assertEqual(obj.body, '') obj.save() self.assertEqual(obj.body, '') def test_save_if_have_body(self): obj = models.Request(path='/some/path/', method='get', body='some text') obj.save() self.assertEqual(obj.body, 'some text') def test_save_if_have_no_end_time(self): obj = models.Request(path='/some/path/', method='get') self.assertEqual(obj.time_taken, None) obj.save() self.assertEqual(obj.time_taken, None) @freeze_time('2016-01-01 12:00:00') def test_save_if_have_end_time(self): date = datetime.datetime(2016, 1, 1, 12, 0, 3, tzinfo=datetime.timezone.utc) obj = models.Request(path='/some/path/', method='get', end_time=date) obj.save() self.assertEqual(obj.end_time, date) self.assertEqual(obj.time_taken, 3000.0) def test_prof_file_default_storage(self): obj = models.Request(path='/some/path/', method='get') self.assertEqual(obj.prof_file.storage.__class__, ProfilerResultStorage) class ResponseTest(TestCase): def setUp(self): self.obj = ResponseFactory.create() def test_uuid_is_primary_key(self): self.assertIsInstance(self.obj.id, uuid.UUID) def test_is_1to1_related_to_request(self): request = RequestMinFactory.create() resp = models.Response.objects.create(status_code=200, request=request) self.assertEqual(request.response, resp) def test_headers_if_has_no_encoded_headers(self): self.assertIsInstance(self.obj.headers, models.CaseInsensitiveDictionary) self.assertFalse(self.obj.headers) def test_headers_if_has_encoded_headers(self): self.obj.encoded_headers = '{"some-header": "some_data"}' self.assertIsInstance(self.obj.headers, models.CaseInsensitiveDictionary) self.assertDictEqual(self.obj.headers, {'some-header': 'some_data'}) def test_content_type_if_no_headers(self): self.assertEqual(self.obj.content_type, None) def test_content_type_if_no_specific_content_type(self): self.obj.encoded_headers = '{"foo": "some_data"}' self.assertEqual(self.obj.content_type, None) def test_content_type_if_header_have_content_type(self): self.obj.encoded_headers = '{"content-type": "some_data"}' self.assertEqual(self.obj.content_type, "some_data") class SQLQueryManagerTest(TestCase): def test_if_no_args_passed(self): pass def test_if_one_arg_passed(self): pass def if_a_few_args_passed(self): pass def if_objs_kw_arg_passed(self): pass def if_not_the_objs_kw_arg_passed(self): pass class SQLQueryTest(TestCase): def setUp(self): self.obj = SQLQueryFactory.create() self.end_time = datetime.datetime(2016, 1, 1, 12, 0, 5, tzinfo=datetime.timezone.utc) self.start_time = datetime.datetime(2016, 1, 1, 12, 0, 0, tzinfo=datetime.timezone.utc) @freeze_time('2016-01-01 12:00:00') def test_start_time_field_default(self): obj = SQLQueryFactory.create() self.assertEqual(obj.start_time, datetime.datetime(2016, 1, 1, 12, 0, 0, tzinfo=datetime.timezone.utc)) def test_is_m2o_related_to_request(self): request = RequestMinFactory() self.obj.request = request self.obj.save() self.assertIn(self.obj, request.queries.all()) def test_query_manager_instance(self): self.assertIsInstance(models.SQLQuery.objects, models.SQLQueryManager) def test_traceback_ln_only(self): self.obj.traceback = """Traceback (most recent call last): File "/home/user/some_script.py", line 10, in some_func pass File "/usr/lib/python2.7/bdb.py", line 20, in trace_dispatch return self.dispatch_return(frame, arg) File "/usr/lib/python2.7/bdb.py", line 30, in dispatch_return if self.quitting: raise BdbQuit BdbQuit""" output = ('Traceback (most recent call last):\n' ' pass\n' ' return self.dispatch_return(frame, arg)\n' ' if self.quitting: raise BdbQuit') self.assertEqual(self.obj.traceback_ln_only, output) def test_formatted_query_if_no_query(self): self.obj.query = "" self.obj.formatted_query def test_formatted_query_if_has_a_query(self): query = """SELECT Book.title AS Title, COUNT(*) AS Authors FROM Book JOIN Book_author ON Book.isbn = Book_author.isbn GROUP BY Book.title;""" self.obj.query = query self.obj.formatted_query def test_num_joins_if_no_joins_in_query(self): query = """SELECT Book.title AS Title, COUNT(*) AS Authors FROM Book GROUP BY Book.title;""" self.obj.query = query self.assertEqual(self.obj.num_joins, 0) def test_num_joins_if_joins_in_query(self): query = """SELECT p.id FROM Person p JOIN address a ON p.Id = a.Person_ID JOIN address_type at ON a.Type_ID = at.Id JOIN `option` o ON p.Id = o.person_Id JOIN option_address_type oat ON o.id = oat.option_id WHERE a.country_id = 1 AND at.id <> oat.type_id;""" self.obj.query = query self.assertEqual(self.obj.num_joins, 4) def test_num_joins_if_no_joins_in_query_but_this_word_searched(self): query = """SELECT Book.title FROM Book WHERE Book.title=`Join the dark side, Luke!`;""" self.obj.query = query self.assertEqual(self.obj.num_joins, 0) def test_num_joins_if_multiple_statement_in_query(self): query = """SELECT Book.title FROM Book WHERE Book.title=`Join the dark side, Luke!`; SELECT Book.joiner FROM Book LEFT OUTER JOIN joined ON Book.joiner = joined.joiner INNER JOIN joined ON Book.joiner = joined.joiner WHERE Book.joiner='Join i am'""" self.obj.query = query self.assertEqual(self.obj.num_joins, 2) def test_tables_involved_if_no_query(self): self.obj.query = '' self.assertEqual(self.obj.tables_involved, []) def test_tables_involved_if_query_has_only_a_from_token(self): query = """SELECT * FROM Book;""" self.obj.query = query self.assertEqual(self.obj.tables_involved, ['Book;']) def test_tables_involved_if_query_has_a_join_token(self): query = """SELECT p.id FROM Person p JOIN Address a ON p.Id = a.Person_ID;""" self.obj.query = query self.assertEqual(self.obj.tables_involved, ['Person', 'Address']) def test_tables_involved_if_query_has_an_as_token(self): query = 'SELECT Book.title AS Title FROM Book GROUP BY Book.title;' self.obj.query = query self.assertEqual(self.obj.tables_involved, ['Title', 'Book']) # FIXME bug, not a feature def test_tables_involved_check_with_fake_a_from_token(self): query = """SELECT * FROM Book WHERE Book.title=`EVIL FROM WITHIN`;""" self.obj.query = query self.assertEqual(self.obj.tables_involved, ['Book', 'WITHIN`;']) # FIXME bug, not a feature def test_tables_involved_check_with_fake_a_join_token(self): query = """SELECT * FROM Book WHERE Book.title=`Luke, join the dark side!`;""" self.obj.query = query self.assertEqual(self.obj.tables_involved, ['Book', 'the']) # FIXME bug, not a feature def test_tables_involved_check_with_fake_an_as_token(self): query = """SELECT * FROM Book WHERE Book.title=`AS SOON AS POSIABLE`;""" self.obj.query = query self.assertEqual(self.obj.tables_involved, ['Book', 'POSIABLE`;']) def test_tables_involved_if_query_has_subquery(self): query = '''SELECT A.Col1, A.Col2, B.Col1,B.Col2 FROM (SELECT RealTableZ.Col1, RealTableY.Col2, RealTableY.ID AS ID FROM RealTableZ LEFT OUTER JOIN RealTableY ON RealTableZ.ForeignKeyY=RealTableY.ID WHERE RealTableY.Col11>14 ) AS B INNER JOIN A ON A.ForeignKeyY=B.ID;''' self.obj.query = query self.assertEqual(self.obj.tables_involved, ['ID', 'RealTableZ', 'RealTableY', 'B', 'A']) # FIXME bug, not a feature def test_tables_involved_if_query_has_django_aliase_on_column_names(self): query = 'SELECT foo AS bar FROM some_table;' self.obj.query = query self.assertEqual(self.obj.tables_involved, ['bar', 'some_table;']) def test_tables_involved_if_query_has_update_token(self): query = """UPDATE Book SET title = 'New Title' WHERE id = 1;""" self.obj.query = query self.assertEqual(self.obj.tables_involved, ['Book']) def test_tables_involved_in_complex_update_query(self): query = '''UPDATE Person p SET p.name = (SELECT c.name FROM Company c WHERE c.id = p.company_id), p.salary = p.salary * 1.1 FROM Department d WHERE p.department_id = d.id AND d.budget > 100000; ''' self.obj.query = query self.assertEqual(self.obj.tables_involved, ['Person', 'Company', 'Department']) def test_tables_involved_in_update_with_subquery(self): query = '''UPDATE Employee e SET e.bonus = (SELECT AVG(salary) FROM Employee WHERE department_id = e.department_id) WHERE e.performance = 'excellent'; ''' self.obj.query = query self.assertEqual(self.obj.tables_involved, ['Employee', 'Employee']) def test_save_if_no_end_and_start_time(self): obj = SQLQueryFactory.create() self.assertEqual(obj.time_taken, None) @freeze_time('2016-01-01 12:00:00') def test_save_if_has_end_time(self): # datetime.datetime(2016, 1, 1, 12, 0, 5, tzinfo=datetime.timezone.utc) obj = SQLQueryFactory.create(end_time=self.end_time) self.assertEqual(obj.time_taken, 5000.0) @freeze_time('2016-01-01 12:00:00') def test_save_if_has_start_time(self): obj = SQLQueryFactory.create(start_time=self.start_time) self.assertEqual(obj.time_taken, None) def test_save_if_has_end_and_start_time(self): obj = SQLQueryFactory.create(start_time=self.start_time, end_time=self.end_time) self.assertEqual(obj.time_taken, 5000.0) def test_save_if_has_pk_and_request(self): self.obj.request = RequestMinFactory.create() self.obj.save() self.assertEqual(self.obj.request.num_sql_queries, 0) def test_save_if_has_no_pk(self): obj = SQLQueryFactory.build(start_time=self.start_time, end_time=self.end_time) obj.request = RequestMinFactory.create() obj.save() self.assertEqual(obj.request.num_sql_queries, 1) # should not rise def test_save_if_has_no_request(self): obj = SQLQueryFactory.build(start_time=self.start_time, end_time=self.end_time) obj.save() # FIXME a bug def test_delete_if_no_related_requests(self): with self.assertRaises(AttributeError): self.obj.delete() # self.assertNotIn(self.obj, models.SQLQuery.objects.all()) def test_delete_if_has_request(self): self.obj.request = RequestMinFactory.create() self.obj.save() self.obj.delete() self.assertNotIn(self.obj, models.SQLQuery.objects.all()) class NoPendingMigrationsTest(TestCase): """ Test if proper migrations are added and the models state is consistent. It should make sure that no new migrations are created for this app, when end-user runs `makemigrations` command. """ def test_no_pending_migrations(self): call_command("makemigrations", "silk", "--check", "--dry-run") @override_settings(DEFAULT_AUTO_FIELD='django.db.models.BigAutoField') def test_check_with_overridden_default_auto_field(self): """ Test with `BigAutoField` set as `DEFAULT_AUTO_FIELD` - which is default when generating proj with Django 3.2. """ self.test_no_pending_migrations() class BaseProfileTest(TestCase): pass class ProfileTest(TestCase): pass ================================================ FILE: project/tests/test_multipart_forms.py ================================================ from unittest.mock import Mock from django.test import TestCase from django.urls import reverse from silk.model_factory import RequestModelFactory, multipart_form class TestMultipartForms(TestCase): def test_no_max_request(self): mock_request = Mock() mock_request.headers = {'content-type': multipart_form} mock_request.GET = {} mock_request.path = reverse('silk:requests') mock_request.method = 'post' mock_request.body = Mock() request_model = RequestModelFactory(mock_request).construct_request_model() self.assertFalse(request_model.body) self.assertEqual(b"Raw body not available for multipart_form data, Silk is not showing file uploads.", request_model.raw_body) mock_request.body.assert_not_called() ================================================ FILE: project/tests/test_profile_dot.py ================================================ # std import cProfile import os import tempfile from contextlib import contextmanager from unittest.mock import MagicMock # 3rd party from django.test import TestCase from networkx.drawing.nx_pydot import read_dot # silk from silk.views.profile_dot import ( _create_dot, _create_profile, _temp_file_from_file_field, ) class ProfileDotViewTestCase(TestCase): @classmethod @contextmanager def _stats_file(cls): """ Context manager to create some arbitrary profiling stats in a temp file, returning the filename on enter, and removing the temp file on exit. """ try: with tempfile.NamedTemporaryFile(delete=False) as stats: pass cProfile.run('1+1', stats.name) yield stats.name finally: os.unlink(stats.name) @classmethod @contextmanager def _stats_data(cls): """ Context manager to create some arbitrary profiling stats in a temp file, returning the data on enter, and removing the temp file on exit. """ with cls._stats_file() as filename: with open(filename, 'rb') as f: yield f.read() @classmethod def _profile(cls): """Create some arbitrary profiling stats.""" with cls._stats_file() as filename: # create profile - we don't need to convert a django file field to a temp file # just use the filename of the temp file already created @contextmanager def dummy(_): yield filename return _create_profile(filename, dummy) @classmethod def _mock_file(cls, data): """ Get a mock object that looks like a file but returns data when read is called. """ i = [0] def read(n): if not i[0]: i[0] += 1 return data stream = MagicMock() stream.open = lambda: None stream.read = read return stream def test_create_dot(self): """ Verify that a dot file is correctly created from pstats data stored in a file field. """ with self._stats_file(): try: # create dot with tempfile.NamedTemporaryFile(delete=False) as dotfile: dot = _create_dot(self._profile(), 5) dotfile.write(dot.encode('utf-8')) # verify generated dot is valid G = read_dot(dotfile.name) self.assertGreater(len(G.nodes()), 0) finally: os.unlink(dotfile.name) def test_temp_file_from_file_field(self): """ Verify that data held in a file like object is copied to a temp file. """ dummy_data = b'dummy data' stream = self._mock_file(dummy_data) with _temp_file_from_file_field(stream) as filename: with open(filename, 'rb') as f: self.assertEqual(f.read(), dummy_data) # file should have been removed on exit self.assertFalse(os.path.exists(filename)) ================================================ FILE: project/tests/test_profile_parser.py ================================================ import contextlib import cProfile import io import re from django.test import TestCase from silk.utils.profile_parser import parse_profile class ProfileParserTestCase(TestCase): def test_profile_parser(self): """ Verify that the function parse_profile produces the expected output. """ with contextlib.closing(io.StringIO()) as stream: with contextlib.redirect_stdout(stream): cProfile.run('print()') stream.seek(0) actual = list(parse_profile(stream)) # Expected format for the profiling output on cPython implementations (PyPy differs) # actual = [ # ["ncalls", "tottime", "percall", "cumtime", "percall", "filename:lineno(function)"], # ["1", "0.000", "0.000", "0.000", "0.000", "| '; } for (j = 0; j < 7; j += 1) { table += ' | ' + options.i18n[options.lang].dayOfWeek[(j + options.dayOfWeekStart) % 7] + ' | '; } table += '
|---|---|
| ' + w + ' | '; } } table += '' +
' ' + d + ' ' +
' | ';
if (start.getDay() === options.dayOfWeekStartPrev) {
table += '