Showing preview only (1,433K chars total). Download the full file or copy to clipboard to get everything.
Repository: crazyguitar/pysheeet
Branch: master
Commit: eca8a13c450c
Files: 178
Total size: 1.3 MB
Directory structure:
gitextract_168c70l7/
├── .clang-format
├── .claude-plugin/
│ ├── marketplace.json
│ └── plugin.json
├── .coveragerc
├── .gitattributes
├── .github/
│ ├── FUNDING.yml
│ ├── dependabot.yml
│ └── workflows/
│ └── pythonpackage.yml
├── .gitignore
├── CITATION.cff
├── LICENSE
├── Makefile
├── Procfile
├── README.rst
├── app.py
├── app_test.py
├── docs/
│ ├── 404.rst
│ ├── Makefile
│ ├── _extra/
│ │ └── robots.txt
│ ├── _static/
│ │ ├── .gitignore
│ │ ├── carbonad.css
│ │ └── style.css
│ ├── _templates/
│ │ ├── carbonad.html
│ │ ├── cheatsheets.html
│ │ ├── layout.html
│ │ ├── link.html
│ │ └── sidebarintro.html
│ ├── conf.py
│ ├── index.rst
│ └── notes/
│ ├── appendix/
│ │ ├── disaggregated-prefill-decode.rst
│ │ ├── index.rst
│ │ ├── megatron-efa-monitoring.rst
│ │ ├── nccl-gin.rst
│ │ ├── python-gdb.rst
│ │ └── python-walrus.rst
│ ├── asyncio/
│ │ ├── index.rst
│ │ ├── python-asyncio-advanced.rst
│ │ ├── python-asyncio-basic.rst
│ │ ├── python-asyncio-guide.rst
│ │ └── python-asyncio-server.rst
│ ├── basic/
│ │ ├── index.rst
│ │ ├── python-basic.rst
│ │ ├── python-dict.rst
│ │ ├── python-func.rst
│ │ ├── python-future.rst
│ │ ├── python-generator.rst
│ │ ├── python-heap.rst
│ │ ├── python-list.rst
│ │ ├── python-object.rst
│ │ ├── python-rexp.rst
│ │ ├── python-set.rst
│ │ ├── python-typing.rst
│ │ └── python-unicode.rst
│ ├── concurrency/
│ │ ├── index.rst
│ │ ├── python-futures.rst
│ │ ├── python-multiprocessing.rst
│ │ └── python-threading.rst
│ ├── database/
│ │ ├── index.rst
│ │ ├── python-sqlalchemy-orm.rst
│ │ ├── python-sqlalchemy-query.rst
│ │ └── python-sqlalchemy.rst
│ ├── extension/
│ │ ├── cpp-from-python.rst
│ │ ├── index.rst
│ │ ├── python-capi.rst
│ │ ├── python-cext-modern.rst
│ │ └── python-ctypes.rst
│ ├── hpc/
│ │ ├── index.rst
│ │ └── slurm.rst
│ ├── llm/
│ │ ├── index.rst
│ │ ├── llm-bench.rst
│ │ ├── llm-serving.rst
│ │ ├── megatron.rst
│ │ └── pytorch.rst
│ ├── network/
│ │ ├── index.rst
│ │ ├── python-socket-async.rst
│ │ ├── python-socket-server.rst
│ │ ├── python-socket-sniffer.rst
│ │ ├── python-socket-ssl.rst
│ │ ├── python-socket.rst
│ │ └── python-ssh.rst
│ ├── os/
│ │ ├── index.rst
│ │ ├── python-date.rst
│ │ ├── python-io.rst
│ │ └── python-os.rst
│ ├── python-new-py3.rst
│ └── security/
│ ├── index.rst
│ ├── python-crypto.rst
│ ├── python-tls.rst
│ └── python-vulnerability.rst
├── requirements.txt
├── runtime.txt
├── skills/
│ └── py/
│ ├── SKILL.md
│ └── references/
│ ├── guidelines.md
│ └── structure.md
└── src/
├── basic/
│ ├── asyncio_.py
│ ├── basic.py
│ ├── cext_.py
│ ├── concurrency_.py
│ ├── crypto_.py
│ ├── datetime_.py
│ ├── dict.py
│ ├── fileio_.py
│ ├── func.py
│ ├── future_.py
│ ├── generator.py
│ ├── heap.py
│ ├── list.py
│ ├── object.py
│ ├── os_.py
│ ├── rexp.py
│ ├── set.py
│ ├── socket_.py
│ ├── sqlalchemy_core.py
│ ├── sqlalchemy_orm.py
│ ├── sqlalchemy_query.py
│ ├── typing_.py
│ └── unicode_.py
├── cext/
│ ├── CMakeLists.txt
│ ├── README.md
│ ├── capi/
│ │ ├── args.c
│ │ ├── errors.c
│ │ ├── gil.c
│ │ ├── setup.py
│ │ ├── simple.c
│ │ ├── test_capi.py
│ │ └── types_demo.c
│ ├── conftest.py
│ ├── example.cpp
│ ├── fib.c
│ ├── gil_example.cpp
│ ├── numpy_example.cpp
│ ├── setup.py
│ ├── test_cext.py
│ └── vector.cpp
├── cpp_from_python/
│ ├── CMakeLists.txt
│ └── cpp_from_py.cpp
├── gin/
│ ├── Dockerfile
│ ├── Makefile
│ ├── run.enroot
│ └── run.sbatch
├── llm/
│ ├── sglang/
│ │ ├── Dockerfile
│ │ ├── Makefile
│ │ ├── README.rst
│ │ ├── bench.sh
│ │ ├── run.sbatch
│ │ └── test.sh
│ ├── tensorrt-llm/
│ │ ├── Dockerfile
│ │ ├── Makefile
│ │ ├── README.rst
│ │ ├── bench.sh
│ │ ├── run.sbatch
│ │ └── test.sh
│ └── vllm/
│ ├── Dockerfile
│ ├── Makefile
│ ├── README.rst
│ ├── bench.sh
│ ├── offline_bench.py
│ ├── offline_bench.sh
│ ├── run.sbatch
│ ├── run.sh
│ ├── sweep.sbatch
│ ├── sweep.sh
│ └── test.sh
├── megatron/
│ ├── Dockerfile
│ ├── Makefile
│ ├── README.md
│ ├── enroot.sh
│ ├── entrypoint.py
│ ├── recipes/
│ │ └── deepseek_v2_lite_pretrain.py
│ ├── srun.sh
│ └── viztracer_plugin.py
├── new_py3/
│ └── py3.py
├── nixl/
│ ├── Dockerfile
│ ├── Makefile
│ ├── bench.sh
│ ├── nixl.sbatch
│ └── vllm.sbatch
└── security/
└── vulnerability_.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .clang-format
================================================
BasedOnStyle: Google
AlignAfterOpenBracket: BlockIndent
AllowAllParametersOfDeclarationOnNextLine: false
BinPackParameters: false
ColumnLimit: 150
================================================
FILE: .claude-plugin/marketplace.json
================================================
{
"name": "pysheeet",
"owner": {
"name": "crazyguitar"
},
"plugins": [
{
"name": "pysheeet",
"source": {
"source": "github",
"repo": "crazyguitar/pysheeet"
},
"description": "Comprehensive Python programming reference covering syntax, concurrency, networking, databases, ML/LLM development, and HPC",
"version": "1.0.0",
"author": {
"name": "crazyguitar"
}
}
]
}
================================================
FILE: .claude-plugin/plugin.json
================================================
{
"name": "pysheeet",
"description": "Comprehensive Python programming reference covering syntax, concurrency, networking, databases, ML/LLM development, and HPC",
"version": "1.0.0",
"author": {
"name": "crazyguitar"
},
"homepage": "https://www.pythonsheets.com",
"repository": "https://github.com/crazyguitar/pysheeet",
"license": "MIT"
}
================================================
FILE: .coveragerc
================================================
[report]
omit =
*/python?.?/*
*/site-packages/*
app_test.py
exclude_lines =
if __name__ == .__main__.:
if .DYNO. in os.environ:
================================================
FILE: .gitattributes
================================================
*.png filter=lfs diff=lfs merge=lfs -text
docs/_static/appendix/*.png !filter !diff !merge
docs/_static/appendix/nixl/*.png !filter !diff !merge
================================================
FILE: .github/FUNDING.yml
================================================
github: crazyguitar
================================================
FILE: .github/dependabot.yml
================================================
version: 2
updates:
- package-ecosystem: pip
directory: "/"
schedule:
interval: daily
time: "21:00"
open-pull-requests-limit: 10
ignore:
- dependency-name: sphinx
versions:
- 3.5.3
================================================
FILE: .github/workflows/pythonpackage.yml
================================================
name: Build
on: [push, pull_request]
jobs:
build:
runs-on: ubuntu-latest
strategy:
max-parallel: 4
matrix:
python: ["3.11", "3.12", "3.13"]
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python }}
- name: Install clang-format and cmake
run: sudo apt-get install -y clang-format cmake
- name: Install
run: make deps
- name: Test
run: make clean && make test
================================================
FILE: .gitignore
================================================
_build/
# Created by https://www.gitignore.io/api/vim,python
### Python ###
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
.hypothesis/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
target/
# Jupyter Notebook
.ipynb_checkpoints
# pyenv
.python-version
# celery beat schedule file
celerybeat-schedule
# SageMath parsed files
*.sage.py
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
### Vim ###
# swap
[._]*.s[a-v][a-z]
[._]*.sw[a-p]
[._]s[a-v][a-z]
[._]sw[a-p]
# session
Session.vim
# temporary
.netrwhist
*~
# auto-generated tag files
tags
# End of https://www.gitignore.io/api/vim,python
================================================
FILE: CITATION.cff
================================================
cff-version: 1.2.0
message: "If you use this software, please cite it as below."
title: "Python Cheatsheet"
authors:
- family-names: "Tsai"
given-names: "Chang-Ning"
orcid: "https://orcid.org/0009-0000-5297-5940"
github: "crazyguitar"
abstract: "A comprehensive Python cheat sheet covering Python 2 and 3 syntax, tips, and code snippets."
version: "master"
repository-code: "https://github.com/crazyguitar/pysheeet"
license: "MIT"
date-released: "2016-02-29"
url: "https://www.pythonsheets.com"
keywords:
- python
- cheatsheet
- python-2
- python-3
================================================
FILE: LICENSE
================================================
MIT License
Copyright (c) 2015-2026 Chang Ning Tsai
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: Makefile
================================================
REQUIREMENT = requirements.txt
VER = $(word 2, $(shell python --version 2>&1))
SRC = app.py app_test.py
PY36 = $(shell expr $(VER) \>= 3.6)
CEXT_DIR = src/cext
CEXT_BUILD = $(CEXT_DIR)/build
CAPI_DIR = src/cext/capi
CPP_FROM_PY_DIR = src/cpp_from_python
CPP_FROM_PY_BUILD = $(CPP_FROM_PY_DIR)/build
PY_ARCH = $(shell python -c "import platform; print(platform.machine())")
.PHONY: build deps test format cext
build: html
%:
cd docs && make $@
clean:
cd docs && make clean
rm -rf $(CEXT_BUILD)
rm -rf $(CPP_FROM_PY_BUILD)
rm -rf $(CAPI_DIR)/build $(CAPI_DIR)/*.so $(CAPI_DIR)/*.egg-info
cext:
@echo "Building C/C++ extensions for $(PY_ARCH)..."
mkdir -p $(CEXT_BUILD) && \
cd $(CEXT_BUILD) && \
cmake -DCMAKE_OSX_ARCHITECTURES=$(PY_ARCH) .. && \
make
cd $(CAPI_DIR) && python setup.py build_ext --inplace
cpp_from_python:
@echo "Building C++ from Python examples..."
mkdir -p $(CPP_FROM_PY_BUILD) && \
cd $(CPP_FROM_PY_BUILD) && \
cmake .. && \
make
test: clean build cext cpp_from_python
pycodestyle $(SRC)
pydocstyle $(SRC)
bandit app.py
coverage run app_test.py && coverage report --fail-under=100 -m $(SRC)
python -m pytest src/basic/*.py src/new_py3/*.py -v
python -m pytest $(CEXT_DIR)/test_cext.py -v
python -m pytest $(CAPI_DIR)/test_capi.py -v
cd $(CPP_FROM_PY_BUILD) && make test
ifeq ($(PY36), 1)
black --quiet --diff --check --line-length 79 $(SRC)
endif
deps:
pip install -r requirements.txt
pip install pybind11
ifeq ($(PY36), 1)
pip install black==22.3.0
endif
format:
black --line-length 79 $(SRC) src/
find src/cext -type f \( -name "*.cpp" -o -name "*.c" -o -name "*.h" \) | xargs -I{} clang-format -style=file -i {}
================================================
FILE: Procfile
================================================
web: make clean && make && gunicorn app:app --log-file -
================================================
FILE: README.rst
================================================
.. raw:: html
<h1 align="center">
<br>
<a href="https://www.pythonsheets.com"><img src="docs/_static/logo.png" alt="pysheeet" width=200"></a>
</h1>
<p align="center">
<a href="https://github.com/crazyguitar/pysheeet/actions">
<img src="https://github.com/crazyguitar/pysheeet/actions/workflows/pythonpackage.yml/badge.svg" alt="Build Status">
</a>
<a href="https://coveralls.io/github/crazyguitar/pysheeet?branch=master">
<img src="https://coveralls.io/repos/github/crazyguitar/pysheeet/badge.svg?branch=master" alt="Coverage">
</a>
<a href="https://raw.githubusercontent.com/crazyguitar/pysheeet/master/LICENSE">
<img src="https://img.shields.io/badge/License-MIT-blue.svg" alt="License MIT">
</a>
<a href="https://doi.org/10.5281/zenodo.15529042">
<img src="https://zenodo.org/badge/52760178.svg" alt="DOI">
</a>
</p>
Introduction
=============
This project was started to bring together useful Python code snippets that make
coding faster, easier, and more enjoyable. You can explore all the cheat sheets at
`Pysheeet <https://www.pythonsheets.com/>`_. Contributions are always welcome—feel
free to fork the repo and submit a pull request to help it grow!
Plugin
======
**pysheeet** is available as a Claude Code plugin. Once installed, Claude
automatically uses the cheat sheets to answer Python questions — just ask
naturally and the skill triggers based on context.
Installation
------------
**As a Claude Code plugin (recommended):**
.. code-block:: bash
# Step 1: Add the marketplace
claude plugin marketplace add crazyguitar/pysheeet
# Step 2: Install the plugin
claude plugin install pysheeet@pysheeet
**Local testing (single session only):**
.. code-block:: bash
claude --plugin-dir /path/to/pysheeet
**Manual installation (requires cloning the repo):**
.. code-block:: bash
git clone https://github.com/crazyguitar/pysheeet.git
mkdir -p ~/.claude/skills
cp -r pysheeet/skills/py ~/.claude/skills/py
What's New In Python 3
======================
This part only provides a quick glance at some important features in Python 3.
If you're interested in all of the most important features, please read the
official document, `What’s New in Python <https://docs.python.org/3/whatsnew/index.html>`_.
- `New in Python3 <docs/notes/python-new-py3.rst>`_
Cheat Sheet
===========
Core Python fundamentals including data types, functions, classes, and commonly
used patterns for everyday programming tasks.
- `From Scratch <docs/notes/basic/python-basic.rst>`_
- `Future <docs/notes/basic/python-future.rst>`_
- `Typing <docs/notes/basic/python-typing.rst>`_
- `Class <docs/notes/basic/python-object.rst>`_
- `Function <docs/notes/basic/python-func.rst>`_
- `Unicode <docs/notes/basic/python-unicode.rst>`_
- `List <docs/notes/basic/python-list.rst>`_
- `Set <docs/notes/basic/python-set.rst>`_
- `Dictionary <docs/notes/basic/python-dict.rst>`_
- `Heap <docs/notes/basic/python-heap.rst>`_
- `Generator <docs/notes/basic/python-generator.rst>`_
- `Regular expression <docs/notes/basic/python-rexp.rst>`_
System
======
Date/time handling, file I/O, and operating system interfaces.
- `Datetime <docs/notes/os/python-date.rst>`_ - Timestamps, formatting, parsing, timezones, timedelta
- `Files and I/O <docs/notes/os/python-io.rst>`_ - Reading, writing, pathlib, shutil, tempfile
- `Operating System <docs/notes/os/python-os.rst>`_ - Processes, environment, system calls
Concurrency
===========
Threading, multiprocessing, and concurrent.futures for parallel execution.
Covers synchronization primitives, process pools, and bypassing the GIL.
- `Threading <docs/notes/concurrency/python-threading.rst>`_ - Threads, locks, semaphores, events, conditions
- `Multiprocessing <docs/notes/concurrency/python-multiprocessing.rst>`_ - Processes, pools, shared memory, IPC
- `concurrent.futures <docs/notes/concurrency/python-futures.rst>`_ - Executors, futures, callbacks
Asyncio
=======
Asynchronous programming with Python's ``asyncio`` module. Covers coroutines,
event loops, tasks, networking, and advanced patterns.
- `A Hitchhiker's Guide to Asynchronous Programming <docs/notes/asyncio/python-asyncio-guide.rst>`_ - Design philosophy and evolution
- `Asyncio Basics <docs/notes/asyncio/python-asyncio-basic.rst>`_ - Coroutines, tasks, gather, timeouts
- `Asyncio Networking <docs/notes/asyncio/python-asyncio-server.rst>`_ - TCP/UDP servers, HTTP, SSL/TLS
- `Asyncio Advanced <docs/notes/asyncio/python-asyncio-advanced.rst>`_ - Synchronization, queues, subprocesses
C/C++ Extensions
================
Native extensions for performance-critical code. Covers modern pybind11 (used by
PyTorch, TensorFlow), ctypes, cffi, Cython, and the traditional Python C API.
Also includes a guide for Python developers learning modern C++ syntax.
- `ctypes <docs/notes/extension/python-ctypes.rst>`_ - Load shared libraries without compilation
- `Python C API <docs/notes/extension/python-capi.rst>`_ - Traditional C extension reference
- `Modern C/C++ Extensions <docs/notes/extension/python-cext-modern.rst>`_ - pybind11, Cython
- `Learn C++ from Python <docs/notes/extension/cpp-from-python.rst>`_ - Modern C++ for Python developers
Security
========
Modern cryptographic practices and common security vulnerabilities. Covers
encryption, TLS/SSL, and why legacy patterns are dangerous.
- `Modern Cryptography <docs/notes/security/python-crypto.rst>`_ - AES-GCM, RSA-OAEP, Ed25519, Argon2
- `TLS/SSL and Certificates <docs/notes/security/python-tls.rst>`_ - HTTPS servers, certificate generation
- `Common Vulnerabilities <docs/notes/security/python-vulnerability.rst>`_ - Padding oracle, injection, timing attacks
Network
=======
Low-level network programming with Python sockets. Covers TCP/UDP communication,
server implementations, asynchronous I/O, SSL/TLS encryption, and packet analysis.
- `Socket Basics <docs/notes/network/python-socket.rst>`_
- `Socket Servers <docs/notes/network/python-socket-server.rst>`_
- `Async Socket I/O <docs/notes/network/python-socket-async.rst>`_
- `SSL/TLS Sockets <docs/notes/network/python-socket-ssl.rst>`_
- `Packet Sniffing <docs/notes/network/python-socket-sniffer.rst>`_
- `SSH and Tunnels <docs/notes/network/python-ssh.rst>`_
Database
========
Database access with SQLAlchemy, Python's most popular ORM. Covers connection
management, raw SQL, object-relational mapping, and common query patterns.
- `SQLAlchemy Basics <docs/notes/database/python-sqlalchemy.rst>`_
- `SQLAlchemy ORM <docs/notes/database/python-sqlalchemy-orm.rst>`_
- `SQLAlchemy Query Recipes <docs/notes/database/python-sqlalchemy-query.rst>`_
LLM
===
Large Language Models (LLM) training, inference, and optimization. Covers PyTorch
for model development, distributed training across GPUs, and vLLM/SGLang for
high-performance LLM inference and serving.
- `PyTorch <docs/notes/llm/pytorch.rst>`_ - Tensors, autograd, neural networks, training loops
- `Megatron <docs/notes/llm/megatron.rst>`_ - NVIDIA Megatron training/fine-tuning framework with enroot/pyxis
- `LLM Serving <docs/notes/llm/llm-serving.rst>`_ - vLLM and SGLang for production inference with TP/PP/DP/EP
- `LLM Benchmark <docs/notes/llm/llm-bench.rst>`_ - Benchmark suite for measuring serving performance
HPC
===
High-Performance Computing tools for cluster management and job scheduling.
Covers Slurm workload manager for distributed computing and GPU clusters.
- `Slurm <docs/notes/hpc/slurm.rst>`_
Blog
====
Supplementary topics covering Python internals, debugging techniques, and
language features that don't fit elsewhere.
- `Is Disaggregated Prefill/Decode a Silver Bullet for LLM Serving? <docs/notes/appendix/disaggregated-prefill-decode.rst>`_
- `Monitoring EFA with NCCL GIN and Nsys <docs/notes/appendix/megatron-efa-monitoring.rst>`_
- `GPU-Initiated Networking for NCCL on AWS <docs/notes/appendix/nccl-gin.rst>`_
- `PEP 572 and the walrus operator <docs/notes/appendix/python-walrus.rst>`_
- `Python Interpreter in GNU Debugger <docs/notes/appendix/python-gdb.rst>`_
PDF Version
============
`pdf`_
.. _pdf: https://media.readthedocs.org/pdf/pysheeet/latest/pysheeet.pdf
How to run the server
=======================
.. code-block:: bash
$ virtualenv venv
$ . venv/bin/activate
$ pip install -r requirements.txt
$ make
$ python app.py
# URL: localhost:5000
================================================
FILE: app.py
================================================
# -*- coding: utf-8 -*-
"""This is a simple cheatsheet webapp."""
import os
from flask import Flask, abort, send_from_directory, render_template
from flask_sslify import SSLify
from flask_seasurf import SeaSurf
from flask_talisman import Talisman
from werkzeug.exceptions import NotFound
from werkzeug.utils import safe_join
DIR = os.path.dirname(os.path.realpath(__file__))
ROOT = os.path.join(DIR, "docs", "_build", "html")
def find_key(token):
"""Find the key from the environment variable."""
if token == os.environ.get("ACME_TOKEN"):
return os.environ.get("ACME_KEY")
for k, v in os.environ.items():
if v == token and k.startswith("ACME_TOKEN_"):
n = k.replace("ACME_TOKEN_", "")
return os.environ.get("ACME_KEY_{}".format(n))
csp = {
"default-src": "'none'",
"style-src": ["'self'", "'unsafe-inline'"],
"script-src": [
"'self'",
"*.cloudflare.com",
"*.cloudflareinsights.com",
"*.googletagmanager.com",
"*.google-analytics.com",
"*.carbonads.com",
"*.carbonads.net",
"cdn.carbonads.com",
"srv.carbonads.net",
"'unsafe-inline'",
"'unsafe-eval'",
],
"connect-src": [
"'self'",
"*.google-analytics.com",
"*.analytics.google.com",
"analytics.google.com",
"*.googletagmanager.com",
"*.carbonads.com",
"*.carbonads.net",
"*.doubleclick.net",
],
"font-src": "'self'",
"form-action": "'self'",
"base-uri": "'self'",
"img-src": "*",
"frame-src": ["ghbtns.com", "*.carbonads.com", "*.carbonads.net"],
"frame-ancestors": "'none'",
"object-src": "'none'",
}
feature_policy = {"geolocation": "'none'"}
app = Flask(__name__, template_folder=ROOT)
app.config["SECRET_KEY"] = os.urandom(16)
app.config["SESSION_COOKIE_NAME"] = "__Secure-session"
app.config["SESSION_COOKIE_SAMESITE"] = "Strict"
app.config["CSRF_COOKIE_NAME"] = "__Secure-csrf-token"
app.config["CSRF_COOKIE_HTTPONLY"] = True
app.config["CSRF_COOKIE_SECURE"] = True
csrf = SeaSurf(app)
talisman = Talisman(
app,
force_https=False,
content_security_policy=csp,
feature_policy=feature_policy,
)
if "DYNO" in os.environ:
sslify = SSLify(app, permanent=True, skips=[".well-known"])
@app.errorhandler(404)
def page_not_found(e):
"""Redirect to 404.html."""
return render_template("404.html"), 404
@app.route("/<path:path>")
def static_proxy(path):
"""Find static files safely."""
try:
return send_from_directory(ROOT, path)
except NotFound:
# Handle file not found or directory errors
return render_template("404.html"), 404
@app.route("/")
def index_redirection():
"""Redirecting index file."""
return send_from_directory(ROOT, "index.html")
@csrf.exempt
@app.route("/.well-known/acme-challenge/<token>")
def acme(token):
"""Find the acme-key from environment variable."""
key = find_key(token)
if key is None:
abort(404)
return key
if __name__ == "__main__":
# Only run the app in debug mode during development
app.run(debug=os.environ.get("FLASK_ENV") == "development")
================================================
FILE: app_test.py
================================================
"""Test app.py."""
import multiprocessing
import platform
import unittest
import requests
import os
from pathlib import Path
from werkzeug.exceptions import NotFound
from flask_testing import LiveServerTestCase
from app import acme, find_key, static_proxy, index_redirection, page_not_found
from app import ROOT
from app import app
if platform.system() == "Darwin":
multiprocessing.set_start_method("fork")
class PysheeetTest(LiveServerTestCase):
"""Test app."""
def create_app(self):
"""Create a app for test."""
# remove env ACME_TOKEN*
for k, v in os.environ.items():
if not k.startswith("ACME_TOKEN"):
continue
del os.environ[k]
self.token = "token"
self.key = "key"
os.environ["ACME_TOKEN"] = self.token
os.environ["ACME_KEY"] = self.key
os.environ["FLASK_ENV"] = "development"
os.environ["FLASK_DEBUG"] = "1"
app.config["TESTING"] = True
app.config["LIVESERVER_PORT"] = 0
return app
def check_security_headers(self, resp):
"""Check security headers."""
headers = resp.headers
self.assertTrue("Content-Security-Policy" in headers)
self.assertTrue("X-Content-Type-Options" in headers)
self.assertTrue("Content-Security-Policy" in headers)
self.assertTrue("Feature-Policy" in headers)
self.assertEqual(headers["Feature-Policy"], "geolocation 'none'")
self.assertEqual(headers["X-Frame-Options"], "SAMEORIGIN")
def check_csrf_cookies(self, resp):
"""Check cookies for csrf."""
cookies = resp.cookies
self.assertTrue(cookies.get("__Secure-session"))
self.assertTrue(cookies.get("__Secure-csrf-token"))
def test_index_redirection_req(self):
"""Test that send a request for the index page."""
url = self.get_server_url()
resp = requests.get(url)
self.check_security_headers(resp)
self.check_csrf_cookies(resp)
self.assertEqual(resp.status_code, 200)
def test_static_proxy_req(self):
"""Test that send a request for notes."""
url = self.get_server_url()
notes = Path(ROOT) / "notes"
for html in notes.rglob("*.html"):
page = html.relative_to(ROOT)
u = f"{url}/{page}"
resp = requests.get(u)
self.check_security_headers(resp)
self.check_csrf_cookies(resp)
self.assertEqual(resp.status_code, 200)
def test_acme_req(self):
"""Test that send a request for a acme key."""
url = self.get_server_url()
u = url + "/.well-known/acme-challenge/token"
resp = requests.get(u)
self.check_security_headers(resp)
self.assertEqual(resp.status_code, 200)
u = url + "/.well-known/acme-challenge/foo"
resp = requests.get(u)
self.check_security_headers(resp)
self.assertEqual(resp.status_code, 404)
def test_find_key(self):
"""Test that find a acme key from the environment."""
token = self.token
key = self.key
self.assertEqual(find_key(token), key)
del os.environ["ACME_TOKEN"]
del os.environ["ACME_KEY"]
os.environ["ACME_TOKEN_ENV"] = token
os.environ["ACME_KEY_ENV"] = key
self.assertEqual(find_key(token), key)
del os.environ["ACME_TOKEN_ENV"]
del os.environ["ACME_KEY_ENV"]
def test_acme(self):
"""Test that send a request for a acme key."""
token = self.token
key = self.key
self.assertEqual(acme(token), key)
token = token + "_env"
key = key + "_env"
os.environ["ACME_TOKEN_ENV"] = token
os.environ["ACME_KEY_ENV"] = key
self.assertEqual(find_key(token), key)
del os.environ["ACME_TOKEN_ENV"]
del os.environ["ACME_KEY_ENV"]
self.assertRaises(NotFound, acme, token)
def test_index_redirection(self):
"""Test index page redirection."""
resp = index_redirection()
self.assertEqual(resp.status_code, 200)
resp.close()
def test_static_proxy(self):
"""Test that request static pages."""
notes = Path(ROOT) / "notes"
for html in notes.rglob("*.html"):
u = html.relative_to(ROOT)
resp = static_proxy(u)
self.assertEqual(resp.status_code, 200)
resp.close()
u = "notes/../conf.py"
_, code = static_proxy(u)
self.assertEqual(code, 404)
def test_page_not_found(self):
"""Test page not found."""
html, status_code = page_not_found(None)
self.assertEqual(status_code, 404)
if __name__ == "__main__":
unittest.main()
================================================
FILE: docs/404.rst
================================================
:orphan:
404 Page Not Found
==================
What you were looking for is just not there.
`Click here to go back to homepage. </>`_
================================================
FILE: docs/Makefile
================================================
# Makefile for Sphinx documentation
#
# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
PAPER =
BUILDDIR = _build
# User-friendly check for sphinx-build
ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1)
$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/)
endif
# Internal variables.
PAPEROPT_a4 = -D latex_paper_size=a4
PAPEROPT_letter = -D latex_paper_size=letter
ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
# the i18n builder cannot share the environment and doctrees with the others
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
.PHONY: help
help:
@echo "Please use \`make <target>' where <target> is one of"
@echo " html to make standalone HTML files"
@echo " dirhtml to make HTML files named index.html in directories"
@echo " singlehtml to make a single large HTML file"
@echo " pickle to make pickle files"
@echo " json to make JSON files"
@echo " htmlhelp to make HTML files and a HTML help project"
@echo " qthelp to make HTML files and a qthelp project"
@echo " applehelp to make an Apple Help Book"
@echo " devhelp to make HTML files and a Devhelp project"
@echo " epub to make an epub"
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
@echo " latexpdf to make LaTeX files and run them through pdflatex"
@echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx"
@echo " text to make text files"
@echo " man to make manual pages"
@echo " texinfo to make Texinfo files"
@echo " info to make Texinfo files and run them through makeinfo"
@echo " gettext to make PO message catalogs"
@echo " changes to make an overview of all changed/added/deprecated items"
@echo " xml to make Docutils-native XML files"
@echo " pseudoxml to make pseudoxml-XML files for display purposes"
@echo " linkcheck to check all external links for integrity"
@echo " doctest to run all doctests embedded in the documentation (if enabled)"
@echo " coverage to run coverage check of the documentation (if enabled)"
.PHONY: clean
clean:
rm -rf $(BUILDDIR)/*
.PHONY: html
html:
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
.PHONY: dirhtml
dirhtml:
$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
.PHONY: singlehtml
singlehtml:
$(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
@echo
@echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."
.PHONY: pickle
pickle:
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
@echo
@echo "Build finished; now you can process the pickle files."
.PHONY: json
json:
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
@echo
@echo "Build finished; now you can process the JSON files."
.PHONY: htmlhelp
htmlhelp:
$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
@echo
@echo "Build finished; now you can run HTML Help Workshop with the" \
".hhp project file in $(BUILDDIR)/htmlhelp."
.PHONY: qthelp
qthelp:
$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
@echo
@echo "Build finished; now you can run "qcollectiongenerator" with the" \
".qhcp project file in $(BUILDDIR)/qthelp, like this:"
@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/pysheeet.qhcp"
@echo "To view the help file:"
@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/pysheeet.qhc"
.PHONY: applehelp
applehelp:
$(SPHINXBUILD) -b applehelp $(ALLSPHINXOPTS) $(BUILDDIR)/applehelp
@echo
@echo "Build finished. The help book is in $(BUILDDIR)/applehelp."
@echo "N.B. You won't be able to view it unless you put it in" \
"~/Library/Documentation/Help or install it in your application" \
"bundle."
.PHONY: devhelp
devhelp:
$(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp
@echo
@echo "Build finished."
@echo "To view the help file:"
@echo "# mkdir -p $$HOME/.local/share/devhelp/pysheeet"
@echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/pysheeet"
@echo "# devhelp"
.PHONY: epub
epub:
$(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
@echo
@echo "Build finished. The epub file is in $(BUILDDIR)/epub."
.PHONY: latex
latex:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo
@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
@echo "Run \`make' in that directory to run these through (pdf)latex" \
"(use \`make latexpdf' here to do that automatically)."
.PHONY: latexpdf
latexpdf:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo "Running LaTeX files through pdflatex..."
$(MAKE) -C $(BUILDDIR)/latex all-pdf
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
.PHONY: latexpdfja
latexpdfja:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo "Running LaTeX files through platex and dvipdfmx..."
$(MAKE) -C $(BUILDDIR)/latex all-pdf-ja
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
.PHONY: text
text:
$(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
@echo
@echo "Build finished. The text files are in $(BUILDDIR)/text."
.PHONY: man
man:
$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
@echo
@echo "Build finished. The manual pages are in $(BUILDDIR)/man."
.PHONY: texinfo
texinfo:
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
@echo
@echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo."
@echo "Run \`make' in that directory to run these through makeinfo" \
"(use \`make info' here to do that automatically)."
.PHONY: info
info:
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
@echo "Running Texinfo files through makeinfo..."
make -C $(BUILDDIR)/texinfo info
@echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo."
.PHONY: gettext
gettext:
$(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale
@echo
@echo "Build finished. The message catalogs are in $(BUILDDIR)/locale."
.PHONY: changes
changes:
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
@echo
@echo "The overview file is in $(BUILDDIR)/changes."
.PHONY: linkcheck
linkcheck:
$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
@echo
@echo "Link check complete; look for any errors in the above output " \
"or in $(BUILDDIR)/linkcheck/output.txt."
.PHONY: doctest
doctest:
$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
@echo "Testing of doctests in the sources finished, look at the " \
"results in $(BUILDDIR)/doctest/output.txt."
.PHONY: coverage
coverage:
$(SPHINXBUILD) -b coverage $(ALLSPHINXOPTS) $(BUILDDIR)/coverage
@echo "Testing of coverage in the sources finished, look at the " \
"results in $(BUILDDIR)/coverage/python.txt."
.PHONY: xml
xml:
$(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml
@echo
@echo "Build finished. The XML files are in $(BUILDDIR)/xml."
.PHONY: pseudoxml
pseudoxml:
$(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml
@echo
@echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml."
================================================
FILE: docs/_extra/robots.txt
================================================
User-agent: *
Allow: /
Sitemap: https://www.pythonsheets.com/sitemap.xml
================================================
FILE: docs/_static/.gitignore
================================================
# Ignore everything in this directory
*
# Except this file
!.gitignore
!guido.png
!logo.svg
!style.css
!carbonad.css
!favicon.ico
================================================
FILE: docs/_static/carbonad.css
================================================
#carbonads {
display: block;
overflow: hidden;
padding: 1em;
padding-bottom: 0.3em;
line-height: 1.5;
margin-top: 10px;
margin-bottom: 10px;
}
#carbonads a {
text-decoration: none !important;
border-bottom: none;
}
#carbonads a:hover {
color: inherit;
}
#carbonads span {
display: block;
overflow: hidden;
}
.carbon-img {
display: block;
margin: 0 auto 8px;
}
.carbon-text {
display: block;
text-align: left;
margin-bottom: .1em;
color: #666;
}
.carbon-poweredby {
display: block;
text-align: left;
font-size: .9em;
color: #888 !important;
}
@media only screen and (min-width: 320px) and (max-width: 875px) {
#carbonads {
float: none;
max-width: 330px;
border: 0;
display: block;
overflow: hidden;
margin-top: 20px;
margin-bottom: 20px;
border-radius: 4px;
text-align: center;
box-shadow: 0 0 0 1px hsla(0, 0%, 0%, .1);
font-size: var(--font-size);
background-color: #eee;
line-height: 1.5;
}
#carbonads span {
position: relative;
}
#carbonads > span {
max-width: none;
}
.carbon-img {
float: left;
margin: 0;
}
.carbon-img img {
max-width: 130px !important;
}
.carbon-text {
float: left;
margin-bottom: 0;
padding: 8px 20px;
text-align: left;
color: #333 !important;
max-width: calc(100% - 130px - 3em);
}
.carbon-poweredby {
left: 130px;
bottom: 0;
display: block;
color: #555 !important;
text-align: right;
width: 100%;
}
}
================================================
FILE: docs/_static/style.css
================================================
nav#table-of-contents {
display: none;
}
div.highlight > pre {
font-size: 14px;
border-radius: 3px;
background: #f6f8fa !important;
border: 1px solid #000000 !important;
}
:root {
--cu-boulder-gold: #CFB87C;
}
.bd-container {
max-width: 99%;
}
.bd-container .bd-container__inner {
max-width: 99%;
}
.bd-main .bd-content .bd-article-container {
max-width: 100em;
}
.code-block-caption {
color: black;
}
.bd-sidebar-primary li.has-children>details>summary .toctree-toggle {
justify-content: left;
}
html[data-theme=light] {
--pst-font-size-base: none;
--pst-color-secondary: #176de8;
--pst-color-primary: #176de8;
}
.graph#doc-flowchart .node text {
font-weight: bold;
}
.bd-content .sd-tab-set .sd-tab-content {
padding: 1.5rem;
}
a {
text-decoration: none;
}
a:hover {
text-decoration: underline;
}
button.theme-switch-button {
display: none !important;
}
blockquote {
background-color: transparent;
border: none;
}
================================================
FILE: docs/_templates/carbonad.html
================================================
<script async type="text/javascript" src="https://cdn.carbonads.com/carbon.js?serve={{ carbonad_serve }}&placement={{ carbonad_placement }}" id="_carbonads_js"></script>
================================================
FILE: docs/_templates/cheatsheets.html
================================================
<h3>Cheat Sheets</h3>
<ul>
<li><a href="https://www.cppcheatsheet.com">C/C++ cheat sheet</a></li>
</ul>
================================================
FILE: docs/_templates/layout.html
================================================
{% extends "!layout.html" %}
{%- block extrahead %}
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="{% if pagename == 'index' %}Comprehensive Python cheat sheet with practical code snippets for developers. Learn Python basics, advanced topics, databases, networking, and more.{% else %}{{ title }} - Python code examples and snippets from the comprehensive Python cheat sheet.{% endif %}">
<meta name="keywords" content="Python, Python cheat sheet, Python tutorial, Python examples, Python code snippets, programming, development">
<meta name="author" content="crazyguitar">
<meta name="robots" content="index, follow">
{%- if pagename == 'index' %}
<link rel="canonical" href="https://www.pythonsheets.com/"/>
<meta property="og:url" content="https://www.pythonsheets.com/">
<meta property="og:description" content="Comprehensive Python cheat sheet with practical code snippets for developers. Learn Python basics, advanced topics, databases, networking, and more.">
{%- elif pagename == '404' -%}
{%- else %}
<link rel="canonical" href="https://www.pythonsheets.com/{{ pagename }}{{ file_suffix }}"/>
<meta property="og:url" content="https://www.pythonsheets.com/{{ pagename }}{{ file_suffix }}">
<meta property="og:description" content="{{ title }} - Python code examples and snippets from the comprehensive Python cheat sheet.">
{%- endif %}
<meta property="og:type" content="article">
<meta property="og:title" content="{{ title }}{{ titlesuffix }}">
<meta property="og:site_name" content="Python Cheat Sheet">
<meta name="twitter:card" content="summary">
<meta name="twitter:title" content="{{ title }}{{ titlesuffix }}">
<meta name="twitter:description" content="{% if pagename == 'index' %}Comprehensive Python cheat sheet with practical code snippets for developers{% else %}{{ title }} - Python code examples and snippets{% endif %}">
{%- if tracking_id %}
<!-- Global site tag (gtag.js) - Google Analytics -->
<script async src="https://www.googletagmanager.com/gtag/js?id={{ tracking_id }}"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', '{{ tracking_id }}');
</script>
{% endif -%}
{%- if pagename == '404' -%}
<script>
gtag('event', '404', {
'event_category': 'error',
'event_label': document.referrer,
'non_interaction': true
});
</script>
{%- endif -%}
{% endblock %}
================================================
FILE: docs/_templates/link.html
================================================
<h3>Useful Links</h3>
<ul>
<li><a href="https://www.pythonsheets.com">pysheeet website</a></li>
<li><a href="https://github.com/crazyguitar/pysheeet">pysheeet @ GitHub</a></li>
<li><a href="https://github.com/crazyguitar/pysheeet/issues">Issue Tracker</a></li>
<li><a href="https://media.readthedocs.org/pdf/pysheeet/latest/pysheeet.pdf">pysheeet as a PDF</a></li>
</ul>
================================================
FILE: docs/_templates/sidebarintro.html
================================================
<p>This project tries to provide many snippets of Python code that make life easier.</p>
================================================
FILE: docs/conf.py
================================================
# -*- coding: utf-8 -*-
#
# python-cheatsheet documentation build configuration file, created by
# sphinx-quickstart on Sun Feb 28 09:26:04 2016.
#
# This file is execfile()d with the current directory set to its
# containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.
from datetime import datetime
import os
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
# sys.path.insert(0, os.path.abspath('.'))
# -- General configuration ------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
#needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
'sphinx.ext.todo',
'sphinx.ext.coverage',
'sphinx.ext.viewcode',
'myst_parser',
'sphinx_copybutton',
'sphinx.ext.graphviz',
'sphinx_design',
'sphinx.ext.extlinks'
]
myst_enable_extensions = [
"colon_fence",
"attrs_inline",
"attrs_block",
"tasklist",
"substitution",
]
myst_enable_checkboxes = True
myst_heading_anchors = 6
copybutton_prompt_text = r'^\$ '
copybutton_prompt_is_regexp = True
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
# source_suffix = ['.rst', '.md']
source_suffix = '.rst'
# The encoding of source files.
#source_encoding = 'utf-8-sig'
# The master toctree document.
master_doc = 'index'
# General information about the project.
year = datetime.now().year
project = u'pysheeet'
copyright = u'2016-{}, crazyguitar'.format(year)
author = u'crazyguitar'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = u'0.1.0'
# The full version, including alpha/beta/rc tags.
release = u'0.1.0'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = 'en'
# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
#today = ''
# Else, today_fmt is used as the format for a strftime call.
#today_fmt = '%B %d, %Y'
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = []
# The reST default role (used for this markup: `text`) to use for all
# documents.
#default_role = None
# If true, '()' will be appended to :func: etc. cross-reference text.
#add_function_parentheses = True
# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
#add_module_names = True
# If true, sectionauthor and moduleauthor directives will be shown in the
# output. They are ignored by default.
#show_authors = False
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# A list of ignored prefixes for module index sorting.
#modindex_common_prefix = []
# If true, keep warnings as "system message" paragraphs in the built documents.
#keep_warnings = False
# If true, `todo` and `todoList` produce output, else they produce nothing.
todo_include_todos = True
# -- Options for HTML output ----------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
html_theme = 'sphinx_book_theme'
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
html_theme_options = {
"repository_url": "https://github.com/crazyguitar/pysheeet",
"use_repository_button": True,
}
# Custom sidebar templates
# Add any paths that contain custom themes here, relative to this directory.
#html_theme_path = []
# The name for this set of Sphinx documents. If None, it defaults to
# "<project> v<release> documentation".
html_title = "Python Cheat Sheet"
# A shorter title for the navigation bar. Default is the same as html_title.
#html_short_title = None
# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
html_logo = "_static/logo.svg"
# The name of an image file (within the static path) to use as favicon of the
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
html_favicon = '_static/favicon.ico'
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
html_css_files = ['style.css']
# Add any extra paths that contain custom files (such as robots.txt or
# .htaccess) here, relative to this directory. These files are copied
# directly to the root of the documentation.
html_extra_path = ['_extra']
html_context = {
"tracking_id": os.environ.get("TRACKING_ID"),
}
has_carbonad = os.environ.get("CARBONAD_SERVE") and os.environ.get("CARBONAD_PLACEMENT")
if has_carbonad:
html_context["carbonad_serve"] = os.environ.get("CARBONAD_SERVE")
html_context["carbonad_placement"] = os.environ.get("CARBONAD_PLACEMENT")
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
#html_last_updated_fmt = '%b %d, %Y'
# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
#html_use_smartypants = True
# Additional templates that should be rendered to pages, maps page names to
# template names.
#html_additional_pages = {}
# If false, no module index is generated.
#html_domain_indices = True
# If false, no index is generated.
#html_use_index = True
# If true, the index is split into individual pages for each letter.
#html_split_index = False
# If true, links to the reST sources are added to the pages.
#html_show_sourcelink = True
# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
#html_show_sphinx = True
# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
#html_show_copyright = True
# If true, an OpenSearch description file will be output, and all pages will
# contain a <link> tag referring to it. The value of this option must be the
# base URL from which the finished HTML is served.
#html_use_opensearch = ''
# This is the file name suffix for HTML files (e.g. ".xhtml").
#html_file_suffix = None
# Language to be used for generating the HTML full-text search index.
# Sphinx supports the following languages:
# 'da', 'de', 'en', 'es', 'fi', 'fr', 'hu', 'it', 'ja'
# 'nl', 'no', 'pt', 'ro', 'ru', 'sv', 'tr'
#html_search_language = 'en'
# A dictionary with options for the search language support, empty by default.
# Now only 'ja' uses this config value
#html_search_options = {'type': 'default'}
# The name of a javascript file (relative to the configuration directory) that
# implements a search results scorer. If empty, the default will be used.
#html_search_scorer = 'scorer.js'
# Output file base name for HTML help builder.
htmlhelp_basename = 'python-cheatsheetdoc'
# -- Options for LaTeX output ---------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#'preamble': '',
# Latex figure (float) alignment
#'figure_align': 'htbp',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
(master_doc, 'python-cheatsheet.tex', u'python-cheatsheet Documentation',
u'crazyguitar', 'manual'),
]
# The name of an image file (relative to this directory) to place at the top of
# the title page.
#latex_logo = None
# For "manual" documents, if this is true, then toplevel headings are parts,
# not chapters.
#latex_use_parts = False
# If true, show page references after internal links.
#latex_show_pagerefs = False
# If true, show URL addresses after external links.
#latex_show_urls = False
# Documents to append as an appendix to all manuals.
#latex_appendices = []
# If false, no module index is generated.
#latex_domain_indices = True
# -- Options for manual page output ---------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
(master_doc, 'python-cheatsheet', u'python-cheatsheet Documentation',
[author], 1)
]
# If true, show URL addresses after external links.
#man_show_urls = False
# -- Options for Texinfo output -------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
(master_doc, 'python-cheatsheet', u'python-cheatsheet Documentation',
author, 'python-cheatsheet', 'One line description of project.',
'Miscellaneous'),
]
html_sidebars = {
"**": [
"navbar-logo.html",
"search-button-field.html",
"sbt-sidebar-nav.html",
"carbonad.html",
]
}
# Documents to append as an appendix to all manuals.
#texinfo_appendices = []
# If false, no module index is generated.
#texinfo_domain_indices = True
# How to display URL addresses: 'footnote', 'no', or 'inline'.
#texinfo_show_urls = 'footnote'
# If true, do not generate a @detailmenu in the "Top" node's menu.
#texinfo_no_detailmenu = False
def add_html_link(app, pagename, templatename, context, doctree):
"""Append html page."""
if pagename in ['404', 'search', 'genindex']:
return
app.sitemaps.append({
'pagename': pagename + ".html",
'priority': '1.0' if pagename == 'index' else '0.8',
'changefreq': 'weekly' if pagename == 'index' else 'monthly'
})
def create_sitemap(app, exception):
"""Generate a sitemap.xml"""
from xml.etree.ElementTree import ElementTree, Element, SubElement
from datetime import datetime
r = Element("urlset")
r.set("xmlns", "http://www.sitemaps.org/schemas/sitemap/0.9")
r.set("xmlns:xsi", "http://www.w3.org/2001/XMLSchema-instance")
r.set("xsi:schemaLocation", "http://www.sitemaps.org/schemas/sitemap/0.9" +
" http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd")
for link_info in app.sitemaps:
url = SubElement(r, "url")
now = datetime.now()
SubElement(url, "loc").text = app.pysheeet + link_info['pagename']
SubElement(url, "lastmod").text = now.date().isoformat()
SubElement(url, "changefreq").text = link_info['changefreq']
SubElement(url, "priority").text = link_info['priority']
f = app.outdir + "/sitemap.xml"
t = ElementTree(r)
t.write(f, xml_declaration=True, encoding='utf-8', method="xml")
def setup(app):
"""Customize setup."""
site = os.environ.get("PYSHEEET")
if not site:
return
if site[-1] != '/':
site += '/'
# create a sitemap
app.pysheeet = site
app.sitemaps = []
app.connect('html-page-context', add_html_link)
app.connect('build-finished', create_sitemap)
================================================
FILE: docs/index.rst
================================================
.. python-cheatsheet documentation master file, created by
sphinx-quickstart on Sun Feb 28 09:26:04 2016.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
.. meta::
:description lang=en: Comprehensive Python cheat sheet with practical code snippets, examples, and tutorials for Python developers. Learn Python basics, advanced topics, databases, networking, and more.
:keywords: Python, Python Cheat Sheet, Python Tutorial, Python Examples, Python Code Snippets, Programming, Development, Python Reference, Python Guide
Python Cheat Sheet - Complete Guide with Code Examples
======================================================
Welcome to **pysheeet** - your ultimate Python cheat sheet! This comprehensive resource contains practical Python
code snippets, examples, and tutorials to make coding easier and more efficient for Python developers of all levels.
From basic Python syntax to advanced topics like databases, networking, and multitasking, this cheat sheet serves as your
complete Python reference guide. Ideal for beginners learning Python fundamentals and experienced developers seeking
quick code examples. Whether you're learning Python for web development, data science, automation, or general programming,
you'll find practical examples that save you time and improve your coding efficiency.
Contributions are always welcome—feel free to share ideas for new snippets, improvements, or clearer explanations!
If you'd like to contribute, `fork pysheeet on GitHub`_.
If there is any question or suggestion, please create an issue on `GitHub Issues`_.
.. _fork pysheeet on GitHub: https://github.com/crazyguitar/pysheeet
.. _GitHub Issues: https://github.com/crazyguitar/pysheeet/issues
Plugin
------
**pysheeet** is available as a `Claude Code <https://claude.com/claude-code>`_ plugin. Once installed,
Claude automatically uses the cheat sheets to answer Python questions.
.. code-block:: bash
# Step 1: Add the marketplace
claude plugin marketplace add crazyguitar/pysheeet
# Step 2: Install the plugin
claude plugin install pysheeet@pysheeet
For local testing and manual installation, see the main `README <https://github.com/crazyguitar/pysheeet/blob/master/README.rst>`_.
What's New In Python 3
----------------------
The official document, `What's New In Python`_, displays all of the most
important changes. However, if you're too busy to read the whole changes,
this part provides a brief glance of new features in Python 3.
.. _What's New In Python: https://docs.python.org/3/whatsnew/index.html
.. toctree::
:maxdepth: 1
notes/python-new-py3
Python Cheat Sheet
------------------
This section focuses on commonly used Python code snippets. The cheat sheet
covers not only core Python features but also essential data structures,
algorithms, and frequently used modules to help programmers efficiently tackle
everyday tasks.
.. toctree::
:maxdepth: 1
notes/basic/index
notes/os/index
notes/concurrency/index
notes/asyncio/index
notes/network/index
notes/database/index
notes/security/index
notes/extension/index
notes/llm/index
notes/hpc/index
notes/appendix/index
================================================
FILE: docs/notes/appendix/disaggregated-prefill-decode.rst
================================================
.. meta::
:description lang=en: Evaluating disaggregated prefill/decode for LLM serving with vLLM, NIXL, and EFA on AWS
:keywords: LLM, vLLM, NIXL, disaggregated prefill decode, KV cache, EFA, inference serving
Is Disaggregated Prefill/Decode a Silver Bullet for LLM Serving?
================================================================
:Date: 2026-03-10
Abstract
--------
Disaggregated prefill/decode has gained traction as a promising architecture for
LLM serving, separating the compute-intensive prefill phase from the
memory-bound decode phase onto dedicated node groups. Proponents argue that this
separation enables independent scaling and eliminates interference between the
two phases. But is it truly a silver bullet? This article puts the claim to the
test by evaluating disaggregated prefill/decode using vLLM with NIXL over the
AWS Elastic Fabric Adapter (EFA) on a 4-node cluster. We compare data
parallelism and simple load-balanced routing as baselines against disaggregated
configurations. Our results show that while disaggregation dramatically reduces
inter-token latency (ITL), it comes at a significant cost to throughput and
time-to-first-token (TTFT), revealing that the architecture is far from a
universal solution.
Introduction
------------
In standard LLM serving, each node handles both prefill and decode for incoming
requests. The prefill phase is compute-bound and processes the entire input
prompt in parallel, while the decode phase is memory-bandwidth-bound and
generates tokens autoregressively. When both phases share the same GPU pool,
long prefill requests can block decode iterations, increasing inter-token
latency for concurrent requests.
Disaggregated prefill/decode addresses this interference by assigning prefill
and decode to separate node groups. After a prefill node completes prompt
processing, the KV cache is transferred to a decode node via a high-bandwidth
interconnect. NIXL [1]_ (NVIDIA Inference Xfer Library) provides the KV cache
transfer mechanism, and on AWS, this transfer occurs over EFA using the
``LIBFABRIC`` backend.
The appeal is intuitive: by isolating decode nodes from prefill interference,
token generation should proceed at a steady, low-latency pace. However, this
separation introduces new costs — KV cache transfer overhead, prefill node
saturation at long input lengths, and reduced effective cluster capacity for
each phase. The question is whether these trade-offs are worthwhile compared to
simpler alternatives like data parallelism or stateless load-balanced routing.
This experiment uses vLLM [2]_ with the
``NixlConnector`` to orchestrate disaggregated serving, and ``vllm-router`` [3]_ as
a reverse proxy to load-balance requests across node groups. The experiment
code is available under `src/nixl <https://github.com/crazyguitar/pysheeet/tree/master/src/nixl>`_ in the companion repository.
Container Image
---------------
The experiment uses a custom Docker image that bundles all required components.
The ``Dockerfile`` builds on ``nvidia/cuda:12.8.1-devel-ubuntu24.04`` and
installs the following stack:
- **GDRCopy** v2.5.1 for GPU-direct memory registration
- **EFA installer** v1.47.0 for AWS Elastic Fabric Adapter support
- **UCX** v1.20.0 built with verbs, rdmacm, and EFA transport
- **NIXL** v0.10.1 with ``LIBFABRIC`` backend for KV cache transfer
- **nixlbench** for standalone NIXL bandwidth/latency microbenchmarks
- **PyTorch** 2.9.1, **flash-attn** 2.8.1, and **DeepGEMM** v2.1.1.post3
- **vLLM** 0.15.1 with ``NixlConnector`` support
- **vllm-router** for load-balancing across disaggregated node groups
The image is built and saved as a portable tarball via the ``Makefile``:
.. code-block:: bash
make docker && make save
This produces ``nixl-latest.tar.gz``, which is distributed to all Slurm nodes
at launch time via ``pigz`` decompression and ``docker load``.
Serving Script
--------------
The ``vllm.sbatch`` script orchestrates multi-node vLLM serving on Slurm. It
accepts two key flags that control the serving topology:
- ``--route R``: splits the allocated nodes into ``R`` identical groups, each
running an independent vLLM instance. A ``vllm-router`` process on the head
node round-robins requests across groups.
- ``--prefill P``: within each group, assigns ``P`` nodes as prefill-only
(``kv_producer``) and the remaining nodes as decode-only (``kv_consumer``).
KV cache transfer between prefill and decode nodes uses ``NixlConnector``
with the ``LIBFABRIC`` backend over EFA.
When ``--prefill 0`` (default), all nodes in a group run standard data-parallel
serving. The script computes ``DP = nodes_per_group * (8 / TP)`` and launches
vLLM with ``--data-parallel-size`` accordingly.
For disaggregated mode, each prefill and decode node runs as an independent
vLLM process with explicit KV transfer configuration:
.. code-block:: bash
# Prefill node
vllm serve ... \
--kv-transfer-config.kv_connector NixlConnector \
--kv-transfer-config.kv_role kv_producer \
--kv-transfer-config.kv_connector_extra_config.backends+ LIBFABRIC
# Decode node
vllm serve ... \
--kv-transfer-config.kv_connector NixlConnector \
--kv-transfer-config.kv_role kv_consumer \
--kv-transfer-config.kv_connector_extra_config.backends+ LIBFABRIC
The router uses ``round_robin`` policy for pure-DP groups and
``consistent_hash`` with ``--vllm-pd-disaggregation`` for PD groups, directing
initial requests to prefill endpoints and subsequent decode traffic to decode
endpoints:
.. code-block:: bash
# Router for pure-DP groups (round-robin across group endpoints)
vllm-router \
--policy round_robin \
--worker-urls http://<GROUP0_IP>:8000 http://<GROUP1_IP>:8001 \
--host 0.0.0.0 --port 8010
# Router for PD disaggregation (consistent hash with prefill/decode split)
vllm-router \
--policy consistent_hash \
--vllm-pd-disaggregation \
--prefill http://<PREFILL0_IP>:8000 \
--decode http://<DECODE0_IP>:8001 --decode http://<DECODE1_IP>:8002 \
--host 0.0.0.0 --port 8010
Each container is launched with ``--privileged``, ``--net=host``, and explicit
``/dev/infiniband/uverbs*`` and ``/dev/gdrdrv`` device mounts to enable
GPU-direct RDMA over EFA.
Benchmark Script
----------------
The ``bench.sh`` script wraps ``vllm bench serve`` and handles Docker image
loading transparently. If the ``vllm`` CLI is not available on the host, the
script re-executes itself inside the container. It points the benchmark client
at the router endpoint (or the direct vLLM endpoint for single-group
configurations):
.. code-block:: bash
bash bench.sh -H <ROUTER_IP> -p <ROUTER_PORT> -- \
--model /fsx/models/deepseek-ai/DeepSeek-V2-Lite \
--dataset-name random \
--random-input-len 512 --random-output-len 256 \
--num-prompts 1024
Experimental Setup
------------------
All experiments run on 4 nodes with 8 GPUs each (TP=8) using
DeepSeek-V2-Lite as the model. The benchmark uses random input/output data
with 1024 prompts via ``vllm bench serve``.
The configurations are:
- **Baseline (data parallelism)**: 4 nodes, TP=8, DP=4. All nodes serve both
prefill and decode. This is the standard data-parallel serving setup.
- **Route 2**: 2 groups of 2 nodes each, TP=8, DP=2 per group. A router
round-robins requests across groups. Each group independently handles both
prefill and decode.
- **Route 4**: 4 groups of 1 node each, TP=8, no data parallelism. A router
distributes requests across all 4 independent nodes.
- **PD 1P3D**: Disaggregated prefill/decode with 1 prefill node and 3 decode
nodes. KV cache is transferred from the prefill node to decode nodes via NIXL.
- **PD 2P2D**: Disaggregated prefill/decode with 2 prefill nodes and 2 decode
nodes.
.. code-block:: bash
# Exp 1: Baseline — 4 nodes, TP=8, pure DP
salloc -N 4 bash vllm.sbatch \
--model /fsx/models/deepseek-ai/DeepSeek-V2-Lite \
--gpu-memory-utilization 0.9
# Exp 2: 2 groups × 2 nodes, DP=2 per group, router round-robins
salloc -N 4 bash vllm.sbatch --route 2 \
--model /fsx/models/deepseek-ai/DeepSeek-V2-Lite \
--gpu-memory-utilization 0.9
# Exp 3: 4 groups × 1 node, no DP, router round-robins
salloc -N 4 bash vllm.sbatch --route 4 \
--model /fsx/models/deepseek-ai/DeepSeek-V2-Lite \
--gpu-memory-utilization 0.9
# Exp 4: 1 prefill + 3 decode
salloc -N 4 bash vllm.sbatch --prefill 1 \
--model /fsx/models/deepseek-ai/DeepSeek-V2-Lite \
--gpu-memory-utilization 0.9
# Exp 5: 2 prefill + 2 decode
salloc -N 4 bash vllm.sbatch --prefill 2 \
--model /fsx/models/deepseek-ai/DeepSeek-V2-Lite \
--gpu-memory-utilization 0.9
Results
-------
We evaluate each configuration along four metrics: output token throughput,
request throughput, time to first token (TTFT), and inter-token latency (ITL).
Each plot contains two panels — the left panel sweeps input length with a fixed
output length of 256 tokens (prefill-dominated regime), while the right panel
sweeps output length with a fixed input length of 512 tokens (decode-dominated
regime). This allows us to observe how each configuration behaves when the
workload shifts from prefill-heavy to decode-heavy.
Microbenchmark: KV Cache Transfer Bandwidth
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Before examining end-to-end serving results, we use ``nixlbench`` to measure
the raw NIXL transfer bandwidth over EFA between two nodes. This establishes
an upper bound on KV cache transfer speed and helps contextualize the TTFT
overhead observed in disaggregated configurations.
The benchmark runs in Multi-GPU (MG) mode with all 8 GPUs per node performing
VRAM-to-VRAM transfers over the ``LIBFABRIC`` backend:
.. code-block:: bash
salloc -N 2 bash nixl.sbatch --backend LIBFABRIC \
--initiator_seg_type VRAM --target_seg_type VRAM \
--mode MG --num_initiator_dev 8 --num_target_dev 8
Block Size (B) Batch Size B/W (GB/Sec) Avg Lat. (us) P99 Tx (us)
---------------------------------------------------------------------------------
4096 1 0.670064 6.1 47.0
8192 1 1.315392 6.2 45.0
16384 1 2.511416 6.5 47.0
32768 1 4.820423 6.8 50.0
65536 1 8.733224 7.5 56.0
131072 1 12.341950 10.6 52.0
262144 1 23.272188 11.3 59.0
524288 1 43.365764 12.1 62.0
1048576 1 74.816773 14.0 77.0
2097152 1 121.086563 17.3 105.0
4194304 1 180.631395 23.2 146.0
8388608 1 239.037623 35.1 247.0
16777216 1 289.500030 58.0 432.0
33554432 1 327.436372 102.5 796.0
67108864 1 349.608429 192.0 1724.0
**Mapping to DeepSeek-V2-Lite KV cache transfer.** DeepSeek-V2-Lite uses
Multi-head Latent Attention (MLA), which compresses the KV cache into a latent
vector per token per layer. The per-token-per-layer KV cache size is
``(kv_lora_rank + qk_rope_head_dim) × dtype_size = (512 + 64) × 2 = 1,152 bytes``.
For 512 input tokens across 27 layers, the total KV cache is approximately **15.2 MB**.
With TP=8, each GPU transfers about **1.9 MB**, which falls in the ~121 GB/s
bandwidth range per the table above. Without tensor parallelism, the full 15.2 MB
transfer achieves approximately ~289 GB/s.
Output Token Throughput
~~~~~~~~~~~~~~~~~~~~~~~
.. image:: https://raw.githubusercontent.com/crazyguitar/pysheeet/master/docs/_static/appendix/nixl/throughput.png
:alt: Output token throughput comparison
The left panel varies input length with a fixed output length of 256 tokens
(prefill-dominated), while the right panel varies output length with a fixed
input length of 512 tokens (decode-dominated).
For prefill-dominated workloads, Route 4 achieves the highest throughput since
each node operates independently without the overhead of data parallelism
coordination. The disaggregated configurations (PD 1P3D and PD 2P2D) show
competitive throughput at shorter input lengths but degrade at longer inputs
where the prefill nodes become the bottleneck.
For decode-dominated workloads, Route 4 again leads, followed by PD 1P3D.
PD 2P2D shows the lowest throughput in this regime, as its two decode nodes
cannot match the decode capacity of other configurations.
Request Throughput
~~~~~~~~~~~~~~~~~~
.. image:: https://raw.githubusercontent.com/crazyguitar/pysheeet/master/docs/_static/appendix/nixl/req_throughput.png
:alt: Request throughput comparison
Request throughput follows a similar pattern. Route 4 consistently achieves the
highest request throughput across all configurations. The disaggregated PD 1P3D
configuration maintains reasonable request throughput for short inputs but drops
significantly at longer input lengths (4096 tokens), where the single prefill
node becomes saturated.
Time to First Token (TTFT)
~~~~~~~~~~~~~~~~~~~~~~~~~~
.. image:: https://raw.githubusercontent.com/crazyguitar/pysheeet/master/docs/_static/appendix/nixl/ttft.png
:alt: TTFT comparison
TTFT is critical for user-perceived latency. The baseline DP and Route 2
configurations show moderate TTFT that scales with input length. Route 4
achieves the lowest TTFT across all input lengths due to the absence of
cross-node coordination.
The disaggregated configurations exhibit higher TTFT, particularly at longer
input lengths. PD 1P3D shows TTFT exceeding 37 seconds at 4096 input tokens,
as all prefill work funnels through a single node. PD 2P2D improves on this
but still lags behind the non-disaggregated configurations. The additional
latency from KV cache transfer over NIXL contributes to the elevated TTFT.
For decode-dominated workloads (right panel), the differences are smaller. At
short output lengths (256–512 tokens), PD 1P3D shows 1–2 seconds higher TTFT
than the baseline, as the KV cache transfer overhead is proportionally more
significant. At longer output lengths (1024+ tokens), the disaggregated
configurations converge with or improve upon the baseline, as the baseline
suffers from increased prefill/decode contention under heavier concurrent
decode load.
Inter-Token Latency (ITL)
~~~~~~~~~~~~~~~~~~~~~~~~~
.. image:: https://raw.githubusercontent.com/crazyguitar/pysheeet/master/docs/_static/appendix/nixl/itl.png
:alt: ITL comparison
ITL measures the latency between consecutive generated tokens during the decode
phase. This is where disaggregated serving shows its primary advantage.
In the prefill-dominated regime (left panel), PD 1P3D achieves the lowest ITL
across all input lengths, with mean ITL as low as 10 ms at 4096 input tokens.
By isolating decode nodes from prefill interference, the decode phase runs
uninterrupted. PD 2P2D also shows reduced ITL compared to the baseline, though
the benefit is less pronounced due to having fewer decode nodes. The baseline DP
and Route configurations show higher ITL, particularly at longer input lengths
where prefill and decode contend for the same GPU resources.
In the decode-dominated regime (right panel), Route 4 achieves the lowest ITL
(~25–29 ms) since each node serves independently without cross-node
coordination. Among the disaggregated configurations, PD 1P3D outperforms
PD 2P2D due to its greater decode capacity (3 decode nodes vs. 2), maintaining
ITL around 26–35 ms. PD 2P2D, with only 2 decode nodes, shows ITL comparable
to the baseline (~45–50 ms). As output length increases, ITL gradually rises
across all configurations, reflecting the growing decode load.
Discussion
----------
So, is disaggregated prefill/decode a silver bullet? The answer is clearly no —
at least not under the conditions tested here. All benchmarks use randomly
generated prompts, meaning every request produces a unique KV cache with zero
prefix cache hit rate. This represents a worst-case scenario for disaggregated
serving, where every prefill must be computed from scratch and the full KV cache
must be transferred over the network. In production workloads with shared system
prompts or repeated prefixes, prefix caching on prefill nodes could
substantially reduce redundant computation and transfer volume, potentially
shifting the balance in favor of disaggregation. Even so, the results reveal a
set of sharp trade-offs that make disaggregation a specialized tool rather than
a universal improvement:
- **ITL wins, but throughput depends on scaling**: Disaggregated configurations
deliver dramatically lower inter-token latency — PD 1P3D achieves as low as
10 ms ITL at long input lengths, up to 14× better than the baseline in
prefill-dominated regimes and 1.4–2.4× better in decode-dominated regimes.
The throughput and TTFT degradation observed here is partly an artifact of a
fixed 4-node cluster: dedicating nodes to one role starves the other. In
practice, prefill and decode pools can be scaled independently — adding more
prefill nodes to eliminate the prefill bottleneck, or more decode nodes to
increase token throughput. The challenge is finding the right ratio between
prefill and decode capacity for a given workload, as over-provisioning either
side increases cost without proportional benefit.
- **Prefill bottleneck is a hard constraint**: With a fixed cluster size,
dedicating nodes to prefill reduces decode capacity and vice versa. PD 1P3D
suffers severe prefill saturation at long input lengths (TTFT > 37s at 4096
tokens), while PD 2P2D has fewer decode nodes, limiting decode throughput.
Frameworks such as `NVIDIA Dynamo <https://github.com/ai-dynamo/dynamo>`_
aim to address this by dynamically scaling prefill and decode pools based on
real-time demand, though this adds operational complexity.
- **Simple routing beats disaggregation on throughput**: Route 4 (pure routing,
no DP, no disaggregation) consistently achieves the highest throughput across
all configurations by eliminating cross-node synchronization entirely. It also
achieves the lowest TTFT in prefill-dominated workloads, though PD 1P3D edges
it out on TTFT in decode-dominated regimes where the fixed 512-token input is
short enough to avoid prefill saturation. This is a surprisingly strong
baseline — for workloads where ITL is not the primary concern, stateless
load-balanced independent nodes outperform both data parallelism and
disaggregated configurations.
- **KV cache transfer is not free**: The NIXL transfer over EFA adds measurable
latency to TTFT in disaggregated configurations. This overhead is amortized
for longer decode sequences but is noticeable for short output lengths,
making disaggregation less attractive for short-response workloads.
In summary, disaggregated prefill/decode aims to optimize both TTFT and ITL by
isolating the two phases, but achieving these goals is not guaranteed. KV cache
transfer over the network introduces additional overhead that can negate the
TTFT benefit, particularly at long input lengths where the transfer volume is
large. While ITL improvements are consistently observed due to the elimination
of prefill interference on decode nodes, the overall serving performance depends
heavily on the prefill-to-decode ratio, workload characteristics, and network
bandwidth. Teams considering this architecture should carefully profile their
input/output length distributions, latency SLAs, and throughput requirements
before committing to the added complexity.
References
----------
.. [1] NVIDIA, "NIXL: NVIDIA Inference Xfer Library," GitHub, 2025.
https://github.com/ai-dynamo/nixl
.. [2] vLLM Project, "vLLM: Easy, fast, and cheap LLM serving," GitHub, 2024.
https://github.com/vllm-project/vllm
.. [3] vLLM Project, "vllm-router: Production-ready router for vLLM," GitHub, 2025.
https://github.com/vllm-project/vllm-router
================================================
FILE: docs/notes/appendix/index.rst
================================================
.. meta::
:description lang=en: Python appendix covering advanced topics including the walrus operator (PEP 572) and Python debugging with GDB
:keywords: Python, Python3, walrus operator, PEP 572, GDB, debugging, advanced Python
Blog
----
This section explores advanced programming topics to help users build a deeper
understanding of complex concepts and practical techniques. Programmers working
in other languages, such as C/C++, often use Python as a versatile debugging
tool. With debuggers like GDB, they may write Python scripts to parse memory
regions, improve output readability, or automate troubleshooting tasks.
More advanced topics and examples can be found in the following link.
.. toctree::
:maxdepth: 1
disaggregated-prefill-decode
megatron-efa-monitoring
nccl-gin
python-walrus
python-gdb
================================================
FILE: docs/notes/appendix/megatron-efa-monitoring.rst
================================================
.. meta::
:description lang=en: Monitoring EFA network performance with NCCL GIN and Nsys during distributed LLM training on AWS
:keywords: EFA, NCCL, GIN, Nsys, Megatron-LM, distributed training, network monitoring, AWS
Monitoring EFA with NCCL GIN and Nsys
======================================
:Date: 2026-02-28
Abstract
--------
Distributed training at scale requires deep visibility into network behavior to
identify bottlenecks and optimize communication patterns. When training large
language models with Megatron-LM on AWS infrastructure using the Elastic Fabric
Adapter (EFA), understanding network performance becomes critical for achieving
optimal throughput. This article demonstrates how to enable NCCL GPU-Initiated
Networking (GIN) in Megatron-LM using Megatron Bridge and leverage Nsys with
EFA metrics to monitor network behavior during distributed training workloads.
The techniques presented here are based on best practices from AWS re:Invent
2024 [1]_.
Introduction
------------
`Megatron-LM <https://github.com/NVIDIA/Megatron-LM>`_ is a widely adopted
framework for training large transformer models using model parallelism,
pipeline parallelism, and data parallelism. When deployed on AWS instances with
EFA, the network fabric provides high-bandwidth, low-latency communication
essential for scaling to hundreds or thousands of GPUs. However, achieving peak
performance requires careful tuning and monitoring of the communication layer.
NCCL GPU-Initiated Networking allows GPUs to initiate network operations
directly without CPU involvement, reducing latency and enabling kernel fusion.
Nsys (NVIDIA Nsight Systems) provides comprehensive profiling of GPU kernels,
CUDA API calls, and network operations. When combined with EFA metrics
collection (``--enable efa_metrics``), Nsys captures detailed network adapter
statistics including bandwidth utilization, packet counts, and error rates,
correlated with GPU execution timelines. This enables practitioners to diagnose
performance issues and validate that the network is operating at expected
capacity.
`Megatron Bridge <https://github.com/NVIDIA/Megatron-LM/tree/main/megatron/bridge>`_
simplifies the configuration and deployment of Megatron-LM training jobs by
providing a high-level recipe-based interface. This eliminates the need to
manually construct complex command-line arguments and makes it easier to enable
advanced features like NCCL GIN and DeepEP for MoE models. Therefore, the
tutorial in this article will use Megatron Bridge.
Prerequisites
-------------
This guide assumes the following environment:
- AWS HyperPod or EC2 instances with EFA support (e.g., P5, P5e, P5en)
- NCCL >= v2.29.3-1 with Device API support
- aws-ofi-nccl plugin with GIN support
- Megatron-LM with Megatron-Bridge
We have demonstrated how to use vLLM with NCCL GIN and DeepEP in a previous
article. If you are interested in building NCCL and aws-ofi-nccl from source,
refer to the `NCCL GIN article
<https://www.pythonsheets.com/notes/appendix/nccl-gin.html>`_
in this repository.
Building the Megatron Container
--------------------------------
The Megatron training environment is packaged as a Docker container and
converted to an Enroot squash file for deployment on Slurm clusters. The
container includes NCCL with Device API support, aws-ofi-nccl with GIN support,
and Megatron-LM with Megatron Bridge.
To build the container and create the Enroot image:
.. code-block:: bash
cd src/megatron
make build
This will create a ``megatron-lm+latest.sqsh`` file that can be used with the
Slurm launcher scripts. For details on the container build process, refer to
the `Dockerfile
<https://github.com/crazyguitar/pysheeet/blob/master/src/megatron/Dockerfile>`_
and `enroot.sh
<https://github.com/crazyguitar/pysheeet/blob/master/src/megatron/enroot.sh>`_
scripts in the repository.
Enabling NCCL GIN in Megatron Bridge
-------------------------------------
Megatron Bridge recipes provide a declarative way to configure training jobs.
To enable NCCL GIN for MoE models using DeepEP, the following environment
variables are set automatically by the ``srun.sh`` launcher script:
.. code-block:: bash
export DEEP_EP_BACKEND=nccl
export NCCL_GIN_TYPE=2 # proxy-based GIN
export LD_LIBRARY_PATH=/opt/amazon/ofi-nccl/lib:$LD_LIBRARY_PATH
``NCCL_GIN_TYPE=2`` selects the proxy-based implementation, where a CPU thread
mediates GPU-initiated transfers. This mode is currently supported on EFA,
while GPU Direct Async Kernel-Initiated (DAKI) networking (``NCCL_GIN_TYPE=3``)
is not yet available on AWS at the time of writing (February 2026).
The ``srun.sh`` script also configures additional EFA-specific settings for
optimal performance:
.. code-block:: bash
export FI_PROVIDER=efa
export FI_EFA_USE_DEVICE_RDMA=1
export FI_EFA_FORK_SAFE=1
export NCCL_NET_PLUGIN=/opt/amazon/ofi-nccl/lib/libnccl-net-ofi.so
export NCCL_TUNER_PLUGIN=/opt/amazon/ofi-nccl/lib/libnccl-tuner-ofi.so
export NCCL_BUFFSIZE=8388608
export NCCL_P2P_NET_CHUNKSIZE=524288
Launching Megatron Training with DeepEP and NCCL GIN
-----------------------------------------------------
The following example demonstrates how to launch a DeepSeek-V2-Lite pretraining
job with DeepEP enabled for MoE token dispatching. The recipe configures the
model to use expert parallelism across 64 ranks with NCCL GIN for low-latency
all-to-all communication.
.. code-block:: bash
cd src/megatron
# Allocate 2 nodes on Slurm
salloc -N 2
# Launch DeepSeek-V2-Lite with DeepEP and NCCL GIN
./srun.sh recipes/deepseek_v2_lite_pretrain.py \
hf_path=/fsx/models/deepseek-ai/DeepSeek-V2-Lite \
moe_token_dispatcher_type=deepep \
model.tensor_model_parallel_size=1 \
model.expert_model_parallel_size=64 \
model.sequence_parallel=false
The ``moe_token_dispatcher_type=deepep`` argument enables DeepEP as the MoE
dispatcher backend. Under the hood, the recipe configures the following
settings:
.. code-block:: python
cfg.model.moe_token_dispatcher_type = "flex"
cfg.model.moe_flex_dispatcher_backend = "deepep"
cfg.model.moe_enable_deepep = True
cfg.model.moe_shared_expert_overlap = False
When the training job starts, verify that NCCL initializes with GIN enabled by
checking the logs for Device API initialization messages:
.. code-block:: text
[NCCL] Device API initialized
[NCCL] GIN proxy mode enabled (type=2)
[NCCL Backend] LOW LATENCY MODE: Rank 0 connecting to all ranks
[NCCL Backend] Initialized global rank 0/64
Monitoring EFA with Nsys and EFA Metrics
-----------------------------------------
Nsys (NVIDIA Nsight Systems) provides comprehensive profiling of GPU kernels,
CUDA API calls, and network operations. The ``--enable efa_metrics`` flag
instructs Nsys to collect EFA adapter statistics in real-time from the EFA
device counters (e.g., rdmap113s0, rdmap114s0) at 10Hz sampling rate, including:
- **TX/RX Bandwidth**: Transmit and receive throughput
- **TX/RX Packets**: Packet counts sent and received
- **Error Counters**: Link errors and dropped packets
Additionally, aws-ofi-nccl uses NVTX annotations to mark NCCL operations in the
timeline, allowing correlation between NCCL collective calls and EFA network
activity. These metrics are embedded in the Nsys timeline and correlated with
GPU kernel execution and NCCL operations, making it easy to identify
communication bottlenecks and validate network saturation.
To profile a Megatron training run with Nsys and capture EFA metrics:
.. code-block:: bash
cd src/megatron
salloc -N 8
./srun.sh --nsys recipes/deepseek_v2_lite_pretrain.py \
hf_path=/fsx/models/deepseek-ai/DeepSeek-V2-Lite \
moe_token_dispatcher_type=deepep \
model.tensor_model_parallel_size=1 \
model.expert_model_parallel_size=64 \
model.sequence_parallel=false \
profiling.use_nsys_profiler=true \
profiling.profile_step_start=10 \
profiling.profile_step_end=15 \
profiling.profile_ranks=[0]
The ``--nsys`` flag enables Nsys profiling with the following configuration:
.. code-block:: bash
nsys profile \
-t cuda,nvtx \
-s none \
--cpuctxsw=none \
--capture-range=cudaProfilerApi \
--capture-range-end=stop \
--enable efa_metrics \
-o nsys-megatron/profile-<hostname>-rank<RANK>.nsys-rep \
--force-overwrite=true
The ``--enable efa_metrics`` flag is the key parameter that enables EFA adapter
monitoring. Nsys will automatically detect all EFA devices (typically
``rdmap182s0``, ``rdmap183s0``, etc.) and collect statistics at regular intervals
throughout the profiling session.
After profiling completes, the ``.nsys-rep`` files can be downloaded and opened
in Nsight Systems GUI for analysis. The EFA metrics appear as additional rows
in the timeline view, showing bandwidth and packet rate correlated with GPU
kernel execution and NCCL collective operations.
.. image:: https://raw.githubusercontent.com/crazyguitar/pysheeet/master/docs/_static/appendix/deepep-nsys.png
Profiling with Viztracer
-------------------------
For Python-level profiling of the training loop, Megatron Bridge supports
Viztracer, a low-overhead tracing tool that captures function calls and timing
information. This is useful for identifying CPU bottlenecks in data loading,
preprocessing, or scheduler logic that may indirectly impact network
performance.
.. code-block:: bash
salloc -N 2
./srun.sh recipes/deepseek_v2_lite_pretrain.py \
hf_path=/fsx/models/deepseek-ai/DeepSeek-V2-Lite \
train.train_iters=100 \
profiling.use_viztracer=true \
profiling.profile_step_start=10 \
profiling.profile_step_end=15 \
profiling.profile_ranks=[0]
The resulting ``.json`` trace files can be visualized in the Viztracer web UI
or Chrome's ``chrome://tracing`` interface. By enabling ``log_torch``, Viztracer
can capture additional PyTorch-level details such as NCCL stream and CUDA stream
operations, providing visibility into the execution flow of collective
communications and GPU kernels. However, to observe detailed EFA adapter
statistics (bandwidth, packet counts, error counters), Nsys with
``--enable efa_metrics`` remains the required tool.
Conclusion
----------
Nsys profiling with ``--enable efa_metrics`` now provides the capability to
monitor both EFA adapter behavior and NCCL operations simultaneously during
distributed training. This visibility is essential for diagnosing whether long
NCCL operation times are caused by actual EFA transmission delays or other
issues such as CPU bottlenecks, memory contention, or suboptimal NCCL
configuration. By examining the correlated timeline of GPU kernels, NCCL
collectives, and EFA bandwidth utilization, practitioners can pinpoint the root
cause of performance bottlenecks and validate that the network fabric is
operating at expected capacity.
In this article, we demonstrated this monitoring approach using Megatron-LM
with NCCL GIN and DeepEP as an example. The recipe-based approach of Megatron
Bridge simplifies the deployment of complex training configurations, making it
easier to adopt advanced features like DeepEP and NCCL GIN for large-scale MoE
model training while maintaining full observability into network performance.
For complete examples and scripts, refer to the `megatron directory
<https://github.com/crazyguitar/pysheeet/blob/master/src/megatron/>`_ in this
repository.
References
----------
.. [1] `AWS re:Invent 2024 - CMP335: Drilling down into performance for distributed training <https://d1.awsstatic.com/onedam/marketing-channels/website/aws/en_US/events/approved/reinvent-2025/reinvent/2024/slides/cmp/CMP335_Drilling-down-into-performance-for-distributed-training.pdf>`_
================================================
FILE: docs/notes/appendix/nccl-gin.rst
================================================
.. meta::
:description lang=en: Enabling GPU-Initiated Networking for NCCL with DeepEP on AWS using EFA
:keywords: NCCL, GIN, GPU-Initiated Networking, DeepEP, EFA, AWS, MoE, HyperPod
GPU-Initiated Networking for NCCL on AWS
========================================
:Date: 2026-02-22
Abstract
--------
GPU-Initiated Networking (GIN) has attracted significant attention as a key
enabler for kernel fusion in large language model (LLM) training and inference.
Mixture-of-Experts (MoE) architectures, such as DeepSeek-V3 and Qwen3-30B,
require efficient token dispatching and combining across MoE layers.
Conventionally, inter-GPU communication is initiated by the CPU through
collective libraries such as NCCL or Gloo, necessitating explicit GPU
synchronization barriers and additional ``cudaLaunchKernel`` calls that
introduce non-trivial overhead. GPU-Initiated Networking eliminates this
CPU-mediated round-trip by allowing data exchange to occur directly within CUDA
kernels, thereby enabling kernel fusion and efficient CUDA Graph capture for
accelerating end-to-end LLM layer computation. This article demonstrates how to
enable NCCL GIN with DeepEP on AWS HyperPod Slurm using the AWS Elastic Fabric
Adapter (EFA).
Introduction
------------
Prior to 2026, adopting DeepEP as a Mixture-of-Experts dispatch and combine
backend on AWS presented a significant challenge. The DeepEP kernel was
originally built on top of InfiniBand with a customized NVSHMEM implementation,
a transport layer unavailable on AWS infrastructure. This incompatibility
effectively prevented users from leveraging DeepEP on instances equipped with
the Elastic Fabric Adapter (EFA). Recent collaborative efforts by NVIDIA and
Amazon Annapurna Labs have addressed this gap by introducing GPU-Initiated
Networking support in NCCL and the EFA provider, enabling DeepEP to operate
over EFA without relying on InfiniBand (see `DeepEP PR #521
<https://github.com/deepseek-ai/DeepEP/pull/521>`_ and `aws-ofi-nccl PR #1069
<https://github.com/aws/aws-ofi-nccl/pull/1069>`_). The following experiment
builds upon these contributions to illustrate how to deploy DeepEP with NCCL
GIN on AWS using EFA.
Build DeepEP
------------
Before deploying DeepEP on AWS HyperPod Slurm, several components must be built
from source. First, NCCL >= v2.29.3-1 is required, as this is the minimum
version that exposes the Device API needed for GPU-Initiated Networking. The
build targets ``sm_90`` (NVIDIA H100) and ``sm_100`` (NVIDIA B200) compute
capabilities to ensure compatibility with current-generation GPU instances.
.. code-block:: bash
NCCL_VERSION=v2.29.3-1
git clone -b ${NCCL_VERSION} https://github.com/NVIDIA/nccl.git /opt/nccl \
&& cd /opt/nccl \
&& make -j $(nproc) src.build CUDA_HOME=/usr/local/cuda \
NVCC_GENCODE="-gencode=arch=compute_90,code=sm_90 -gencode=arch=compute_100,code=sm_100"
Optionally, the NCCL Device API examples can be built to verify that
GPU-initiated communication functions correctly in the target environment. In
addition, the latest release of nccl-tests (v2.17.9) ships with a GIN-enabled
microbenchmark for the ``alltoall`` collective, which is useful for validating
inter-GPU bandwidth and latency before running full-scale MoE workloads (see
`nccl-tests alltoall.cu <https://github.com/NVIDIA/nccl-tests/blob/v2.17.9/src/alltoall.cu#L214-L291>`_).
.. code-block:: bash
## Build NCCL Device API examples
cd /opt/nccl/examples/06_device_api \
&& make -j $(nproc) NCCL_HOME=/opt/nccl/build CUDA_HOME=/usr/local/cuda MPI=1 MPI_HOME=/opt/amazon/openmpi
NCCL_TESTS_VERSION=v2.17.9
git clone -b ${NCCL_TESTS_VERSION} https://github.com/NVIDIA/nccl-tests.git /opt/nccl-tests \
&& cd /opt/nccl-tests \
&& make -j $(nproc) \
MPI=1 \
MPI_HOME=/opt/amazon/openmpi/ \
CUDA_HOME=/usr/local/cuda \
NCCL_HOME=/opt/nccl/build \
NVCC_GENCODE="-gencode=arch=compute_90,code=sm_90 -gencode=arch=compute_100,code=sm_100"
To test DeepEP on HyperPod Slurm, both DeepEP and aws-ofi-nccl must be pinned
to specific commits that include the NCCL GIN transport path. The DeepEP fork
by Aamir Shafi introduces an NCCL-based communication backend as an alternative
to the original NVSHMEM/InfiniBand path, while the aws-ofi-nccl plugin provides
the libfabric-to-NCCL translation layer required for EFA. Note that the NCCL
GIN implementation has since been merged into the aws-ofi-nccl main branch; the
commit hash is pinned here for reproducibility.
.. code-block:: bash
## Install DeepEP with NCCL GIN backend (PR #521)
unset NVSHMEM_DIR NVSHMEM_HOME \
&& export ENABLE_NCCL=1 \
&& export NCCL_DIR=/opt/nccl/build \
&& export LD_LIBRARY_PATH=/opt/nccl/build/lib:$LD_LIBRARY_PATH \
&& export LD_PRELOAD=/opt/nccl/build/lib/libnccl.so.2 \
&& git clone -b nccl https://github.com/aamirshafi/DeepEP.git /opt/DeepEP \
&& cd /opt/DeepEP \
&& git checkout 6d29f34 \
&& python3 setup.py build_ext --inplace \
&& pip install --break-system-packages --no-build-isolation .
AWS_OFI_NCCL_VERSION=5f4202f11db1585d878196db4430aeda0e834a0c
git clone https://github.com/aws/aws-ofi-nccl.git /tmp/aws-ofi-nccl \
&& cd /tmp/aws-ofi-nccl \
&& git checkout ${AWS_OFI_NCCL_VERSION} \
&& ./autogen.sh \
&& ./configure --prefix=/opt/amazon/ofi-nccl \
--with-libfabric=/opt/amazon/efa \
--with-cuda=/usr/local/cuda \
&& make -j$(nproc) \
&& make install \
&& rm -rf /tmp/aws-ofi-nccl
For a complete build with all necessary dependencies, refer to the
`Dockerfile <https://github.com/crazyguitar/pysheeet/blob/master/src/gin/Dockerfile>`_
provided in this repository.
Test NCCL GIN
-------------
With the Docker image (or Enroot squash file) prepared in the previous section,
NCCL GIN functionality can be validated on a Slurm cluster. The following
examples demonstrate how to launch the NCCL Device API samples and nccl-tests
benchmarks. The corresponding Slurm wrapper scripts are available under the
`gin <https://github.com/crazyguitar/pysheeet/blob/master/src/gin/>`_ directory
in this repository.
.. code-block:: bash
make docker && make save # build a docker image and import an Enroot squash file
# 01_allreduce_lsa (single node only)
salloc -N 1 ./run.enroot /opt/nccl/examples/06_device_api/01_allreduce_lsa/allreduce_lsa
# 01_allreduce_lsa (multi-node) — requires MNNVL (e.g. P6e-GB200), does NOT work over RDMA/EFA
salloc -N 2 ./run.enroot /opt/nccl/examples/06_device_api/01_allreduce_lsa/allreduce_lsa
# 02_alltoall_gin (multi-node)
salloc -N 2 ./run.enroot /opt/nccl/examples/06_device_api/02_alltoall_gin/alltoall_gin
# 03_alltoall_hybrid (multi-node)
salloc -N 2 ./run.enroot /opt/nccl/examples/06_device_api/03_alltoall_hybrid/alltoall_hybrid
The nccl-tests ``alltoall`` benchmark exposes two critical flags for selecting
the GIN transport mode and memory registration strategy:
The ``-D`` flag selects the device-side implementation for the ``alltoall``
collective:
.. code-block:: text
-D 0 — Host API (default)
-D 1 — NVL simple (LSA/NVLink only)
-D 2 — NVL optimized (LSA/NVLink only)
-D 3 — GIN only (network)
-D 4 — Hybrid (LSA intra-node + GIN inter-node)
The ``-R`` flag controls memory registration. Symmetric memory allocation
(``NCCL_MEM_SHARED``) is required for any device-side implementation
(``-D > 0``), as it maps GPU memory across all ranks to enable direct
remote read and write over the network:
.. code-block:: text
-R 0 — no registration (default)
-R 1 — register memory with ncclMemAlloc
-R 2 — register memory with symmetric memory allocation (NCCL_MEM_SHARED)
The following examples launch the nccl-tests ``alltoall_perf`` benchmark in
GIN-only mode (``-D 3``) and hybrid mode (``-D 4``), sweeping message sizes
from 32 MB to 2048 MB. The ``--blocking 0`` flag enables non-blocking
collectives, which is representative of how MoE layers overlap communication
with computation in production workloads:
.. code-block:: bash
# alltoall_perf with GIN (-D 3)
salloc -N 2 ./run.enroot /opt/nccl-tests/build/alltoall_perf \
-D 3 -R 2 -b 32M -e 2048M -f 2 -n 1000 -w 10 --blocking 0
# alltoall_perf with Hybrid LSA+GIN (-D 4)
salloc -N 2 ./run.enroot /opt/nccl-tests/build/alltoall_perf \
-D 4 -R 2 -b 32M -e 2048M -f 2 -n 1000 -w 10 --blocking 0
Serving MoE Models with vLLM and DeepEP over NCCL GIN
-----------------------------------------------------
With NCCL GIN and EFA validated on AWS HyperPod Slurm, this section
demonstrates an end-to-end inference deployment using vLLM with DeepEP as the
MoE all-to-all communication backend. DeepEP's low-latency dispatch and combine
kernels, now operating over NCCL GIN rather than NVSHMEM, enable efficient
expert-parallel inference for large MoE models such as DeepSeek-V3.
The Slurm launch script ``run.sbatch`` is the same one used to launch a vLLM
server in the `vllm example directory
<https://github.com/crazyguitar/pysheeet/blob/master/src/llm/vllm/>`_. However,
to direct the DeepEP backend to use NCCL GIN, the following environment
variables must be set at launch time:
.. code-block:: bash
DEEP_EP_BACKEND=nccl
NCCL_GIN_TYPE=2 # proxy-based GIN
``NCCL_GIN_TYPE=2`` selects the proxy-based GIN path, in which a CPU-side proxy
thread mediates network transfers on behalf of the GPU. ``NCCL_GIN_TYPE=3``
would enable GPU Direct Async Kernel-Initiated (DAKI) networking, which
bypasses the proxy entirely; however, DAKI is not yet supported on AWS with EFA
at the time of writing.
For additional details on serving configurations and benchmarking, refer to
`llm-serving.rst
<https://github.com/crazyguitar/pysheeet/blob/master/docs/notes/llm/llm-serving.rst>`_
or the `vLLM README
<https://github.com/crazyguitar/pysheeet/blob/master/src/llm/vllm/README.rst>`_.
The following example launches a multi-node vLLM inference server for
DeepSeek-V3-0324 with expert parallelism enabled and the DeepEP low-latency
all-to-all backend:
.. code-block:: bash
IMAGE="${PWD}/src/gin/nccl+latest.tar.gz"
MODEL="/fsx/models/deepseek-ai/DeepSeek-V3-0324"
salloc -N 4 bash run.sbatch "${MODEL}" \
--image "${IMAGE}" \
--all2all-backend deepep_low_latency \
--tensor-parallel-size 8 \
--enable-expert-parallel \
--gpu-memory-utilization 0.8 \
--enforce-eager
Upon successful launch, the vLLM server logs confirm that DeepEP is active as
the all-to-all backend and that NCCL GIN is being used for inter-GPU
communication. The key indicators are the ``DeepEPLLAll2AllManager`` manager
selection and the ``[NCCL Backend]`` initialization messages showing
communicator setup, symmetric memory allocation, and window registration across
all ranks:
.. code-block:: bash
...
INFO 02-22 19:06:49 [serve.py:100] Defaulting api_server_count to data_parallel_size (4).
INFO 02-22 19:06:49 [utils.py:325]
INFO 02-22 19:06:49 [utils.py:325] █ █ █▄ ▄█
INFO 02-22 19:06:49 [utils.py:325] ▄▄ ▄█ █ █ █ ▀▄▀ █ version 0.15.1
INFO 02-22 19:06:49 [utils.py:325] █▄█▀ █ █ █ █ model /fsx/models/deepseek-ai/DeepSeek-V3-0324
INFO 02-22 19:06:49 [utils.py:325] ▀▀ ▀▀▀▀▀ ▀▀▀▀▀ ▀ ▀
INFO 02-22 19:06:49 [utils.py:325]
...
INFO 02-22 19:07:51 [cuda_communicator.py:124] Using DeepEPLLAll2AllManager all2all manager.
...
[NCCL Backend] LOW LATENCY MODE: Rank 0 connecting to all 32 ranks
[NCCL Backend] NCCL version: 2.29.3 (loaded library)
[NCCL Backend] Initializing 2 communicator(s) (qps_per_rank=8) for rank 0/32
[NCCL Backend] Rank 0 successfully initialized 2 communicator(s)
[NCCL Backend] Rank 0 created 2 device communication(s) with 32 barrier sessions each
[NCCL Backend] Initialized global rank 0/32 (comm rank 0/32)
[NCCL Backend - Memory Alloc] Rank 0: Allocated ptr=0xf882000000, size=3816818816
[NCCL Backend - Memory Register] Rank 0: Copying 2 NCCL windows to GPU
[NCCL Backend - Memory Register] Rank 0: Successfully copied windows to GPU
[NCCL Backend - Memory Register] Rank 0: Registered windows for ptr=0xf882000000, size=3816818816
Once the server is ready, inference requests can be issued via the
OpenAI-compatible completions API:
.. code-block:: bash
curl -sf -X POST http://<VLLM_HOST>:8000/v1/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "/fsx/models/deepseek-ai/DeepSeek-V3-0324",
"prompt": "Hello",
"max_tokens": 10
}'
# output
{"id":"cmpl-b6e9530a07561f11","object":"text_completion" ... }
Conclusion
----------
This article has demonstrated how to deploy vLLM with DeepEP and NCCL GIN on
AWS HyperPod Slurm using the Elastic Fabric Adapter. As this integration is
still under active development, certain limitations remain at the time of
writing. For instance, although DeepEP's low-latency mode supports CUDA Graph
capture, enabling it by removing ``--enforce-eager`` currently results in a
startup failure in vLLM. Additionally, performance over EFA may not yet match
that of InfiniBand-based deployments, as further optimizations are ongoing.
This article is intended as an early reference for evaluating DeepEP with NCCL
GIN on AWS. For production workloads, it is advisable to wait for official
stable releases from NVIDIA and Amazon Annapurna Labs.
================================================
FILE: docs/notes/appendix/python-gdb.rst
================================================
.. meta::
:description lang=en: Python interpreter in GNU Debugger (GDB)
:keywords: Python, Python3, GDB
==================================
Python Interpreter in GNU Debugger
==================================
:Date: 2025-08-30
Abstract
--------
The GNU Debugger (GDB) is the most powerful debugging tool for developers to
troubleshoot errors in their code. However, it is hard for beginners to learn,
and that is why many programmers prefer to insert ``print`` to examine runtime
status. Fortunately, `GDB Text User Interface (TUI)`_ provides a way for
developers to review their source code and debug simultaneously. More
excitingly, In GDB 7, **Python Interpreter** was built into GDB. This feature
offers more straightforward ways to customize GDB printers and commands through
the Python library. By discussing examples, this article tries to explore
advanced debugging techniques via Python to develop tool kits for GDB.
Introduction
------------
Troubleshooting software bugs is a big challenge for developers. While GDB
provides many “debug commands” to inspect programs’ runtime status, its
non-intuitive usages impede programmers to use it to solve problems. Indeed,
mastering GDB is a long-term process. However, a quick start is not complicated;
you must unlearn what you have learned like Yoda. To better understand how to
use Python in GDB, this article will focus on discussing Python interpreter in
GDB.
Define Commands
---------------
GDB supports customizing commands by using ``define``. It is useful to run a
batch of commands to troubleshoot at the same time. For example, a developer
can display the current frame information by defining a ``sf`` command.
.. code-block:: bash
# define in .gdbinit
define sf
where # find out where the program is
info args # show arguments
info locals # show local variables
end
However, writing a user-defined command may be inconvenient due to limited APIs.
Fortunately, by interacting with Python interpreter in GDB, developers can
utilize Python libraries to establish their debugging tool kits readily. The
following sections show how to use Python to simplify debugging processes.
Dump Memory
-----------
Inspecting a process’s memory information is an effective way to troubleshoot
memory issues. Developers can acquire memory contents by ``info proc mappings``
and ``dump memory``. To simplify these steps, defining a customized command is
useful. However, the implementation is not straightforward by using pure GDB
syntax. Even though GDB supports conditions, processing output is not intuitive.
To solve this problem, using Python API in GDB would be helpful because Python
contains many useful operations for handling strings.
.. code-block:: python
# mem.py
import gdb
import time
import re
class DumpMemory(gdb.Command):
"""Dump memory info into a file."""
def __init__(self):
super().__init__("dm", gdb.COMMAND_USER)
def get_addr(self, p, tty):
"""Get memory addresses."""
cmd = "info proc mappings"
out = gdb.execute(cmd, tty, True)
addrs = []
for l in out.split("\n"):
if re.match(f".*{p}*", l):
s, e, *_ = l.split()
addrs.append((s, e))
return addrs
def dump(self, addrs):
"""Dump memory result."""
if not addrs:
return
for s, e in addrs:
f = int(time.time() * 1000)
gdb.execute(f"dump memory {f}.bin {s} {e}")
def invoke(self, args, tty):
try:
# cat /proc/self/maps
addrs = self.get_addr(args, tty)
# dump memory
self.dump(addrs)
except Exception as e:
print("Usage: dm [pattern]")
DumpMemory()
Running the ``dm`` command will invoke ``DumpMemory.invoke``. By sourcing
or implementing Python scripts in *.gdbinit*, developers can utilize
user-defined commands to trace bugs when a program is running. For example, the
following steps show how to invoke ``DumpMemory`` in GDB.
.. code-block:: bash
(gdb) start
...
(gdb) source mem.py # source commands
(gdb) dm stack # dump stack to ${timestamp}.bin
(gdb) shell ls # ls current dir
1577283091687.bin a.cpp a.out mem.py
Dump JSON
---------
Parsing JSON is helpful when a developer is inspecting a JSON string in a
running program. GDB can parse a ``std::string`` via ``gdb.parse_and_eval``
and return it as a ``gdb.Value``. By processing ``gdb.Value``, developers can
pass a JSON string into Python ``json`` API and print it in a pretty format.
.. code-block:: python
# dj.py
import gdb
import re
import json
class DumpJson(gdb.Command):
"""Dump std::string as a styled JSON."""
def __init__(self):
super().__init__("dj", gdb.COMMAND_USER)
def get_json(self, args):
"""Parse std::string to JSON string."""
ret = gdb.parse_and_eval(args)
typ = str(ret.type)
if re.match("^std::.*::string", typ):
return json.loads(str(ret))
return None
def invoke(self, args, tty):
try:
# string to json string
s = self.get_json(args)
# json string to object
o = json.loads(s)
print(json.dumps(o, indent=2))
except Exception as e:
print(f"Parse json error! {args}")
DumpJson()
The command ``dj`` displays a more readable JSON format in GDB. This command
helps improve visual recognization when a JSON string large. Also, by using
this command, it can detect or monitor whether a ``std::string`` is JSON or
not.
.. code-block:: bash
(gdb) start
(gdb) list
1 #include <string>
2
3 int main(int argc, char *argv[])
4 {
5 std::string json = R"({"foo": "FOO","bar": "BAR"})";
6 return 0;
7 }
...
(gdb) ptype json
type = std::string
(gdb) p json
$1 = "{\"foo\": \"FOO\",\"bar\": \"BAR\"}"
(gdb) source dj.py
(gdb) dj json
{
"foo": "FOO",
"bar": "BAR"
}
Highlight Syntax
----------------
Syntax highlighting is useful for developers to trace source code or to
troubleshoot issues. By using `Pygments`_, applying color to the source is easy
without defining ANSI escape code manually. The following example shows how to
apply color to the ``list`` command output.
.. code-block:: python
import gdb
from pygments import highlight
from pygments.lexers import CLexer
from pygments.formatters import TerminalFormatter
class PrettyList(gdb.Command):
"""Print source code with color."""
def __init__(self):
super().__init__("pl", gdb.COMMAND_USER)
self.lex = CLexer()
self.fmt = TerminalFormatter()
def invoke(self, args, tty):
try:
out = gdb.execute(f"l {args}", tty, True)
print(highlight(out, self.lex, self.fmt))
except Exception as e:
print(e)
PrettyList()
Tracepoints
-----------
Although a developer can insert ``printf``, ``std::cout``, or ``syslog`` to
inspect functions, printing messages is not an effective way to debug when a
project is enormous. Developers may waste their time in building source code
and may acquire little information. Even worse, the output may become too much
to detect problems. In fact, inspecting functions or variables do not require
to embed *print functions* in code. By writing a Python script with GDB API,
developers can customize watchpoints to trace issues dynamically at runtime.
For example, by implementing a ``gdb.Breakpoint`` and a ``gdb.Command``, it is
useful for developers to acquire essential information, such as parameters,
call stacks, or memory usage.
.. code-block:: python
# tp.py
import gdb
tp = {}
class Tracepoint(gdb.Breakpoint):
def __init__(self, *args):
super().__init__(*args)
self.silent = True
self.count = 0
def stop(self):
self.count += 1
frame = gdb.newest_frame()
block = frame.block()
sym_and_line = frame.find_sal()
framename = frame.name()
filename = sym_and_line.symtab.filename
line = sym_and_line.line
# show tracepoint info
print(f"{framename} @ {filename}:{line}")
# show args and vars
for s in block:
if not s.is_argument and not s.is_variable:
continue
typ = s.type
val = s.value(frame)
size = typ.sizeof
name = s.name
print(f"\t{name}({typ}: {val}) [{size}]")
# do not stop at tracepoint
return False
class SetTracepoint(gdb.Command):
def __init__(self):
super().__init__("tp", gdb.COMMAND_USER)
def invoke(self, args, tty):
try:
global tp
tp[args] = Tracepoint(args)
except Exception as e:
print(e)
def finish(event):
for t, p in tp.items():
c = p.count
print(f"Tracepoint '{t}' Count: {c}")
gdb.events.exited.connect(finish)
SetTracepoint()
Instead of inserting ``std::cout`` at the beginning of functions, using a
tracepoint at a function's entry point provides useful information to inspect
arguments, variables, and stacks. For instance, by setting a tracepoint at
``fib``, it is helpful to examine memory usage, stack, and the number of calls.
.. code-block:: cpp
int fib(int n)
{
if (n < 2) {
return 1;
}
return fib(n-1) + fib(n-2);
}
int main(int argc, char *argv[])
{
fib(3);
return 0;
}
The following output shows the result of an inspection of the function ``fib``.
In this case, tracepoints display all information a developer needs, including
arguments' value, recursive flow, and variables' size. By using tracepoints,
developers can acquire more useful information comparing with ``std::cout``.
.. code-block:: bash
(gdb) source tp.py
(gdb) tp main
Breakpoint 1 at 0x647: file a.cpp, line 12.
(gdb) tp fib
Breakpoint 2 at 0x606: file a.cpp, line 3.
(gdb) r
Starting program: /root/a.out
main @ a.cpp:12
argc(int: 1) [4]
argv(char **: 0x7fffffffe788) [8]
fib @ a.cpp:3
n(int: 3) [4]
fib @ a.cpp:3
n(int: 2) [4]
fib @ a.cpp:3
n(int: 1) [4]
fib @ a.cpp:3
n(int: 0) [4]
fib @ a.cpp:3
n(int: 1) [4]
[Inferior 1 (process 5409) exited normally]
Tracepoint 'main' Count: 1
Tracepoint 'fib' Count: 5
Profiling
---------
Without inserting timestamps, profiling is still feasible through tracepoints.
By using a ``gdb.FinishBreakpoint`` after a ``gdb.Breakpoint``, GDB sets a
temporary breakpoint at the return address of a frame for developers to get
the current timestamp and to calculate the time difference. Note that profiling
via GDB is not precise. Other tools, such as `Linux perf`_ or `Valgrind`_,
provide more useful and accurate information to trace performance issues.
.. code-block:: python
import gdb
import time
class EndPoint(gdb.FinishBreakpoint):
def __init__(self, breakpoint, *a, **kw):
super().__init__(*a, **kw)
self.silent = True
self.breakpoint = breakpoint
def stop(self):
# normal finish
end = time.time()
start, out = self.breakpoint.stack.pop()
diff = end - start
print(out.strip())
print(f"\tCost: {diff}")
return False
class StartPoint(gdb.Breakpoint):
def __init__(self, *a, **kw):
super().__init__(*a, **kw)
self.silent = True
self.stack = []
def stop(self):
start = time.time()
# start, end, diff
frame = gdb.newest_frame()
sym_and_line = frame.find_sal()
func = frame.function().name
filename = sym_and_line.symtab.filename
line = sym_and_line.line
block = frame.block()
args = []
for s in block:
if not s.is_argument:
continue
name = s.name
typ = s.type
val = s.value(frame)
args.append(f"{name}: {val} [{typ}]")
# format
out = ""
out += f"{func} @ {filename}:{line}\n"
for a in args:
out += f"\t{a}\n"
# append current status to a breakpoint stack
self.stack.append((start, out))
EndPoint(self, internal=True)
return False
class Profile(gdb.Command):
def __init__(self):
super().__init__("prof", gdb.COMMAND_USER)
def invoke(self, args, tty):
try:
StartPoint(args)
except Exception as e:
print(e)
Profile()
The following output shows the profiling result by setting a tracepoint at the
function ``fib``. It is convenient to inspect the function's performance and
stack at the same time.
.. code-block:: bash
(gdb) source prof.py
(gdb) prof fib
Breakpoint 1 at 0x606: file a.cpp, line 3.
(gdb) r
Starting program: /root/a.out
fib(int) @ a.cpp:3
n: 1 [int]
Cost: 0.0007786750793457031
fib(int) @ a.cpp:3
n: 0 [int]
Cost: 0.002572298049926758
fib(int) @ a.cpp:3
n: 2 [int]
Cost: 0.008517265319824219
fib(int) @ a.cpp:3
n: 1 [int]
Cost: 0.0014069080352783203
fib(int) @ a.cpp:3
n: 3 [int]
Cost: 0.01870584487915039
Pretty Print
------------
Although ``set print pretty on`` in GDB offers a better format to inspect
variables, developers may require to parse variables' value for readability.
Take the system call ``stat`` as an example. While it provides useful information
to examine file attributes, the output values, such as the permission, may not
be readable for debugging. By implementing a user-defined pretty print,
developers can parse ``struct stat`` and output information in a readable format.
.. code-block:: python
import gdb
import pwd
import grp
import stat
import time
from datetime import datetime
class StatPrint:
def __init__(self, val):
self.val = val
def get_filetype(self, st_mode):
if stat.S_ISDIR(st_mode):
return "directory"
if stat.S_ISCHR(st_mode):
return "character device"
if stat.S_ISBLK(st_mode):
return "block device"
if stat.S_ISREG:
return "regular file"
if stat.S_ISFIFO(st_mode):
return "FIFO"
if stat.S_ISLNK(st_mode):
return "symbolic link"
if stat.S_ISSOCK(st_mode):
return "socket"
return "unknown"
def get_access(self, st_mode):
out = "-"
info = ("r", "w", "x")
perm = [
(stat.S_IRUSR, stat.S_IWUSR, stat.S_IXUSR),
(stat.S_IRGRP, stat.S_IRWXG, stat.S_IXGRP),
(stat.S_IROTH, stat.S_IWOTH, stat.S_IXOTH),
]
for pm in perm:
for c, p in zip(pm, info):
out += p if st_mode & c else "-"
return out
def get_time(self, st_time):
tv_sec = int(st_time["tv_sec"])
return datetime.fromtimestamp(tv_sec).isoformat()
def to_string(self):
st = self.val
st_ino = int(st["st_ino"])
st_mode = int(st["st_mode"])
st_uid = int(st["st_uid"])
st_gid = int(st["st_gid"])
st_size = int(st["st_size"])
st_blksize = int(st["st_blksize"])
st_blocks = int(st["st_blocks"])
st_atim = st["st_atim"]
st_mtim = st["st_mtim"]
st_ctim = st["st_ctim"]
out = "{\n"
out += f"Size: {st_size}\n"
out += f"Blocks: {st_blocks}\n"
out += f"IO Block: {st_blksize}\n"
out += f"Inode: {st_ino}\n"
out += f"Access: {self.get_access(st_mode)}\n"
out += f"File Type: {self.get_filetype(st_mode)}\n"
out += f"Uid: ({st_uid}/{pwd.getpwuid(st_uid).pw_name})\n"
out += f"Gid: ({st_gid}/{grp.getgrgid(st_gid).gr_name})\n"
out += f"Access: {self.get_time(st_atim)}\n"
out += f"Modify: {self.get_time(st_mtim)}\n"
out += f"Change: {self.get_time(st_ctim)}\n"
out += "}"
return out
p = gdb.printing.RegexpCollectionPrettyPrinter("sp")
p.add_printer("stat", "^stat$", StatPrint)
o = gdb.current_objfile()
gdb.printing.register_pretty_printer(o, p)
By sourcing the previous Python script, the ``PrettyPrinter`` can recognize
``struct stat`` and output a readable format for developers to inspect file
attributes. Without inserting functions to parse and print ``struct stat``, it
is a more convenient way to acquire a better output from Python API.
.. code-block:: bash
(gdb) list 15
10 struct stat st;
11
12 if ((rc = stat("./a.cpp", &st)) < 0) {
13 perror("stat failed.");
14 goto end;
15 }
16
17 rc = 0;
18 end:
19 return rc;
(gdb) source st.py
(gdb) b 17
Breakpoint 1 at 0x762: file a.cpp, line 17.
(gdb) r
Starting program: /root/a.out
Breakpoint 1, main (argc=1, argv=0x7fffffffe788) at a.cpp:17
17 rc = 0;
(gdb) p st
$1 = {
Size: 298
Blocks: 8
IO Block: 4096
Inode: 1322071
Access: -rw-rw-r--
File Type: regular file
Uid: (0/root)
Gid: (0/root)
Access: 2019-12-28T15:53:17
Modify: 2019-12-28T15:53:01
Change: 2019-12-28T15:53:01
}
Note that developers can disable a user-defined pretty-print via the command
``disable``. For example, the previous Python script registers a pretty printer
under the global pretty-printers. By calling ``disable pretty-print``, the
printer ``sp`` will be disabled.
.. code-block:: bash
(gdb) disable pretty-print global sp
1 printer disabled
1 of 2 printers enabled
(gdb) i pretty-print
global pretty-printers:
builtin
mpx_bound128
sp [disabled]
stat
Additionally, developers can exclude a printer in the current GDB debugging
session if it is no longer required. The following snippet shows how to delete
the ``sp`` printer through ``gdb.pretty_printers.remove``.
.. code-block:: bash
(gdb) python
>import gdb
>for p in gdb.pretty_printers:
> if p.name == "sp":
> gdb.pretty_printers.remove(p)
>end
(gdb) i pretty-print
global pretty-printers:
builtin
mpx_bound128
Conclusion
----------
Integrating Python interpreter into GDB offers many flexible ways to
troubleshoot issues. While many integrated development environments (IDEs) may
embed GDB to debug visually, GDB allows developers to implement their commands
and parse variables’ output at runtime. By using debugging scripts, developers
can monitor and record necessary information without modifying their code.
Honestly, inserting or enabling debugging code blocks may change a program’s
behaviors, and developers should get rid of this bad habit. Also, when a problem
is reproduced, GDB can attach that process and examine its status without stopping
it. Obviously, debugging via GDB is inevitable if a challenging issue emerges.
Thanks to integrating Python into GDB, developing a script to troubleshoot becomes
more accessible that leads to developers establishing their debugging methods
diversely.
Reference
---------
1. `Extending GDB using Python`_
2. `gcc/gcc/gdbhooks.py`_
3. `gdbinit/Gdbinit`_
4. `cyrus-and/gdb-dashboard`_
5. `hugsy/gef`_
6. `sharkdp/stack-inspector`_
7. `gdb Debugging Full Example (Tutorial)`_
.. _Pygments: https://pygments.org/
.. _Extending GDB using Python: https://sourceware.org/gdb/onlinedocs/gdb/Python.html
.. _gcc/gcc/gdbhooks.py: https://github.com/gcc-mirror/gcc/blob/master/gcc/gdbhooks.py
.. _hugsy/gef: https://github.com/hugsy/gef
.. _cyrus-and/gdb-dashboard: https://github.com/cyrus-and/gdb-dashboard
.. _gdbinit/Gdbinit: https://github.com/gdbinit/Gdbinit
.. _sharkdp/stack-inspector: https://github.com/sharkdp/stack-inspector
.. _GDB Text User Interface (TUI): https://sourceware.org/gdb/onlinedocs/gdb/TUI.html
.. _Linux perf: https://github.com/torvalds/linux/tree/master/tools/perf
.. _Valgrind: https://valgrind.org/
.. _gdb Debugging Full Example (Tutorial): http://www.brendangregg.com/blog/2016-08-09/gdb-example-ncurses.html
================================================
FILE: docs/notes/appendix/python-walrus.rst
================================================
.. meta::
:description lang=en: Design philosophy of pep 572, the walrus operator
:keywords: Python3, PEP 572, walrus operator
PEP 572 and The Walrus Operator
===============================
:Date: 2025-08-30
Abstract
--------
`PEP 572`_ is one of the most contentious proposals in Python3 history because
assigning a value within an expression seems unnecessary. Also, it is ambiguous
for developers to distinguish the difference between **the walrus operator**
(``:=``) and the equal operator (``=``). Even though sophisticated developers
can use "``:=``" smoothly, they may concern the readability of their code. To
better understand the usage of "``:=``," this article discusses its design
philosophy and what kind of problems it tries to solve.
Introduction
------------
For C/C++ developer, assigning a function return to a variable is common due
to error code style handling. Managing function errors includes two steps;
one is to check the return value; another is to check ``errno``. For example,
.. code-block:: cpp
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <errno.h>
int main(int argc, char *argv[]) {
int rc = -1;
// assign access return to rc and check its value
if ((rc = access("hello_walrus", R_OK)) == -1) {
fprintf(stderr, "%s", strerror(errno));
goto end;
}
rc = 0;
end:
return rc;
}
In this case, ``access`` will assign its return value to the variable ``rc``
first. Then, the program will compare the ``rc`` value with ``-1`` to check
whether the execution of ``access`` is successful or not. However, Python did
not allow assigning values to variables within an expression before 3.8. To fix
this problem, therefore, PEP 572 introduced the walrus operator for developers.
The following Python snippet is equal to the previous C example.
.. code-block:: python
>>> import os
>>> from ctypes import *
>>> libc = CDLL("libc.dylib", use_errno=True)
>>> access = libc.access
>>> path = create_string_buffer(b"hello_walrus")
>>> if (rc := access(path, os.R_OK)) == -1:
... errno = get_errno()
... print(os.strerror(errno), file=sys.stderr)
...
No such file or directory
Why ``:=`` ?
------------
Developers may confuse the difference between "``:=``" and "``=``." In fact, they
serve the same purpose, assigning somethings to variables. Why Python introduced
"``:=``" instead of using "``=``"? What is the benefit of using "``:=``"? One
reason is to reinforce the visual recognition due to a common mistake made by
C/C++ developers. For instance,
.. code-block:: cpp
int rc = access("hello_walrus", R_OK);
// rc is unintentionally assigned to -1
if (rc = -1) {
fprintf(stderr, "%s", strerror(errno));
goto end;
}
Rather than comparison, the variable, ``rc``, is mistakenly assigned to -1. To
prevent this error, some people advocate using `Yoda conditions`_ within an
expression.
.. code-block:: cpp
int rc = access("hello_walrus", R_OK);
// -1 = rc will raise a compile error
if (-1 == rc) {
fprintf(stderr, "%s", strerror(errno));
goto end;
}
However, Yoda style is not readable enough like Yoda speaks non-standardized
English. Also, unlike C/C++ can detect assigning error during the compile-time
via compiler options (e.g., -Wparentheses), it is difficult for Python interpreter
to distinguish such mistakes throughout the runtime. Thus, the final result
of PEP 572 was to use a new syntax as a solution to implement *assignment
expressions*.
The walrus operator was not the first solution for PEP 572. The original proposal
used ``EXPR as NAME`` to assign values to variables. Unfortunately, there are
some rejected reasons in this solution and other solutions as well. After
intense debates, the final decision was ``:=``.
Scopes
------
Unlike other expressions, which a variable is bound to a scope, an assignment
expression belongs to the current scope. The purpose of this design is to
allow a compact way to write code.
.. code-block:: python3
>>> if not (env := os.environ.get("HOME")):
... raise KeyError("env HOME does not find!")
...
>>> print(env)
/root
In PEP 572, another benefit is to conveniently capture a "witness" for an
``any()`` or an ``all()`` expression. Although capturing function inputs can
assist an interactive debugger, the advantage is not so obvious, and examples
lack readability. Therefore, this benefit does not discuss here. Note that
other languages (e.g., C/C++ or Go) may bind an assignment to a scope. Take
Golang as an example.
.. code-block:: go
package main
import (
"fmt"
"os"
)
func main() {
if env := os.Getenv("HOME"); env == "" {
panic(fmt.Sprintf("Home does not find"))
}
fmt.Print(env) // <--- compile error: undefined: env
}
Pitfalls
--------
Although an assigning expression allows writing compact code, there are many
pitfalls when a developer uses it in a list comprehension. A common ``SyntaxError``
is to rebind iteration variables.
.. code-block:: python3
>>> [i := i+1 for i in range(5)] # invalid
However, updating an iteration variable will reduce readability and introduce
bugs. Even if Python 3.8 did not implement the walrus operator, a programmer
should avoid reusing iteration variables within a scope.
Another pitfall is Python prohibits using assignment expressions within a
comprehension under a class scope.
.. code-block:: python3
>>> class Example:
... [(j := i) for i in range(5)] # invalid
...
This limitation was from `bpo-3692`_. The interpreter's behavior is
unpredictable when a class declaration contains a list comprehension. To avoid
this corner case, assigning expression is invalid under a class.
.. code-block:: python3
>>> class Foo:
... a = [1, 2, 3]
... b = [4, 5, 6]
... c = [i for i in zip(a, b)] # b is defined
...
>>> class Bar:
... a = [1,2,3]
... b = [4,5,6]
... c = [x * y for x in a for y in b] # b is undefined
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in Bar
File "<stdin>", line 4, in <listcomp>
NameError: name 'b' is not defined
Conclusion
----------
The reason why the walrus operator (``:=``) is so controversial is that code
readability may decrease. In fact, in the discussion `mail thread <https://mail.python.org/pipermail/python-ideas/2018-March/049409.html>`_,
the author of PEP 572, Christoph Groth, had considered using "``=``" to implement
inline assignment like C/C++. Without judging "``:=``" is ugly, many developers
argue that distinguishing the functionality between "``:=``" and "``=``" is
difficult because they serve the same purpose, but behaviors are not consistent.
Also, writing compact code is not persuasive enough because smaller is not
always better. However, in some cases, the walrus operator can enhance
readability (if you understand how to use ``:=``). For example,
.. code-block:: python3
buf = b""
while True:
data = read(1024)
if not data:
break
buf += data
By using ``:=``, the previous example can be simplified.
.. code-block:: python3
buf = b""
while (data := read(1024)):
buf += data
`Python document`_ and GitHub `issue-8122`_ provides many great examples about
improving code readability by "``:=``". However, using the walrus operator
should be careful. Some cases, such as ``foo(x := 3, cat='vector')``, may
introduce new bugs if developers are not aware of scopes. Although PEP 572
may be risky for developers to write buggy code, an in-depth understanding of
design philosophy and useful examples will help us use it to write readable
code at the right time.
References
----------
1. `PEP 572 - Assignment Expressions`_
2. `What’s New In Python 3.8`_
3. `PEP 572 and decision-making in Python`_
4. `The PEP 572 endgame`_
5. `Use assignment expression in stdlib (combined PR)`_
6. `Improper scope in list comprehension, when used in class declaration`_
.. _PEP 572: https://www.python.org/dev/peps/pep-0572/
.. _PEP 572 - Assignment Expressions: https://www.python.org/dev/peps/pep-0572/
.. _What’s New In Python 3.8: https://docs.python.org/3/whatsnew/3.8.html
.. _PEP 572 and decision-making in Python: https://lwn.net/Articles/757713/
.. _The PEP 572 endgame: https://lwn.net/Articles/759558/
.. _Use assignment expression in stdlib (combined PR): https://github.com/python/cpython/pull/8122/files
.. _improper scope in list comprehension, when used in class declaration: https://bugs.python.org/issue3692
.. _Yoda conditions: https://en.wikipedia.org/wiki/Yoda_conditions
.. _bpo-3692: https://bugs.python.org/issue3692
.. _Python document: https://docs.python.org/3/whatsnew/3.8.html#assignment-expressions
.. _issue-8122: https://github.com/python/cpython/pull/8122/files
================================================
FILE: docs/notes/asyncio/index.rst
================================================
.. meta::
:description lang=en: Python asyncio tutorial covering coroutines, event loops, tasks, async/await syntax, networking, and asynchronous programming patterns
:keywords: Python, Python3, asyncio, async, await, coroutine, event loop, asynchronous, concurrent, networking, TCP, UDP
Asyncio
=======
Python's ``asyncio`` module provides infrastructure for writing single-threaded
concurrent code using coroutines, multiplexing I/O access over sockets and other
resources, running network clients and servers, and other related primitives.
Unlike threading, asyncio uses cooperative multitasking, where tasks voluntarily
yield control to allow other tasks to run. This makes it ideal for I/O-bound
applications like web servers, database clients, and network services where
waiting for external resources is the primary bottleneck.
This section covers asyncio from basic concepts to advanced patterns, including
the event loop, coroutines, tasks, synchronization primitives, and real-world
examples like TCP/UDP servers, HTTP clients, and connection pools.
.. toctree::
:maxdepth: 1
python-asyncio-guide
python-asyncio-basic
python-asyncio-server
python-asyncio-advanced
================================================
FILE: docs/notes/asyncio/python-asyncio-advanced.rst
================================================
.. meta::
:description lang=en: Python asyncio advanced - synchronization, queues, subprocesses, debugging, patterns
:keywords: Python, Python3, Asyncio, Synchronization, Queue, Semaphore, Lock, Subprocess, Debugging
=================
Asyncio Advanced
=================
:Source: `src/basic/asyncio_.py <https://github.com/crazyguitar/pysheeet/blob/master/src/basic/asyncio_.py>`_
.. contents:: Table of Contents
:backlinks: none
Introduction
------------
Beyond basic coroutines and networking, asyncio provides synchronization
primitives, queues, subprocess management, and debugging tools. This section
covers advanced patterns for building robust async applications, including
producer-consumer patterns, rate limiting, graceful shutdown, and integration
with synchronous code.
Locks
-----
``asyncio.Lock`` prevents multiple coroutines from accessing a shared resource
simultaneously. Unlike threading locks, async locks must be used with ``await``
and only work within the same event loop.
.. code-block:: python
import asyncio
class SharedCounter:
def __init__(self):
self.value = 0
self._lock = asyncio.Lock()
async def increment(self):
async with self._lock:
current = self.value
await asyncio.sleep(0.01) # Simulate work
self.value = current + 1
async def worker(counter, name, count):
for _ in range(count):
await counter.increment()
print(f"{name} done")
async def main():
counter = SharedCounter()
await asyncio.gather(
worker(counter, "A", 100),
worker(counter, "B", 100),
worker(counter, "C", 100),
)
print(f"Final value: {counter.value}") # Should be 300
asyncio.run(main())
Semaphores for Rate Limiting
----------------------------
``asyncio.Semaphore`` limits the number of concurrent operations. This is
essential for rate limiting API calls, limiting database connections, or
controlling resource usage.
.. code-block:: python
import asyncio
async def fetch(url, semaphore):
async with semaphore:
print(f"Fetching {url}")
await asyncio.sleep(1) # Simulate network request
return f"Response from {url}"
async def main():
# Limit to 3 concurrent requests
semaphore = asyncio.Semaphore(3)
urls = [f"https://api.example.com/{i}" for i in range(10)]
tasks = [fetch(url, semaphore) for url in urls]
results = await asyncio.gather(*tasks)
for r in results:
print(r)
asyncio.run(main())
Events for Signaling
--------------------
``asyncio.Event`` allows coroutines to wait for a signal from another coroutine.
This is useful for coordinating startup, shutdown, or state changes between
multiple tasks.
.. code-block:: python
import asyncio
async def waiter(event, name):
print(f"{name} waiting for event")
await event.wait()
print(f"{name} got the event!")
async def setter(event):
print("Setting event in 2 seconds...")
await asyncio.sleep(2)
event.set()
print("Event set!")
async def main():
event = asyncio.Event()
await asyncio.gather(
waiter(event, "Task 1"),
waiter(event, "Task 2"),
waiter(event, "Task 3"),
setter(event),
)
asyncio.run(main())
Conditions for Complex Synchronization
--------------------------------------
``asyncio.Condition`` combines a lock with the ability to wait for a condition.
This is useful for producer-consumer patterns where consumers need to wait
for specific conditions.
.. code-block:: python
import asyncio
class Buffer:
def __init__(self, size):
self.buffer = []
self.size = size
self.condition = asyncio.Condition()
async def put(self, item):
async with self.condition:
while len(self.buffer) >= self.size:
await self.condition.wait()
self.buffer.append(item)
self.condition.notify()
async def get(self):
async with self.condition:
while not self.buffer:
await self.condition.wait()
item = self.buffer.pop(0)
self.condition.notify()
return item
async def producer(buffer, name):
for i in range(5):
await buffer.put(f"{name}-{i}")
print(f"Produced: {name}-{i}")
await asyncio.sleep(0.1)
async def consumer(buffer, name):
for _ in range(5):
item = await buffer.get()
print(f"{name} consumed: {item}")
await asyncio.sleep(0.2)
async def main():
buffer = Buffer(size=2)
await asyncio.gather(
producer(buffer, "P1"),
consumer(buffer, "C1"),
consumer(buffer, "C2"),
)
asyncio.run(main())
Queues for Producer-Consumer
----------------------------
``asyncio.Queue`` is the preferred way to implement producer-consumer patterns.
It handles synchronization internally and provides blocking get/put operations
with optional timeouts.
.. code-block:: python
import asyncio
async def producer(queue, name):
for i in range(5):
item = f"{name}-item-{i}"
await queue.put(item)
print(f"Produced: {item}")
await asyncio.sleep(0.5)
async def consumer(queue, name):
while True:
try:
item = await asyncio.wait_for(queue.get(), timeout=2.0)
print(f"{name} consumed: {item}")
queue.task_done()
await asyncio.sleep(0.1)
except asyncio.TimeoutError:
print(f"{name} timed out, exiting")
break
async def main():
queue = asyncio.Queue(maxsize=3)
producers = [
asyncio.create_task(producer(queue, "P1")),
asyncio.create_task(producer(queue, "P2")),
]
consumers = [
asyncio.create_task(consumer(queue, "C1")),
asyncio.create_task(consumer(queue, "C2")),
]
await asyncio.gather(*producers)
await queue.join() # Wait for all items to be processed
for c in consumers:
c.cancel()
asyncio.run(main())
Priority Queue
--------------
``asyncio.PriorityQueue`` processes items by priority. Lower priority values
are processed first. Items must be comparable or wrapped in tuples with
priority as the first element.
.. code-block:: python
import asyncio
async def producer(queue):
items = [
(3, "low priority"),
(1, "high priority"),
(2, "medium priority"),
]
for priority, item in items:
await queue.put((priority, item))
print(f"Added: {item} (priority {priority})")
async def consumer(queue):
while not queue.empty():
priority, item = await queue.get()
print(f"Processing: {item} (priority {priority})")
await asyncio.sleep(0.5)
queue.task_done()
async def main():
queue = asyncio.PriorityQueue()
await producer(queue)
await consumer(queue)
asyncio.run(main())
Running Subprocesses
--------------------
Asyncio can run and communicate with subprocesses asynchronously. This is
useful for running shell commands, external tools, or parallel processes
without blocking the event loop.
.. code-block:: python
import asyncio
async def run_command(cmd):
proc = await asyncio.create_subprocess_shell(
cmd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE
)
stdout, stderr = await proc.communicate()
return {
'cmd': cmd,
'returncode': proc.returncode,
'stdout': stdout.decode().strip(),
'stderr': stderr.decode().strip()
}
async def main():
commands = [
"echo 'Hello World'",
"python --version",
"date",
]
results = await asyncio.gather(*[run_command(c) for c in commands])
for r in results:
print(f"Command: {r['cmd']}")
print(f"Output: {r['stdout']}")
print()
asyncio.run(main())
Subprocess with Streaming Output
--------------------------------
For long-running processes, you can stream output line by line instead of
waiting for the process to complete. This is useful for monitoring logs or
progress.
.. code-block:: python
import asyncio
async def stream_subprocess(cmd):
proc = await asyncio.create_subprocess_shell(
cmd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.STDOUT
)
while True:
line = await proc.stdout.readline()
if not line:
break
print(f"[{cmd[:20]}] {line.decode().strip()}")
await proc.wait()
return proc.returncode
async def main():
# Run multiple commands and stream their output
await asyncio.gather(
stream_subprocess("for i in 1 2 3; do echo $i; sleep 1; done"),
stream_subprocess("for i in a b c; do echo $i; sleep 0.5; done"),
)
asyncio.run(main())
Graceful Shutdown
-----------------
Proper shutdown handling ensures all tasks complete cleanly and resources
are released. Use signal handlers to catch SIGINT/SIGTERM and cancel tasks
gracefully.
.. code-block:: python
import asyncio
import signal
async def worker(name):
try:
while True:
print(f"{name} working...")
await asyncio.sleep(1)
except asyncio.CancelledError:
print(f"{name} cancelled, cleaning up...")
await asyncio.sleep(0.5) # Cleanup time
print(f"{name} cleanup done")
raise
async def main():
loop = asyncio.get_event_loop()
tasks = [
asyncio.create_task(worker("Worker-1")),
asyncio.create_task(worker("Worker-2")),
]
def shutdown():
print("\nShutdown requested...")
for task in tasks:
task.cancel()
loop.add_signal_handler(signal.SIGINT, shutdown)
loop.add_signal_handler(signal.SIGTERM, shutdown)
try:
await asyncio.gather(*tasks)
except asyncio.CancelledError:
print("All tasks cancelled")
asyncio.run(main())
Running Async Code in Threads
-----------------------------
When you need to run async code from synchronous code (e.g., in a callback
or from another thread), use ``asyncio.run_coroutine_threadsafe()``.
.. code-block:: python
import asyncio
import threading
import time
async def async_task(value):
await asyncio.sleep(1)
return value * 2
def thread_function(loop):
# Run async code from a different thread
future = asyncio.run_coroutine_threadsafe(
async_task(21), loop
)
result = future.result(timeout=5)
print(f"Thread got result: {result}")
async def main():
loop = asyncio.get_event_loop()
# Start a thread that will call async code
thread = threading.Thread(target=thread_function, args=(loop,))
thread.start()
# Keep the event loop running
await asyncio.sleep(2)
thread.join()
asyncio.run(main())
Debugging Asyncio
-----------------
Enable debug mode to catch common mistakes like blocking calls, unawaited
coroutines, and slow callbacks. Debug mode adds overhead so use it only
during development.
.. code-block:: python
import asyncio
import logging
# Enable debug logging
logging.basicConfig(level=logging.DEBUG)
async def slow_callback():
import time
time.sleep(0.2) # This will trigger a warning in debug mode
async def main():
await slow_callback()
# Method 1: Environment variable
# PYTHONASYNCIODEBUG=1 python script.py
# Method 2: asyncio.run with debug=True
asyncio.run(main(), debug=True)
Custom Event Loop
-----------------
You can customize the event loop behavior by subclassing or patching. This
is useful for debugging, profiling, or adding custom functionality.
.. code-block:: python
import asyncio
class DebugEventLoop(asyncio.SelectorEventLoop):
def _run_once(self):
# Track number of scheduled callbacks
num_ready = len(self._ready)
num_scheduled = len(self._scheduled)
if num_ready or num_scheduled:
print(f"Ready: {num_ready}, Scheduled: {num_scheduled}")
super()._run_once()
async def task(n):
await asyncio.sleep(n)
print(f"Task {n} done")
# Use custom event loop
loop = DebugEventLoop()
asyncio.set_event_loop(loop)
try:
loop.run_until_complete(asyncio.gather(
task(0.1),
task(0.2),
task(0.3),
))
finally:
loop.close()
Timeout Patterns
----------------
Different timeout patterns for various use cases: per-operation timeout,
overall timeout, and timeout with fallback.
.. code-block:: python
import asyncio
async def fetch(url, delay):
await asyncio.sleep(delay)
return f"Response from {url}"
async def fetch_with_timeout(url, delay, timeout):
"""Per-operation timeout."""
try:
return await asyncio.wait_for(fetch(url, delay), timeout)
except asyncio.TimeoutError:
return f"Timeout for {url}"
async def fetch_all_with_timeout(urls, timeout):
"""Overall timeout for all operations."""
async def fetch_all():
return await asyncio.gather(*[fetch(u, i) for i, u in enumerate(urls)])
try:
return await asyncio.wait_for(fetch_all(), timeout)
except asyncio.TimeoutError:
return ["Overall timeout"]
async def fetch_with_fallback(url, delay, timeout, fallback):
"""Timeout with fallback value."""
try:
return await asyncio.wait_for(fetch(url, delay), timeout)
except asyncio.TimeoutError:
return fallback
async def main():
# Per-operation timeout
result = await fetch_with_timeout("slow.com", 5, 1)
print(result)
# Timeout with fallback
result = await fetch_with_fallback("slow.com", 5, 1, "cached response")
print(result)
asyncio.run(main())
Retry Pattern
-------------
Implement retry logic for transient failures with exponential backoff.
This is essential for robust network clients.
.. code-block:: python
import asyncio
import random
class RetryError(Exception):
pass
async def unreliable_operation():
"""Simulates an operation that fails randomly."""
if random.random() < 0.7:
raise ConnectionError("Network error")
return "Success!"
async def retry(coro_func, max_retries=3, base_delay=1.0):
"""Retry with exponential backoff."""
last_exception = None
for attempt in range(max_retries):
try:
return await coro_func()
except Exception as e:
last_exception = e
if attempt < max_retries - 1:
delay = base_delay * (2 ** attempt)
jitter = random.uniform(0, 0.1 * delay)
print(f"Attempt {attempt + 1} failed, retrying in {delay:.2f}s")
await asyncio.sleep(delay + jitter)
raise RetryError(f"Failed after {max_retries} attempts") from last_exception
async def main():
try:
result = await retry(unreliable_operation, max_retries=5)
print(f"Result: {result}")
except RetryError as e:
print(f"All retries failed: {e}")
asyncio.run(main())
Async Context Variable
----------------------
Context variables (Python 3.7+) provide task-local storage, similar to
thread-local storage but for async tasks. Useful for request IDs, user
context, or database connections.
.. code-block:: python
import asyncio
import contextvars
# Create context variable
request_id = contextvars.ContextVar('request_id', default=None)
async def process_request(rid):
request_id.set(rid)
await step1()
await step2()
async def step1():
rid = request_id.get()
print(f"[{rid}] Step 1")
await asyncio.sleep(0.1)
async def step2():
rid = request_id.get()
print(f"[{rid}] Step 2")
await asyncio.sleep(0.1)
async def main():
await asyncio.gather(
process_request("req-001"),
process_request("req-002"),
process_request("req-003"),
)
asyncio.run(main())
TaskGroup (Python 3.11+)
------------------------
``TaskGroup`` provides structured concurrency, ensuring all tasks complete
or are cancelled together. Exceptions in any task cancel all other tasks
in the group.
.. code-block:: python
import asyncio
async def task(name, delay, should_fail=False):
await asyncio.sleep(delay)
if should_fail:
raise ValueError(f"{name} failed!")
return f"{name} done"
async def main():
try:
async with asyncio.TaskGroup() as tg:
tg.create_task(task("A", 1))
tg.create_task(task("B", 2))
tg.create_task(task("C", 0.5, should_fail=True))
except* ValueError as eg:
for exc in eg.exceptions:
print(f"Caught: {exc}")
# Python 3.11+
asyncio.run(main())
================================================
FILE: docs/notes/asyncio/python-asyncio-basic.rst
================================================
.. meta::
:description lang=en: Python asyncio basics - coroutines, tasks, event loop, async/await syntax
:keywords: Python, Python3, Asyncio, Coroutines, Event Loop, async await, Asynchronous Programming
================
Asyncio Basics
================
:Source: `src/basic/asyncio_.py <https://github.com/crazyguitar/pysheeet/blob/master/src/basic/asyncio_.py>`_
.. contents:: Table of Contents
:backlinks: none
Introduction
------------
The ``asyncio`` module, introduced in Python 3.4 and significantly improved in
Python 3.5+ with ``async/await`` syntax, provides a foundation for writing
asynchronous code. Unlike threads which use preemptive multitasking (the OS
decides when to switch), asyncio uses cooperative multitasking where coroutines
explicitly yield control using ``await``. This eliminates race conditions common
in threaded code and makes reasoning about program flow much easier.
Key concepts:
- **Coroutine**: A function defined with ``async def`` that can be paused and resumed
- **Event Loop**: The central scheduler that runs coroutines and handles I/O events
- **Task**: A wrapper around a coroutine that schedules it for execution
- **Future**: A placeholder for a result that will be available later
Running Coroutines with asyncio.run
-----------------------------------
The simplest way to run async code is ``asyncio.run()``, introduced in Python 3.7.
It creates an event loop, runs the coroutine until completion, and cleans up
automatically. This is the recommended entry point for asyncio programs.
.. code-block:: python
import asyncio
async def hello():
print("Hello")
await asyncio.sleep(1)
print("World")
# Python 3.7+
asyncio.run(hello())
For file I/O or other blocking operations, use ``run_in_executor`` to avoid
blocking the event loop:
.. code-block:: python
import asyncio
from concurrent.futures import ThreadPoolExecutor
async def read_file(path):
loop = asyncio.get_event_loop()
with ThreadPoolExecutor() as pool:
with open(path) as f:
return await loop.run_in_executor(pool, f.read)
content = asyncio.run(read_file('/etc/hosts'))
Creating and Managing Tasks
---------------------------
Tasks allow multiple coroutines to run concurrently. When you create a task,
it's scheduled to run on the event loop immediately. Use ``asyncio.create_task()``
(Python 3.7+) or ``loop.create_task()`` to create tasks.
.. code-block:: python
import asyncio
async def fetch(name, delay):
await asyncio.sleep(delay)
return f"{name} done"
async def main():
# Create tasks - they start running immediately
task1 = asyncio.create_task(fetch("A", 2))
task2 = asyncio.create_task(fetch("B", 1))
# Wait for both to complete
result1 = await task1
result2 = await task2
print(result1, result2)
asyncio.run(main())
Gathering Multiple Coroutines
-----------------------------
``asyncio.gather()`` runs multiple coroutines concurrently and collects their
results in order. This is the most common way to run multiple async operations
in parallel and wait for all of them to complete.
.. code-block:: python
import asyncio
async def fetch(url, delay):
await asyncio.sleep(delay)
return f"Response from {url}"
async def main():
urls = ["site1.com", "site2.com", "site3.com"]
coros = [fetch(url, i * 0.5) for i, url in enumerate(urls)]
# Run all concurrently, results in same order as input
results = await asyncio.gather(*coros)
for r in results:
print(r)
asyncio.run(main())
Waiting with Timeout
--------------------
Use ``asyncio.wait_for()`` to set a timeout on async operations. This is
essential for network operations where you don't want to wait indefinitely
for a response that may never come.
.. code-block:: python
import asyncio
async def slow_operation():
await asyncio.sleep(10)
return "done"
async def main():
try:
result = await asyncio.wait_for(slow_operation(), timeout=2.0)
except asyncio.TimeoutError:
print("Operation timed out!")
asyncio.run(main())
Waiting for First Completed
---------------------------
``asyncio.wait()`` provides more control than ``gather()``. You can wait for
the first task to complete, first exception, or all tasks. This is useful
when you want to process results as they become available.
.. code-block:: python
import asyncio
async def fetch(name, delay):
await asyncio.sleep(delay)
return f"{name}: {delay}s"
async def main():
tasks = [
asyncio.create_task(fetch("fast", 1)),
asyncio.create_task(fetch("slow", 3)),
]
# Wait for first to complete
done, pending = await asyncio.wait(
tasks, return_when=asyncio.FIRST_COMPLETED
)
for task in done:
print(f"Completed: {task.result()}")
print(f"Still pending: {len(pending)}")
# Cancel pending tasks
for task in pending:
task.cancel()
asyncio.run(main())
Asynchronous Iteration
----------------------
Async iterators allow you to iterate over data that arrives asynchronously,
such as streaming responses or database cursors. Implement ``__aiter__`` and
``__anext__`` methods to create custom async iterators.
.. code-block:: python
import asyncio
class AsyncRange:
"""Async iterator that yields numbers with delays."""
def __init__(self, start, stop):
self.current = start
self.stop = stop
def __aiter__(self):
return self
async def __anext__(self):
if self.current >= self.stop:
raise StopAsyncIteration
await asyncio.sleep(0.5)
value = self.current
self.current += 1
return value
async def main():
async for num in AsyncRange(0, 5):
print(num)
asyncio.run(main())
Asynchronous Context Managers
-----------------------------
Async context managers are essential for managing resources that require
async setup or cleanup, such as database connections, file handles, or
network sessions. Use ``async with`` to ensure proper resource management.
.. code-block:: python
import asyncio
class AsyncConnection:
"""Simulated async database connection."""
async def __aenter__(self):
print("Connecting...")
await asyncio.sleep(1)
print("Connected")
return self
async def __aexit__(self, exc_type, exc_val, exc_tb):
print("Disconnecting...")
await asyncio.sleep(0.5)
print("Disconnected")
async def query(self, sql):
await asyncio.sleep(0.1)
return f"Result of: {sql}"
async def main():
async with AsyncConnection() as conn:
result = await conn.query("SELECT * FROM users")
print(result)
asyncio.run(main())
Using @asynccontextmanager
--------------------------
The ``@asynccontextmanager`` decorator (Python 3.7+) provides a simpler way
to create async context managers using generator syntax, similar to the
synchronous ``@contextmanager`` decorator.
.. code-block:: python
import asyncio
from contextlib import asynccontextmanager
@asynccontextmanager
async def managed_resource(name):
print(f"Acquiring {name}")
await asyncio.sleep(0.5)
try:
yield name
finally:
print(f"Releasing {name}")
await asyncio.sleep(0.2)
async def main():
async with managed_resource("database") as resource:
print(f"Using {resource}")
asyncio.run(main())
Running Blocking Code in Executor
---------------------------------
When you need to call blocking code (file I/O, CPU-intensive operations,
or libraries without async support), use ``run_in_executor()`` to run it
in a thread pool without blocking the event loop.
.. code-block:: python
import asyncio
import time
from concurrent.futures import ThreadPoolExecutor
def blocking_io():
"""Simulates blocking I/O operation."""
time.sleep(2)
return "IO complete"
def cpu_bound():
"""Simulates CPU-intensive operation."""
return sum(i * i for i in range(10**6))
async def main():
loop = asyncio.get_event_loop()
# Run in default executor (ThreadPoolExecutor)
result1 = await loop.run_in_executor(None, blocking_io)
print(result1)
# Run in custom executor
with ThreadPoolExecutor(max_workers=4) as pool:
result2 = await loop.run_in_executor(pool, cpu_bound)
print(result2)
asyncio.run(main())
Async Generators
----------------
Async generators (Python 3.6+) combine generators with async/await, allowing
you to yield values asynchronously. They're useful for streaming data or
implementing async iterators more concisely.
.. code-block:: python
import asyncio
async def async_range(start, stop):
"""Async generator that yields numbers with delays."""
for i in range(start, stop):
await asyncio.sleep(0.5)
yield i
async def main():
async for num in async_range(0, 5):
print(num)
# Async comprehension
results = [x async for x in async_range(0, 3)]
print(results)
asyncio.run(main())
Exception Handling in Tasks
---------------------------
Exceptions in tasks are stored and re-raised when you await the task or
call ``result()``. Unhandled exceptions in tasks that are never awaited
will be logged but may be silently ignored, so always await your tasks.
.. code-block:: python
import asyncio
async def failing_task():
await asyncio.sleep(1)
raise ValueError("Something went wrong")
async def main():
task = asyncio.create_task(failing_task())
try:
await task
except ValueError as e:
print(f"Caught exception: {e}")
# Using gather with return_exceptions
tasks = [
asyncio.create_task(asyncio.sleep(1)),
asyncio.create_task(failing_task()),
]
results = await asyncio.gather(*tasks, return_exceptions=True)
for r in results:
if isinstance(r, Exception):
print(f"Task failed: {r}")
else:
print(f"Task succeeded: {r}")
asyncio.run(main())
Cancelling Tasks
----------------
Tasks can be cancelled using ``task.cancel()``. The cancelled task will
raise ``asyncio.CancelledError`` at the next await point. Handle this
exception to perform cleanup when a task is cancelled.
.. code-block:: python
import asyncio
async def long_running():
try:
while True:
print("Working...")
await asyncio.sleep(1)
except asyncio.CancelledError:
print("Task was cancelled, cleaning up...")
raise # Re-raise to mark task as cancelled
async def main():
task = asyncio.create_task(long_running())
await asyncio.sleep(3)
task.cancel()
try:
await task
except asyncio.CancelledError:
print("Task cancellation confirmed")
asyncio.run(main())
================================================
FILE: docs/notes/asyncio/python-asyncio-guide.rst
================================================
.. meta::
:description lang=en: A comprehensive guide to understanding asynchronous programming in Python, from blocking I/O to event loops, callbacks, generators, and async/await syntax
:keywords: Python, Python3, asyncio, coroutine, event loop, async await, asynchronous programming, C10k problem, non-blocking I/O, selectors, generators, callback
================================================
A Hitchhiker's Guide to Asynchronous Programming
================================================
.. contents:: Table of Contents
:backlinks: none
Abstract
--------
The `C10k problem`_ remains a fundamental challenge for programmers seeking to
handle massive concurrent connections efficiently. Traditionally, developers
address extensive I/O operations using **threads**, **epoll**, or **kqueue** to
prevent software from blocking on expensive operations. However, developing
readable and bug-free concurrent code is challenging due to complexities around
data sharing and task dependencies. Even powerful tools like `Valgrind`_ that
help detect deadlocks and race conditions cannot eliminate the time-consuming
debugging process as software scales.
To address these challenges, many programming languages—including Python,
JavaScript, and C++—have developed better libraries, frameworks, and syntaxes
to help programmers manage concurrent tasks properly. Rather than focusing on
how to use modern parallel APIs, this article concentrates on the **design
philosophy** behind asynchronous programming patterns, tracing the evolution
from blocking I/O to the elegant ``async/await`` syntax.
Using threads is the most natural approach for dispatching tasks without
blocking the main thread. However, threads introduce performance overhead from
context switching and require careful locking of critical sections for atomic
operations. While event loops can enhance performance in I/O-bound scenarios,
writing readable event-driven code is challenging due to callback complexity
(commonly known as "callback hell"). Fortunately, Python introduced the
``async/await`` syntax to help developers write understandable code with high
performance. The following figure illustrates how ``async/await`` enables
handling socket connections with the simplicity of threads but the efficiency
of event loops.
.. image:: https://raw.githubusercontent.com/crazyguitar/pysheeet/master/docs/_static/appendix/event-loop-vs-thread.png
Introduction
------------
Handling I/O operations such as network connections is among the most expensive
tasks in any program. Consider a simple TCP blocking echo server (shown below).
If a client connects without sending any data, it blocks all other connections.
Even when clients send data promptly, the server cannot handle concurrent
requests because it wastes significant time waiting for I/O responses from
hardware like network interfaces. Thus, socket programming with concurrency
becomes essential for managing high request volumes.
.. code-block:: python
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(("127.0.0.1", 5566))
s.listen(10)
while True:
conn, addr = s.accept()
msg = conn.recv(1024)
conn.send(msg)
One solution to prevent blocking is dispatching tasks to separate threads. The
following example demonstrates handling connections simultaneously using threads.
However, creating numerous threads consumes computing resources without
proportional throughput gains. Worse, applications may waste time waiting for
locks when processing tasks in critical sections. While threads solve blocking
issues, factors like CPU utilization and memory overhead remain critical for
solving the C10k problem. Without creating unlimited threads, the **event loop**
provides an alternative solution for managing connections efficiently.
.. code-block:: python
import threading
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(("127.0.0.1", 5566))
s.listen(10240)
def handler(conn):
while True:
msg = conn.recv(65535)
conn.send(msg)
while True:
conn, addr = s.accept()
t = threading.Thread(target=handler, args=(conn,))
t.start()
A simple event-driven socket server comprises three main components: an **I/O
multiplexing module** (e.g., `select`_), a **scheduler** (the loop), and
**callback functions** (event handlers). The following server uses Python's
high-level I/O multiplexing module, `selectors`_, within a loop to check
whether I/O operations are ready. When data becomes available for reading or
writing, the loop retrieves I/O events and executes the appropriate callback
functions—``accept``, ``read``, or ``write``—to complete tasks.
.. code-block:: python
import socket
from selectors import DefaultSelector, EVENT_READ, EVENT_WRITE
from functools import partial
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(("127.0.0.1", 5566))
s.listen(10240)
s.setblocking(False)
sel = DefaultSelector()
def accept(s, mask):
conn, addr = s.accept()
conn.setblocking(False)
sel.register(conn, EVENT_READ, read)
def read(conn, mask):
msg = conn.recv(65535)
if not msg:
sel.unregister(conn)
return conn.close()
sel.modify(conn, EVENT_WRITE, partial(write, msg=msg))
def write(conn, mask, msg=None):
if msg:
conn.send(msg)
sel.modify(conn, EVENT_READ, read)
sel.register(s, EVENT_READ, accept)
while True:
events = sel.select()
for e, m in events:
cb = e.data
cb(e.fileobj, m)
Although managing connections via threads may be inefficient, event-loop-based
programs are harder to read and maintain. To enhance code readability, many
programming languages—including Python—introduce abstract concepts such as
**coroutines**, **futures**, and **async/await** to handle I/O multiplexing
elegantly. The following sections explore these concepts and the problems they
solve.
Callback Functions
------------------
Callback functions control data flow at runtime when events occur. However,
preserving state across callbacks is challenging. For example, implementing a
handshake protocol over TCP requires storing previous state somewhere accessible
to subsequent callbacks.
.. code-block:: python
import socket
from selectors import DefaultSelector, EVENT_READ, EVENT_WRITE
from functools import partial
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(("127.0.0.1", 5566))
s.listen(10240)
s.setblocking(False)
sel = DefaultSelector()
is_hello = {}
def accept(s, mask):
conn, addr = s.accept()
conn.setblocking(False)
is_hello[conn] = False
sel.register(conn, EVENT_READ, read)
def read(conn, mask):
msg = conn.recv(65535)
if not msg:
sel.unregister(conn)
return conn.close()
# Check whether handshake is successful
if is_hello[conn]:
sel.modify(conn, EVENT_WRITE, partial(write, msg=msg))
return
# Perform handshake
if msg.decode("utf-8").strip() != "hello":
sel.unregister(conn)
return conn.close()
is_hello[conn] = True
def write(conn, mask, msg=None):
if msg:
conn.send(msg)
sel.modify(conn, EVENT_READ, read)
sel.register(s, EVENT_READ, accept)
while True:
events = sel.select()
for e, m in events:
cb = e.data
cb(e.fileobj, m)
Although the ``is_hello`` dictionary stores state to track handshake status,
the code becomes difficult to understand. The underlying logic is actually
simple—equivalent to this blocking version:
.. code-block:: python
def accept(s):
conn, addr = s.accept()
success = handshake(conn)
if not success:
conn.close()
def handshake(conn):
data = conn.recv(65535)
if not data:
return False
if data.decode('utf-8').strip() != "hello":
return False
conn.send(b"hello")
return True
To achieve similar structure in non-blocking code, a function (or task) must
snapshot its current state—including arguments, local variables, and execution
position—when waiting for I/O operations. The scheduler must then be able to
**re-enter** the function and execute remaining code after I/O completes.
Unlike languages like C++, Python achieves this naturally because **generators**
preserve all state and can be re-entered by calling ``next()``. By utilizing
generators, handling I/O operations in a non-blocking manner with readable,
linear code—called *inline callbacks*—becomes possible within an event loop.
Event Loop
----------
An event loop is a user-space scheduler that manages tasks within a program
instead of relying on operating system thread scheduling. The following snippet
demonstrates a simple event loop handling socket connections asynchronously.
The implementation appends tasks to a FIFO job queue and registers with a
*selector* when I/O operations are not ready. A *generator* preserves task
state, allowing execution to resume without callback functions when I/O results
become available. Understanding how this event loop works reveals that a Python
generator is indeed a form of **coroutine**.
.. code-block:: python
# loop.py
from selectors import DefaultSelector, EVENT_READ, EVENT_WRITE
class Loop:
def __init__(self):
self.sel = DefaultSelector()
self.queue = []
def create_task(self, task):
self.queue.append(task)
def polling(self):
for e, m in self.sel.select(0):
self.queue.append((e.data, None))
self.sel.unregister(e.fileobj)
def is_registered(self, fileobj):
try:
self.sel.get_key(fileobj)
except KeyError:
return False
return True
def register(self, t, data):
if not data:
return False
event_type, fileobj = data
if event_type in (EVENT_READ, EVENT_WRITE):
if self.is_registered(fileobj):
self.sel.modify(fileobj, event_type, t)
else:
self.sel.register(fileobj, event_type, t)
return True
return False
def accept(self, s):
while True:
try:
conn, addr = s.accept()
except BlockingIOError:
yield (EVENT_READ, s)
else:
break
return conn, addr
def recv(self, conn, size):
while True:
try:
msg = conn.recv(size)
except BlockingIOError:
yield (EVENT_READ, conn)
else:
break
return msg
def send(self, conn, msg):
while True:
try:
size = conn.send(msg)
except BlockingIOError:
yield (EVENT_WRITE, conn)
else:
break
return size
def once(self):
self.polling()
unfinished = []
for t, data in self.queue:
try:
data = t.send(data)
except StopIteration:
continue
if self.register(t, data):
unfinished.append((t, None))
self.queue = unfinished
def run(self):
while self.queue or self.sel.get_map():
self.once()
By assigning jobs to an event loop, the programming pattern resembles using
threads but with a user-level scheduler. `PEP 380`_ introduced generator
delegation via ``yield from``, allowing a generator to wait for other generators
to complete. The following snippet is far more intuitive and readable than
callback-based I/O handling:
.. code-block:: python
# server.py
# $ python3 server.py &
# $ nc localhost 5566
import socket
from loop import Loop
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(("127.0.0.1", 5566))
s.listen(10240)
s.setblocking(False)
loop = Loop()
def handler(conn):
while True:
msg = yield from loop.recv(conn, 1024)
if not msg:
conn.close()
break
yield from loop.send(conn, msg)
def main():
while True:
conn, addr = yield from loop.accept(s)
conn.setblocking(False)
loop.create_task((handler(conn), None))
loop.create_task((main(), None))
loop.run()
Using an event loop with ``yield from`` manages connections without blocking
the main thread—this was how ``asyncio`` worked before Python 3.5. However,
``yield from`` is ambiguous: why does adding ``@asyncio.coroutine`` transform
a generator into a coroutine? Instead of overloading generator syntax for
asynchronous operations, `PEP 492`_ proposed that coroutines should become a
**standalone concept** in Python. This led to the introduction of ``async/await``
syntax, dramatically improving readability for asynchronous programming.
What is a Coroutine?
--------------------
Python documentation defines coroutines as "a generalized form of subroutines."
This definition, while technically accurate, can be confusing. Based on our
discussion, an event loop schedules generators to perform specific tasks—similar
to how an OS dispatches jobs to threads. In this context, generators serve as
"routine workers." A **coroutine** is simply a task scheduled by an event loop
within a program, rather than by the operating system.
The following snippet illustrates what ``@coroutine`` does. This decorator
transforms a function into a generator function and wraps it with
``types.coroutine`` for backward compatibility:
.. code-block:: python
import asyncio
import inspect
import types
from functools import wraps
from asyncio.futures import Future
def coroutine(func):
"""Simple prototype of coroutine decorator"""
if inspect.isgeneratorfunction(func):
return types.coroutine(func)
@wraps(func)
def coro(*a, **k):
res = func(*a, **k)
if isinstance(res, Future) or inspect.isgenerator(res):
res = yield from res
return res
return types.coroutine(coro)
@coroutine
def foo():
yield from asyncio.sleep(1)
print("Hello Foo")
loop = asyncio.get_event_loop()
loop.run_until_complete(loop.create_task(foo()))
loop.close()
With Python 3.5+, the ``async def`` syntax creates native coroutines directly,
and ``await`` replaces ``yield from`` for suspending execution. This makes the
intent explicit: ``async def`` declares a coroutine, and ``await`` marks
suspension points where the event loop can switch to other tasks.
Conclusion
----------
Asynchronous programming via event loops has become more straightforward and
readable thanks to modern syntax and library support. Most programming
languages, including Python, implement libraries that manage task scheduling
through integration with new syntaxes. While ``async/await`` may seem enigmatic
initially, it provides a way for programmers to develop logical, linear code
structure—similar to using threads—while gaining the performance benefits of
event-driven I/O.
Without callback functions passing state between handlers, programmers no
longer need to worry about preserving local variables and arguments across
asynchronous boundaries. This allows developers to focus on application logic
rather than spending time troubleshooting concurrency issues. The evolution
from callbacks to generators to ``async/await`` represents a significant
advancement in making concurrent programming accessible and maintainable.
References
----------
1. `asyncio — Asynchronous I/O`_
2. `PEP 342 - Coroutines via Enhanced Generators`_
3. `PEP 380 - Syntax for Delegating to a Subgenerator`_
4. `PEP 492 - Coroutines with async and await syntax`_
.. _C10k problem: https://en.wikipedia.org/wiki/C10k_problem
.. _Valgrind: https://valgrind.org/
.. _select: https://docs.python.org/3/library/select.html
.. _selectors: https://docs.python.org/3/library/selectors.html
.. _asyncio — Asynchronous I/O: https://docs.python.org/3/library/asyncio.html
.. _PEP 492: https://www.python.org/dev/peps/pep-0492/
.. _PEP 380: https://www.python.org/dev/peps/pep-0380/
.. _PEP 342 - Coroutines via Enhanced Generators: https://www.python.org/dev/peps/pep-0342/
.. _PEP 492 - Coroutines with async and await syntax: https://www.python.org/dev/peps/pep-0492/
.. _PEP 380 - Syntax for Delegating to a Subgenerator: https://www.python.org/dev/peps/pep-0380/
================================================
FILE: docs/notes/asyncio/python-asyncio-server.rst
================================================
.. meta::
:description lang=en: Python asyncio networking - TCP/UDP servers, HTTP clients, SSL/TLS, protocols
:keywords: Python, Python3, Asyncio, TCP Server, UDP Server, HTTP Client, SSL TLS, Network Programming
===================
Asyncio Networking
===================
:Source: `src/basic/asyncio_.py <https://github.com/crazyguitar/pysheeet/blob/master/src/basic/asyncio_.py>`_
.. contents:: Table of Contents
:backlinks: none
Introduction
------------
Asyncio excels at network programming because network I/O is inherently
asynchronous - you send a request and wait for a response. Instead of blocking
a thread while waiting, asyncio allows other tasks to run. This section covers
building TCP/UDP servers and clients, HTTP requests, SSL/TLS encryption, and
the Transport/Protocol API for low-level control.
TCP Echo Server with Streams
----------------------------
The streams API (``asyncio.start_server``, ``open_connection``) provides a
high-level interface for TCP networking. It handles buffering, encoding, and
connection management automatically, making it the recommended approach for
most applications.
.. code-block:: python
import asyncio
async def handle_client(reader, writer):
addr = writer.get_extra_info('peername')
print(f"Connected: {addr}")
while True:
data = await reader.read(1024)
if not data:
break
message = data.decode()
print(f"Received: {message!r} from {addr}")
writer.write(data)
await writer.drain()
print(f"Disconnected: {addr}")
writer.close()
await writer.wait_closed()
async def main():
server = await asyncio.start_server(
handle_client, 'localhost', 8888
)
addr = server.sockets[0].getsockname()
print(f"Serving on {addr}")
async with server:
await server.serve_forever()
asyncio.run(main())
TCP Client with Streams
-----------------------
The client side uses ``asyncio.open_connection()`` to establish a connection.
The returned reader and writer objects provide async methods for sending and
receiving data.
.. code-block:: python
import asyncio
async def tcp_client(message):
reader, writer = await asyncio.open_connection(
'localhost', 8888
)
print(f"Sending: {message!r}")
writer.write(message.encode())
await writer.drain()
data = await reader.read(1024)
print(f"Received: {data.decode()!r}")
writer.close()
await writer.wait_closed()
asyncio.run(tcp_client("Hello, Server!"))
Low-Level TCP with Sockets
--------------------------
For more control, you can use raw sockets with the event loop's socket methods.
This approach is useful when you need fine-grained control over socket options
or when integrating with existing socket-based code.
.. code-block:: python
import asyncio
import socket
async def handle_client(loop, conn):
while True:
data = await loop.sock_recv(conn, 1024)
if not data:
break
await loop.sock_sendall(conn, data)
conn.close()
async def server():
loop = asyncio.get_event_loop()
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.setblocking(False)
sock.bind(('localhost', 8888))
sock.listen(100)
print("Server listening on localhost:8888")
while True:
conn, addr = await loop.sock_accept(sock)
print(f"Connected: {addr}")
asyncio.create_task(handle_client(loop, conn))
asyncio.run(server())
UDP Echo Server
---------------
UDP is connectionless, so the API is different from TCP. Use
``create_datagram_endpoint()`` with a protocol class to handle UDP packets.
Each packet is independent and may arrive out of order or not at all.
.. code-block:: python
import asyncio
class EchoUDPProtocol(asyncio.DatagramProtocol):
def connection_made(self, transport):
self.transport = transport
def datagram_received(self, data, addr):
message = data.decode()
print(f"Received {message!r} from {addr}")
self.transport.sendto(data, addr)
async def main():
loop = asyncio.get_event_loop()
transport, protocol = await loop.create_datagram_endpoint(
EchoUDPProtocol,
local_addr=('localhost', 9999)
)
print("UDP server listening on localhost:9999")
try:
await asyncio.sleep(3600) # Run for 1 hour
finally:
transport.close()
asyncio.run(main())
HTTP Client with SSL
--------------------
Making HTTPS requests requires SSL context configuration. This example shows
how to fetch web pages using low-level streams with proper SSL verification.
.. code-block:: python
import asyncio
import ssl
async def fetch_https(host, path="/"):
# Create SSL context with certificate verification
ctx = ssl.create_default_context()
reader, writer = await asyncio.open_connection(
host, 443, ssl=ctx
)
# Send HTTP request
request = f"GET {path} HTTP/1.1\r\nHost: {host}\r\nConnection: close\r\n\r\n"
writer.write(request.encode())
await writer.drain()
# Read response
response = await reader.read()
writer.close()
await writer.wait_closed()
return response.decode()
async def main():
urls = [
("www.python.org", "/"),
("github.com", "/"),
]
tasks = [fetch_https(host, path) for host, path in urls]
responses = await asyncio.gather(*tasks)
for (host, _), resp in zip(urls, responses):
status = resp.split('\r\n')[0]
print(f"{host}: {status}")
asyncio.run(main())
HTTPS Server with SSL
---------------------
Creating an HTTPS server requires SSL certificates. This example shows a
simple HTTPS server that serves static content with TLS encryption.
.. code-block:: python
import asyncio
import ssl
async def handle_request(reader, writer):
request = await reader.read(1024)
response = b"HTTP/1.1 200 OK\r\n"
response += b"Content-Type: text/html\r\n\r\n"
response += b"<html><body><h1>Hello HTTPS!</h1></body></html>"
writer.write(response)
await writer.drain()
writer.close()
await writer.wait_closed()
async def main():
# Create SSL context
ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
ctx.load_cert_chain('cert.pem', 'key.pem')
server = await asyncio.start_server(
handle_request, 'localhost', 8443, ssl=ctx
)
print("HTTPS server on https://localhost:8443")
async with server:
await server.serve_forever()
# Generate self-signed cert:
# openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes
asyncio.run(main())
Transport and Protocol API
--------------------------
The Transport/Protocol API provides low-level control over network connections.
Transports handle the actual I/O while Protocols handle the data processing.
This separation allows for flexible and reusable network code.
.. code-block:: python
import asyncio
class EchoProtocol(asyncio.Protocol):
def connection_made(self, transport):
self.transport = transport
peername = transport.get_extra_info('peername')
print(f"Connection from {peername}")
def data_received(self, data):
print(f"Received: {data.decode()!r}")
self.transport.write(data)
def connection_lost(self, exc):
print("Connection closed")
async def main():
loop = asyncio.get_event_loop()
server = await loop.create_server(
EchoProtocol, 'localhost', 8888
)
async with server:
await server.serve_forever()
asyncio.run(main())
DNS Resolution
--------------
Asyncio provides async DNS resolution through ``getaddrinfo()``. This is
useful when you need to resolve hostnames without blocking the event loop.
.. code-block:: python
import asyncio
import socket
async def resolve_host(host, port=80):
loop = asyncio.get_event_loop()
infos = await loop.getaddrinfo(
host, port,
family=socket.AF_UNSPEC,
type=socket.SOCK_STREAM
)
for family, type_, proto, canonname, sockaddr in infos:
ip, port = sockaddr[:2]
family_name = "IPv4" if family == socket.AF_INET else "IPv6"
print(f"{host} -> {ip} ({family_name})")
async def main():
hosts = ["python.org", "github.com", "google.com"]
await asyncio.gather(*[resolve_host(h) for h in hosts])
asyncio.run(main())
Simple HTTP Server
------------------
A minimal HTTP server implementation showing how to parse requests and
send responses. For production use, consider frameworks like aiohttp or
FastAPI.
.. code-block:: python
import asyncio
async def handle_http(reader, writer):
request = await reader.read(1024)
request_line = request.decode().split('\r\n')[0]
method, path, _ = request_line.split(' ')
print(f"{method} {path}")
# Simple routing
if path == '/':
body = b"<h1>Home</h1>"
status = "200 OK"
elif path == '/about':
body = b"<h1>About</h1>"
status = "200 OK"
else:
body = b"<h1>404 Not Found</h1>"
status = "404 Not Found"
response = f"HTTP/1.1 {status}\r\n"
response += f"Content-Length: {len(body)}\r\n"
response += "Content-Type: text/html\r\n\r\n"
writer.write(response.encode() + body)
await writer.drain()
writer.close()
await writer.wait_closed()
async def main():
server = await asyncio.start_server(
handle_http, 'localhost', 8080
)
print("HTTP server on http://localhost:8080")
async with server:
await server.serve_forever()
asyncio.run(main())
Using sendfile for Efficient File Transfer
------------------------------------------
The ``sendfile()`` method (Python 3.7+) efficiently transfers file contents
to a transport using the OS's sendfile syscall, avoiding copying data through
Python.
.. code-block:: python
import asyncio
async def handle_request(reader, writer):
await reader.read(1024) # Read request
with open('index.html', 'rb') as f:
# Get file size
f.seek(0, 2)
size = f.tell()
f.seek(0)
# Send headers
headers = f"HTTP/1.1 200 OK\r\n"
headers += f"Content-Length: {size}\r\n"
headers += "Content-Type: text/html\r\n\r\n"
writer.write(headers.encode())
# Send file efficiently
loop = asyncio.get_event_loop()
await loop.sendfile(writer.transport, f)
writer.close()
await writer.wait_closed()
async def main():
server = await asyncio.start_server(
handle_request, 'localhost', 8080
)
async with server:
await server.serve_forever()
asyncio.run(main())
Connection Pool
---------------
Connection pools reuse connections to avoid the overhead of establishing
new connections for each request. This is essential for high-performance
clients that make many requests to the same server.
.. code-block:: python
import asyncio
from collections import deque
class ConnectionPool:
def __init__(self, host, port, size=5):
self.host = host
self.port = port
self.size = size
self._pool = deque()
self._lock = asyncio.Lock()
async def get(self):
async with self._lock:
if self._pool:
return self._pool.popleft()
# Create new connection
reader, writer = await asyncio.open_connection(
self.host, self.port
)
return reader, writer
async def put(self, reader, writer):
async with self._lock:
if len(self._pool) < self.size:
self._pool.append((reader, writer))
else:
writer.close()
await writer.wait_closed()
async def close(self):
async with self._lock:
while self._pool:
reader, writer = self._pool.popleft()
writer.close()
await writer.wait_closed()
async def fetch(pool, message):
reader, writer = await pool.get()
try:
writer.write(message.encode())
await writer.drain()
data = await reader.read(1024)
return data.decode()
finally:
await pool.put(reader, writer)
async def main():
pool = ConnectionPool('localhost', 8888, size=3)
try:
tasks = [fetch(pool, f"msg{i}") for i in range(10)]
results = await asyncio.gather(*tasks)
for r in results:
print(r)
finally:
await pool.close()
asyncio.run(main())
================================================
FILE: docs/notes/basic/index.rst
================================================
.. meta::
:description lang=en: Python basics cheat sheet covering syntax, data types, functions, classes, generators, typing, and essential Python programming concepts
:keywords: Python, Python3, basics, syntax, data types, functions, classes, generators, typing, list, dict, set, comprehension
Quick Start
===========
This cheat sheet is designed to help developers learn Python syntax from the
ground up. It covers the fundamentals while also introducing common patterns
and idioms that experienced Python developers use, which may feel unfamiliar to
beginners. For instance, constructs like ``for ... else ...`` are rarely seen in
other programming languages. Additionally, we’ll explore interesting topics such
as ``__future__``, typing, and Unicode—concepts you may have heard of but never
fully understood. By working through this cheat sheet, you’ll gain a solid
foundation in Python and learn to write code that feels truly Pythonic.
.. toctree::
:maxdepth: 1
python-basic
python-future
python-func
python-object
python-typing
python-list
python-set
python-dict
python-heap
python-generator
python-unicode
python-rexp
================================================
FILE: docs/notes/basic/python-basic.rst
================================================
.. meta::
:description lang=en: Python basics tutorial covering fundamental syntax, data types, control flow, string formatting, and essential Python programming concepts
:keywords: Python, Python3, basics, syntax, data types, control flow, string formatting, variables, operators, conditionals
============
From Scratch
============
.. contents:: Table of Contents
:backlinks: none
The main goal of this cheat sheet is to collect some common and basic semantics
or snippets. The cheat sheet includes some syntax, which we have already known
but still ambiguous in our mind, or some snippets, which we google them again
and again. In addition, because **the end Of life date for Python 2** is coming.
Most of the snippets are mainly based on **Python 3**'s syntax.
Hello world!
------------
When we start to learn a new language, we usually learn from printing
**Hello world!**. In Python, we can use another way to print the message by
importing ``__hello__`` module. The source code can be found on
`frozen.c <https://github.com/python/cpython/blob/master/Python/frozen.c>`_.
.. code-block:: python
>>> print("Hello world!")
Hello world!
>>> import __hello__
Hello world!
>>> import __phello__
Hello world!
>>> import __phello__.spam
Hello world!
Python Version
--------------
It is important for a programmer to know current Python version because
not every syntax will work in the current version. In this case, we can get the
Python version by ``python -V`` or using the module, ``sys``.
.. code-block:: python
>>> import sys
>>> print(sys.version)
3.7.1 (default, Nov 6 2018, 18:46:03)
[Clang 10.0.0 (clang-1000.11.45.5)]
We can also use ``platform.python_version`` to get Python version.
.. code-block:: python
>>> import platform
>>> pla
gitextract_168c70l7/
├── .clang-format
├── .claude-plugin/
│ ├── marketplace.json
│ └── plugin.json
├── .coveragerc
├── .gitattributes
├── .github/
│ ├── FUNDING.yml
│ ├── dependabot.yml
│ └── workflows/
│ └── pythonpackage.yml
├── .gitignore
├── CITATION.cff
├── LICENSE
├── Makefile
├── Procfile
├── README.rst
├── app.py
├── app_test.py
├── docs/
│ ├── 404.rst
│ ├── Makefile
│ ├── _extra/
│ │ └── robots.txt
│ ├── _static/
│ │ ├── .gitignore
│ │ ├── carbonad.css
│ │ └── style.css
│ ├── _templates/
│ │ ├── carbonad.html
│ │ ├── cheatsheets.html
│ │ ├── layout.html
│ │ ├── link.html
│ │ └── sidebarintro.html
│ ├── conf.py
│ ├── index.rst
│ └── notes/
│ ├── appendix/
│ │ ├── disaggregated-prefill-decode.rst
│ │ ├── index.rst
│ │ ├── megatron-efa-monitoring.rst
│ │ ├── nccl-gin.rst
│ │ ├── python-gdb.rst
│ │ └── python-walrus.rst
│ ├── asyncio/
│ │ ├── index.rst
│ │ ├── python-asyncio-advanced.rst
│ │ ├── python-asyncio-basic.rst
│ │ ├── python-asyncio-guide.rst
│ │ └── python-asyncio-server.rst
│ ├── basic/
│ │ ├── index.rst
│ │ ├── python-basic.rst
│ │ ├── python-dict.rst
│ │ ├── python-func.rst
│ │ ├── python-future.rst
│ │ ├── python-generator.rst
│ │ ├── python-heap.rst
│ │ ├── python-list.rst
│ │ ├── python-object.rst
│ │ ├── python-rexp.rst
│ │ ├── python-set.rst
│ │ ├── python-typing.rst
│ │ └── python-unicode.rst
│ ├── concurrency/
│ │ ├── index.rst
│ │ ├── python-futures.rst
│ │ ├── python-multiprocessing.rst
│ │ └── python-threading.rst
│ ├── database/
│ │ ├── index.rst
│ │ ├── python-sqlalchemy-orm.rst
│ │ ├── python-sqlalchemy-query.rst
│ │ └── python-sqlalchemy.rst
│ ├── extension/
│ │ ├── cpp-from-python.rst
│ │ ├── index.rst
│ │ ├── python-capi.rst
│ │ ├── python-cext-modern.rst
│ │ └── python-ctypes.rst
│ ├── hpc/
│ │ ├── index.rst
│ │ └── slurm.rst
│ ├── llm/
│ │ ├── index.rst
│ │ ├── llm-bench.rst
│ │ ├── llm-serving.rst
│ │ ├── megatron.rst
│ │ └── pytorch.rst
│ ├── network/
│ │ ├── index.rst
│ │ ├── python-socket-async.rst
│ │ ├── python-socket-server.rst
│ │ ├── python-socket-sniffer.rst
│ │ ├── python-socket-ssl.rst
│ │ ├── python-socket.rst
│ │ └── python-ssh.rst
│ ├── os/
│ │ ├── index.rst
│ │ ├── python-date.rst
│ │ ├── python-io.rst
│ │ └── python-os.rst
│ ├── python-new-py3.rst
│ └── security/
│ ├── index.rst
│ ├── python-crypto.rst
│ ├── python-tls.rst
│ └── python-vulnerability.rst
├── requirements.txt
├── runtime.txt
├── skills/
│ └── py/
│ ├── SKILL.md
│ └── references/
│ ├── guidelines.md
│ └── structure.md
└── src/
├── basic/
│ ├── asyncio_.py
│ ├── basic.py
│ ├── cext_.py
│ ├── concurrency_.py
│ ├── crypto_.py
│ ├── datetime_.py
│ ├── dict.py
│ ├── fileio_.py
│ ├── func.py
│ ├── future_.py
│ ├── generator.py
│ ├── heap.py
│ ├── list.py
│ ├── object.py
│ ├── os_.py
│ ├── rexp.py
│ ├── set.py
│ ├── socket_.py
│ ├── sqlalchemy_core.py
│ ├── sqlalchemy_orm.py
│ ├── sqlalchemy_query.py
│ ├── typing_.py
│ └── unicode_.py
├── cext/
│ ├── CMakeLists.txt
│ ├── README.md
│ ├── capi/
│ │ ├── args.c
│ │ ├── errors.c
│ │ ├── gil.c
│ │ ├── setup.py
│ │ ├── simple.c
│ │ ├── test_capi.py
│ │ └── types_demo.c
│ ├── conftest.py
│ ├── example.cpp
│ ├── fib.c
│ ├── gil_example.cpp
│ ├── numpy_example.cpp
│ ├── setup.py
│ ├── test_cext.py
│ └── vector.cpp
├── cpp_from_python/
│ ├── CMakeLists.txt
│ └── cpp_from_py.cpp
├── gin/
│ ├── Dockerfile
│ ├── Makefile
│ ├── run.enroot
│ └── run.sbatch
├── llm/
│ ├── sglang/
│ │ ├── Dockerfile
│ │ ├── Makefile
│ │ ├── README.rst
│ │ ├── bench.sh
│ │ ├── run.sbatch
│ │ └── test.sh
│ ├── tensorrt-llm/
│ │ ├── Dockerfile
│ │ ├── Makefile
│ │ ├── README.rst
│ │ ├── bench.sh
│ │ ├── run.sbatch
│ │ └── test.sh
│ └── vllm/
│ ├── Dockerfile
│ ├── Makefile
│ ├── README.rst
│ ├── bench.sh
│ ├── offline_bench.py
│ ├── offline_bench.sh
│ ├── run.sbatch
│ ├── run.sh
│ ├── sweep.sbatch
│ ├── sweep.sh
│ └── test.sh
├── megatron/
│ ├── Dockerfile
│ ├── Makefile
│ ├── README.md
│ ├── enroot.sh
│ ├── entrypoint.py
│ ├── recipes/
│ │ └── deepseek_v2_lite_pretrain.py
│ ├── srun.sh
│ └── viztracer_plugin.py
├── new_py3/
│ └── py3.py
├── nixl/
│ ├── Dockerfile
│ ├── Makefile
│ ├── bench.sh
│ ├── nixl.sbatch
│ └── vllm.sbatch
└── security/
└── vulnerability_.py
SYMBOL INDEX (1343 symbols across 46 files)
FILE: app.py
function find_key (line 16) | def find_key(token):
function page_not_found (line 83) | def page_not_found(e):
function static_proxy (line 89) | def static_proxy(path):
function index_redirection (line 99) | def index_redirection():
function acme (line 106) | def acme(token):
FILE: app_test.py
class PysheeetTest (line 23) | class PysheeetTest(LiveServerTestCase):
method create_app (line 26) | def create_app(self):
method check_security_headers (line 44) | def check_security_headers(self, resp):
method check_csrf_cookies (line 54) | def check_csrf_cookies(self, resp):
method test_index_redirection_req (line 60) | def test_index_redirection_req(self):
method test_static_proxy_req (line 68) | def test_static_proxy_req(self):
method test_acme_req (line 80) | def test_acme_req(self):
method test_find_key (line 93) | def test_find_key(self):
method test_acme (line 109) | def test_acme(self):
method test_index_redirection (line 126) | def test_index_redirection(self):
method test_static_proxy (line 132) | def test_static_proxy(self):
method test_page_not_found (line 145) | def test_page_not_found(self):
FILE: docs/conf.py
function add_html_link (line 329) | def add_html_link(app, pagename, templatename, context, doctree):
function create_sitemap (line 340) | def create_sitemap(app, exception):
function setup (line 364) | def setup(app):
FILE: src/basic/asyncio_.py
class TestAsyncioBasics (line 7) | class TestAsyncioBasics:
method test_asyncio_run (line 10) | def test_asyncio_run(self):
method test_create_task (line 19) | def test_create_task(self):
method test_gather (line 33) | def test_gather(self):
method test_wait_for_timeout (line 46) | def test_wait_for_timeout(self):
method test_wait_first_completed (line 58) | def test_wait_first_completed(self):
class TestAsyncIterator (line 83) | class TestAsyncIterator:
method test_async_iterator (line 86) | def test_async_iterator(self):
method test_async_generator (line 114) | def test_async_generator(self):
class TestAsyncContextManager (line 129) | class TestAsyncContextManager:
method test_async_context_manager (line 132) | def test_async_context_manager(self):
method test_asynccontextmanager_decorator (line 154) | def test_asynccontextmanager_decorator(self):
class TestSynchronization (line 172) | class TestSynchronization:
method test_lock (line 175) | def test_lock(self):
method test_semaphore (line 194) | def test_semaphore(self):
method test_event (line 215) | def test_event(self):
class TestQueue (line 237) | class TestQueue:
method test_queue (line 240) | def test_queue(self):
method test_priority_queue (line 263) | def test_priority_queue(self):
class TestExceptionHandling (line 283) | class TestExceptionHandling:
method test_task_exception (line 286) | def test_task_exception(self):
method test_gather_return_exceptions (line 299) | def test_gather_return_exceptions(self):
class TestCancellation (line 316) | class TestCancellation:
method test_cancel_task (line 319) | def test_cancel_task(self):
class TestExecutor (line 347) | class TestExecutor:
method test_run_in_executor (line 350) | def test_run_in_executor(self):
class TestSubprocess (line 366) | class TestSubprocess:
method test_subprocess (line 369) | def test_subprocess(self):
class TestTimeout (line 386) | class TestTimeout:
method test_wait_for_timeout (line 389) | def test_wait_for_timeout(self):
FILE: src/basic/basic.py
function get_version_info (line 12) | def get_version_info() -> tuple:
function get_version_string (line 17) | def get_version_string() -> str:
function check_version (line 22) | def check_version(major: int, minor: int) -> bool:
function classify_number (line 28) | def classify_number(x: int) -> str:
function is_even (line 38) | def is_even(x: int) -> bool:
function sum_range (line 44) | def sum_range(n: int) -> int:
function find_first_even (line 52) | def find_first_even(numbers: list) -> int | None:
function sum_odd_only (line 60) | def sum_odd_only(numbers: list) -> int:
function loop_completed (line 70) | def loop_completed(items: list, target) -> bool:
function safe_divide (line 79) | def safe_divide(a: float, b: float) -> float | None:
function parse_int (line 87) | def parse_int(s: str) -> int | None:
function divide_or_raise (line 95) | def divide_or_raise(a: float, b: float) -> float:
function squares (line 103) | def squares(n: int) -> list:
function even_numbers (line 108) | def even_numbers(n: int) -> list:
function square_dict (line 113) | def square_dict(n: int) -> dict:
function is_truthy (line 119) | def is_truthy(value) -> bool:
function swap (line 125) | def swap(a, b) -> tuple:
function first_and_rest (line 130) | def first_and_rest(items: list) -> tuple:
class TestVersion (line 137) | class TestVersion:
method test_get_version_info (line 138) | def test_get_version_info(self):
method test_check_version (line 143) | def test_check_version(self):
class TestControlFlow (line 148) | class TestControlFlow:
method test_classify_number (line 149) | def test_classify_number(self):
method test_is_even (line 154) | def test_is_even(self):
class TestLoops (line 159) | class TestLoops:
method test_sum_range (line 160) | def test_sum_range(self):
method test_find_first_even (line 163) | def test_find_first_even(self):
method test_sum_odd_only (line 167) | def test_sum_odd_only(self):
method test_loop_completed (line 170) | def test_loop_completed(self):
class TestExceptions (line 175) | class TestExceptions:
method test_safe_divide (line 176) | def test_safe_divide(self):
method test_parse_int (line 180) | def test_parse_int(self):
method test_divide_or_raise (line 184) | def test_divide_or_raise(self):
class TestComprehensions (line 190) | class TestComprehensions:
method test_squares (line 191) | def test_squares(self):
method test_even_numbers (line 194) | def test_even_numbers(self):
method test_square_dict (line 197) | def test_square_dict(self):
class TestTruthiness (line 201) | class TestTruthiness:
method test_falsy (line 202) | def test_falsy(self):
method test_truthy (line 208) | def test_truthy(self):
class TestAssignment (line 214) | class TestAssignment:
method test_swap (line 215) | def test_swap(self):
method test_first_and_rest (line 218) | def test_first_and_rest(self):
FILE: src/basic/cext_.py
function has_c_compiler (line 19) | def has_c_compiler():
class TestCtypesBasic (line 38) | class TestCtypesBasic:
method test_libc_strlen (line 41) | def test_libc_strlen(self):
method test_libc_abs (line 56) | def test_libc_abs(self):
method test_math_sqrt (line 68) | def test_math_sqrt(self):
class TestCtypesStructures (line 84) | class TestCtypesStructures:
method test_simple_structure (line 87) | def test_simple_structure(self):
method test_nested_structure (line 101) | def test_nested_structure(self):
method test_array_in_structure (line 116) | def test_array_in_structure(self):
class TestCtypesPointers (line 131) | class TestCtypesPointers:
method test_pointer_to_int (line 134) | def test_pointer_to_int(self):
method test_byref (line 144) | def test_byref(self):
class TestCtypesCustomLibrary (line 154) | class TestCtypesCustomLibrary:
method fib_library (line 158) | def fib_library(self, tmp_path):
method test_fib (line 210) | def test_fib(self, fib_library):
method test_add (line 217) | def test_add(self, fib_library):
method test_multiply (line 223) | def test_multiply(self, fib_library):
class TestCffi (line 229) | class TestCffi:
method cffi_available (line 233) | def cffi_available(self):
method test_cffi_libc (line 242) | def test_cffi_libc(self, cffi_available):
method test_cffi_math (line 263) | def test_cffi_math(self, cffi_available):
class TestPythonPerformance (line 285) | class TestPythonPerformance:
method test_python_fib (line 288) | def test_python_fib(self):
method test_python_fib_iterative (line 301) | def test_python_fib_iterative(self):
method test_python_fib_memoized (line 317) | def test_python_fib_memoized(self):
FILE: src/basic/concurrency_.py
function _mp_square (line 11) | def _mp_square(x):
function _mp_add (line 15) | def _mp_add(a, b):
function _mp_worker (line 19) | def _mp_worker(q, n):
function _mp_increment (line 23) | def _mp_increment(counter):
function _mp_double (line 29) | def _mp_double(arr):
class TestThreading (line 34) | class TestThreading:
method test_thread_creation (line 37) | def test_thread_creation(self):
method test_thread_with_return_value (line 52) | def test_thread_with_return_value(self):
method test_lock (line 67) | def test_lock(self):
method test_rlock (line 85) | def test_rlock(self):
method test_semaphore (line 105) | def test_semaphore(self):
method test_event (line 126) | def test_event(self):
method test_condition (line 148) | def test_condition(self):
method test_barrier (line 177) | def test_barrier(self):
method test_queue (line 197) | def test_queue(self):
class TestMultiprocessing (line 221) | class TestMultiprocessing:
method test_process_creation (line 224) | def test_process_creation(self):
method test_pool_map (line 238) | def test_pool_map(self):
method test_pool_starmap (line 247) | def test_pool_starmap(self):
method test_shared_value (line 256) | def test_shared_value(self):
method test_shared_array (line 271) | def test_shared_array(self):
class TestConcurrentFutures (line 283) | class TestConcurrentFutures:
method test_thread_pool_map (line 286) | def test_thread_pool_map(self):
method test_thread_pool_submit (line 297) | def test_thread_pool_submit(self):
method test_as_completed (line 309) | def test_as_completed(self):
method test_future_callback (line 323) | def test_future_callback(self):
method test_future_exception (line 341) | def test_future_exception(self):
method test_future_timeout (line 353) | def test_future_timeout(self):
method test_future_cancel (line 367) | def test_future_cancel(self):
method test_process_pool_map (line 385) | def test_process_pool_map(self):
class TestProducerConsumer (line 395) | class TestProducerConsumer:
method test_basic_producer_consumer (line 398) | def test_basic_producer_consumer(self):
method test_multiple_consumers (line 426) | def test_multiple_consumers(self):
FILE: src/basic/crypto_.py
class TestSecureRandom (line 23) | class TestSecureRandom:
method test_token_bytes (line 26) | def test_token_bytes(self):
method test_token_urlsafe (line 32) | def test_token_urlsafe(self):
method test_token_hex (line 36) | def test_token_hex(self):
method test_randbelow (line 40) | def test_randbelow(self):
class TestHashing (line 46) | class TestHashing:
method test_sha256 (line 49) | def test_sha256(self):
method test_sha3_256 (line 56) | def test_sha3_256(self):
method test_blake2b (line 61) | def test_blake2b(self):
method test_blake2b_keyed (line 66) | def test_blake2b_keyed(self):
class TestHMAC (line 78) | class TestHMAC:
method test_hmac_create_verify (line 81) | def test_hmac_create_verify(self):
method test_hmac_tamper_detection (line 89) | def test_hmac_tamper_detection(self):
class TestKeyDerivation (line 98) | class TestKeyDerivation:
method test_pbkdf2 (line 101) | def test_pbkdf2(self):
method test_pbkdf2_verify (line 113) | def test_pbkdf2_verify(self):
method test_hkdf (line 133) | def test_hkdf(self):
class TestAESGCM (line 146) | class TestAESGCM:
method test_encrypt_decrypt (line 149) | def test_encrypt_decrypt(self):
method test_with_associated_data (line 158) | def test_with_associated_data(self):
method test_tamper_detection (line 168) | def test_tamper_detection(self):
class TestChaCha20Poly1305 (line 180) | class TestChaCha20Poly1305:
method test_encrypt_decrypt (line 183) | def test_encrypt_decrypt(self):
class TestFernet (line 193) | class TestFernet:
method test_encrypt_decrypt (line 196) | def test_encrypt_decrypt(self):
method test_different_tokens (line 204) | def test_different_tokens(self):
class TestRSA (line 214) | class TestRSA:
method test_key_generation (line 217) | def test_key_generation(self):
method test_key_serialization (line 226) | def test_key_serialization(self):
method test_oaep_encrypt_decrypt (line 238) | def test_oaep_encrypt_decrypt(self):
method test_pss_sign_verify (line 263) | def test_pss_sign_verify(self):
class TestEd25519 (line 290) | class TestEd25519:
method test_sign_verify (line 293) | def test_sign_verify(self):
method test_invalid_signature (line 301) | def test_invalid_signature(self):
class TestX25519 (line 310) | class TestX25519:
method test_key_exchange (line 313) | def test_key_exchange(self):
class TestHybridEncryption (line 326) | class TestHybridEncryption:
method test_hybrid_encrypt_decrypt (line 329) | def test_hybrid_encrypt_decrypt(self):
FILE: src/basic/datetime_.py
function test_current_datetime (line 8) | def test_current_datetime():
function test_create_datetime (line 20) | def test_create_datetime():
function test_timestamp_conversion (line 34) | def test_timestamp_conversion():
function test_strftime_formatting (line 45) | def test_strftime_formatting():
function test_strptime_parsing (line 56) | def test_strptime_parsing():
function test_timedelta_arithmetic (line 68) | def test_timedelta_arithmetic():
function test_timezone_operations (line 87) | def test_timezone_operations():
function test_date_comparison (line 102) | def test_date_comparison():
function test_weekday_operations (line 118) | def test_weekday_operations():
function test_start_end_of_day (line 132) | def test_start_end_of_day():
function test_start_end_of_month (line 143) | def test_start_end_of_month():
function test_calendar_operations (line 157) | def test_calendar_operations():
function test_date_range (line 166) | def test_date_range():
function test_age_calculation (line 183) | def test_age_calculation():
function test_time_ago (line 205) | def test_time_ago():
function test_business_days (line 232) | def test_business_days():
FILE: src/basic/dict.py
function create_dict_literal (line 12) | def create_dict_literal():
function create_dict_constructor (line 17) | def create_dict_constructor():
function create_dict_comprehension (line 22) | def create_dict_comprehension(n: int) -> dict:
function get_keys (line 28) | def get_keys(d: dict) -> list:
function get_values (line 33) | def get_values(d: dict) -> list:
function get_items (line 38) | def get_items(d: dict) -> list:
function find_common_keys (line 44) | def find_common_keys(a: dict, b: dict) -> set:
function setdefault_example (line 50) | def setdefault_example():
function defaultdict_example (line 57) | def defaultdict_example():
function merge_dicts_operator (line 65) | def merge_dicts_operator(a: dict, b: dict) -> dict:
function merge_dicts_unpack (line 70) | def merge_dicts_unpack(a: dict, b: dict) -> dict:
function dict_comprehension_filter (line 76) | def dict_comprehension_filter(n: int) -> dict:
function swap_keys_values (line 81) | def swap_keys_values(d: dict) -> dict:
class EmuDict (line 87) | class EmuDict:
method __init__ (line 90) | def __init__(self, data=None):
method __repr__ (line 93) | def __repr__(self):
method __getitem__ (line 96) | def __getitem__(self, key):
method __setitem__ (line 99) | def __setitem__(self, key, val):
method __delitem__ (line 102) | def __delitem__(self, key):
method __contains__ (line 105) | def __contains__(self, key):
method __iter__ (line 108) | def __iter__(self):
method __len__ (line 111) | def __len__(self):
class LRUCache (line 116) | class LRUCache:
method __init__ (line 119) | def __init__(self, maxsize=128):
method get (line 123) | def get(self, key):
method put (line 129) | def put(self, key, value):
function fibonacci (line 138) | def fibonacci(n: int) -> int:
class TestDictCreation (line 146) | class TestDictCreation:
method test_literal (line 147) | def test_literal(self):
method test_constructor (line 150) | def test_constructor(self):
method test_comprehension (line 153) | def test_comprehension(self):
class TestDictAccess (line 157) | class TestDictAccess:
method test_get_keys (line 158) | def test_get_keys(self):
method test_get_values (line 161) | def test_get_values(self):
method test_get_items (line 164) | def test_get_items(self):
class TestDictOperations (line 168) | class TestDictOperations:
method test_find_common_keys (line 169) | def test_find_common_keys(self):
method test_setdefault (line 172) | def test_setdefault(self):
method test_defaultdict (line 175) | def test_defaultdict(self):
method test_merge_operator (line 178) | def test_merge_operator(self):
method test_merge_unpack (line 181) | def test_merge_unpack(self):
class TestDictComprehension (line 185) | class TestDictComprehension:
method test_filter (line 186) | def test_filter(self):
method test_swap (line 189) | def test_swap(self):
class TestEmuDict (line 193) | class TestEmuDict:
method test_getitem (line 194) | def test_getitem(self):
method test_setitem (line 198) | def test_setitem(self):
method test_contains (line 203) | def test_contains(self):
method test_len (line 208) | def test_len(self):
class TestLRUCache (line 212) | class TestLRUCache:
method test_get_put (line 213) | def test_get_put(self):
method test_eviction (line 219) | def test_eviction(self):
method test_fibonacci (line 226) | def test_fibonacci(self):
FILE: src/basic/fileio_.py
function test_read_write_text (line 11) | def test_read_write_text(tmp_path):
function test_read_lines (line 25) | def test_read_lines(tmp_path):
function test_write_modes (line 38) | def test_write_modes(tmp_path):
function test_binary_files (line 53) | def test_binary_files(tmp_path):
function test_pathlib_properties (line 67) | def test_pathlib_properties(tmp_path):
function test_pathlib_with_suffix (line 77) | def test_pathlib_with_suffix():
function test_pathlib_read_write (line 85) | def test_pathlib_read_write(tmp_path):
function test_pathlib_bytes (line 95) | def test_pathlib_bytes(tmp_path):
function test_list_directory (line 106) | def test_list_directory(tmp_path):
function test_glob_pattern (line 121) | def test_glob_pattern(tmp_path):
function test_recursive_glob (line 131) | def test_recursive_glob(tmp_path):
function test_mkdir_parents (line 142) | def test_mkdir_parents(tmp_path):
function test_path_exists (line 151) | def test_path_exists(tmp_path):
function test_temporary_file (line 168) | def test_temporary_file():
function test_temporary_directory (line 179) | def test_temporary_directory():
function test_csv_read_write (line 190) | def test_csv_read_write(tmp_path):
function test_csv_dictreader (line 212) | def test_csv_dictreader(tmp_path):
function test_json_read_write (line 225) | def test_json_read_write(tmp_path):
function test_gzip_read_write (line 239) | def test_gzip_read_write(tmp_path):
function test_zipfile_create_extract (line 253) | def test_zipfile_create_extract(tmp_path):
function test_symlink (line 280) | def test_symlink(tmp_path):
function test_file_stat (line 293) | def test_file_stat(tmp_path):
function test_shutil_copy (line 303) | def test_shutil_copy(tmp_path):
function test_shutil_copytree (line 327) | def test_shutil_copytree(tmp_path):
function test_shutil_copytree_ignore (line 346) | def test_shutil_copytree_ignore(tmp_path):
function test_shutil_copytree_dirs_exist_ok (line 362) | def test_shutil_copytree_dirs_exist_ok(tmp_path):
function test_shutil_move (line 380) | def test_shutil_move(tmp_path):
function test_shutil_rmtree (line 400) | def test_shutil_rmtree(tmp_path):
function test_shutil_disk_usage (line 414) | def test_shutil_disk_usage(tmp_path):
function test_shutil_which (line 424) | def test_shutil_which():
function test_shutil_make_archive (line 435) | def test_shutil_make_archive(tmp_path):
function test_shutil_unpack_archive (line 450) | def test_shutil_unpack_archive(tmp_path):
FILE: src/basic/func.py
function greet (line 12) | def greet(name: str, greeting: str = "Hello") -> str:
function good_default (line 17) | def good_default(items=None):
function sum_all (line 26) | def sum_all(*args) -> int:
function format_info (line 31) | def format_info(**kwargs) -> str:
function mixed_args (line 36) | def mixed_args(a, b=None, *args, **kwargs):
function keyword_only (line 42) | def keyword_only(a, b, *, kw):
function keyword_only_default (line 47) | def keyword_only_default(a, *, kw=10):
function positional_only (line 53) | def positional_only(a, b, /, c):
function combined_args (line 58) | def combined_args(a, /, b, *, c):
function make_multiplier (line 70) | def make_multiplier(n: int):
function make_counter (line 79) | def make_counter():
function fibonacci (line 92) | def fibonacci(n: int):
function log_calls (line 101) | def log_calls(func):
function repeat (line 114) | def repeat(times: int):
class CountCalls (line 131) | class CountCalls:
method __init__ (line 134) | def __init__(self, func):
method __call__ (line 139) | def __call__(self, *args, **kwargs):
function fib_cached (line 146) | def fib_cached(n: int) -> int:
function power (line 154) | def power(base: int, exponent: int) -> int:
function process (line 165) | def process(arg):
function _ (line 171) | def _(arg):
function _ (line 176) | def _(arg):
class Adder (line 181) | class Adder:
method __init__ (line 184) | def __init__(self, n: int):
method __call__ (line 187) | def __call__(self, x: int) -> int:
function apply_twice (line 192) | def apply_twice(func, x):
class TestDefaultArguments (line 198) | class TestDefaultArguments:
method test_default (line 199) | def test_default(self):
method test_custom (line 202) | def test_custom(self):
method test_mutable_default (line 205) | def test_mutable_default(self):
class TestVariableArguments (line 210) | class TestVariableArguments:
method test_args (line 211) | def test_args(self):
method test_kwargs (line 214) | def test_kwargs(self):
method test_mixed (line 219) | def test_mixed(self):
class TestKeywordOnly (line 227) | class TestKeywordOnly:
method test_keyword_only (line 228) | def test_keyword_only(self):
method test_keyword_only_default (line 231) | def test_keyword_only_default(self):
class TestPositionalOnly (line 236) | class TestPositionalOnly:
method test_positional_only (line 237) | def test_positional_only(self):
method test_combined (line 241) | def test_combined(self):
class TestLambda (line 246) | class TestLambda:
method test_square (line 247) | def test_square(self):
method test_add (line 250) | def test_add(self):
method test_conditional (line 253) | def test_conditional(self):
class TestClosure (line 258) | class TestClosure:
method test_multiplier (line 259) | def test_multiplier(self):
method test_counter (line 265) | def test_counter(self):
class TestGenerator (line 272) | class TestGenerator:
method test_fibonacci (line 273) | def test_fibonacci(self):
class TestDecorator (line 277) | class TestDecorator:
method test_log_calls (line 278) | def test_log_calls(self):
method test_repeat (line 288) | def test_repeat(self):
class TestClassDecorator (line 299) | class TestClassDecorator:
method test_count_calls (line 300) | def test_count_calls(self):
class TestCache (line 310) | class TestCache:
method test_fib_cached (line 311) | def test_fib_cached(self):
class TestPartial (line 319) | class TestPartial:
method test_square (line 320) | def test_square(self):
method test_cube (line 323) | def test_cube(self):
class TestSingledispatch (line 327) | class TestSingledispatch:
method test_default (line 328) | def test_default(self):
method test_int (line 331) | def test_int(self):
method test_list (line 334) | def test_list(self):
class TestCallable (line 338) | class TestCallable:
method test_adder (line 339) | def test_adder(self):
class TestHigherOrder (line 345) | class TestHigherOrder:
method test_apply_twice (line 346) | def test_apply_twice(self):
method test_map (line 349) | def test_map(self):
method test_filter (line 352) | def test_filter(self):
method test_reduce (line 355) | def test_reduce(self):
FILE: src/basic/future_.py
function get_all_features (line 13) | def get_all_features() -> list[str]:
function get_feature_info (line 18) | def get_feature_info(name: str) -> tuple:
class Node (line 27) | class Node:
method __init__ (line 30) | def __init__(self, value: int, next: Node | None = None):
method append (line 34) | def append(self, value: int) -> Node:
function get_annotations (line 41) | def get_annotations(func) -> dict:
function true_division (line 47) | def true_division(a: int, b: int) -> float:
function floor_division (line 52) | def floor_division(a: int, b: int) -> int:
function check_version (line 58) | def check_version(major: int, minor: int) -> bool:
function get_version_string (line 63) | def get_version_string() -> str:
class TestFutureFeatures (line 69) | class TestFutureFeatures:
method test_get_all_features (line 70) | def test_get_all_features(self):
method test_get_feature_info (line 76) | def test_get_feature_info(self):
class TestAnnotations (line 82) | class TestAnnotations:
method test_node_creation (line 83) | def test_node_creation(self):
method test_node_append (line 88) | def test_node_append(self):
method test_annotations_are_strings (line 94) | def test_annotations_are_strings(self):
class TestDivision (line 102) | class TestDivision:
method test_true_division (line 103) | def test_true_division(self):
method test_floor_division (line 107) | def test_floor_division(self):
class TestVersionCheck (line 112) | class TestVersionCheck:
method test_check_version (line 113) | def test_check_version(self):
method test_get_version_string (line 119) | def test_get_version_string(self):
FILE: src/basic/generator.py
function simple_gen (line 14) | def simple_gen():
function countdown (line 21) | def countdown(n: int):
function fibonacci (line 28) | def fibonacci(n: int):
function infinite_counter (line 36) | def infinite_counter(start: int = 0):
function gen_expr_sum (line 45) | def gen_expr_sum(n: int) -> int:
function accumulator (line 51) | def accumulator():
function average (line 61) | def average():
function chain (line 75) | def chain(*iterables):
function flatten (line 81) | def flatten(nested):
class Range (line 91) | class Range:
method __init__ (line 94) | def __init__(self, start: int, end: int):
method __iter__ (line 98) | def __iter__(self):
method __reversed__ (line 104) | def __reversed__(self):
function filter_positive (line 112) | def filter_positive(nums):
function double (line 119) | def double(nums):
function read_lines (line 125) | def read_lines(lines):
function filter_comments (line 131) | def filter_comments(lines):
function gen_with_exception (line 139) | def gen_with_exception():
function gen_with_cleanup (line 148) | def gen_with_cleanup():
function capture_output (line 159) | def capture_output():
class TestGeneratorBasics (line 166) | class TestGeneratorBasics:
method test_simple_gen (line 167) | def test_simple_gen(self):
method test_countdown (line 170) | def test_countdown(self):
method test_fibonacci (line 173) | def test_fibonacci(self):
method test_infinite_counter (line 176) | def test_infinite_counter(self):
class TestGeneratorExpression (line 183) | class TestGeneratorExpression:
method test_gen_expr_sum (line 184) | def test_gen_expr_sum(self):
method test_unpack (line 187) | def test_unpack(self):
method test_unpack_multiple (line 191) | def test_unpack_multiple(self):
class TestSend (line 197) | class TestSend:
method test_accumulator (line 198) | def test_accumulator(self):
class TestGeneratorReturn (line 206) | class TestGeneratorReturn:
method test_average (line 207) | def test_average(self):
class TestYieldFrom (line 219) | class TestYieldFrom:
method test_chain (line 220) | def test_chain(self):
method test_flatten (line 223) | def test_flatten(self):
class TestIterableClass (line 227) | class TestIterableClass:
method test_range (line 228) | def test_range(self):
method test_reversed_range (line 231) | def test_reversed_range(self):
class TestPipeline (line 235) | class TestPipeline:
method test_pipeline (line 236) | def test_pipeline(self):
method test_text_pipeline (line 241) | def test_text_pipeline(self):
class TestThrowClose (line 247) | class TestThrowClose:
method test_throw (line 248) | def test_throw(self):
method test_close (line 253) | def test_close(self):
class TestGeneratorState (line 259) | class TestGeneratorState:
method test_states (line 260) | def test_states(self):
class TestGeneratorType (line 275) | class TestGeneratorType:
method test_isinstance (line 276) | def test_isinstance(self):
class TestContextManager (line 284) | class TestContextManager:
method test_capture (line 285) | def test_capture(self):
function prime (line 292) | def prime(n: int):
function closure_gen (line 306) | def closure_gen(start: int = 0):
function fib (line 315) | def fib(n: int) -> int:
function g_fib (line 322) | def g_fib(n: int):
function run_scheduler (line 328) | def run_scheduler(tasks: list) -> list:
function tokenize (line 351) | def tokenize(text: str):
class Node (line 370) | class Node:
method __init__ (line 373) | def __init__(self, *args):
class Number (line 378) | class Number(Node):
class BinOp (line 382) | class BinOp(Node):
function parse (line 386) | def parse(toks):
class NodeVisitor (line 421) | class NodeVisitor:
method visit (line 424) | def visit(self, node):
method genvisit (line 437) | def genvisit(self, node):
class Evaluator (line 444) | class Evaluator(NodeVisitor):
method visit_Number (line 447) | def visit_Number(self, node):
method visit_BinOp (line 450) | def visit_BinOp(self, node):
function evaluate (line 462) | def evaluate(exp: str):
class AsyncIter (line 470) | class AsyncIter:
method __init__ (line 473) | def __init__(self, n):
method __aiter__ (line 476) | def __aiter__(self):
method __anext__ (line 479) | async def __anext__(self):
function agen (line 486) | async def agen(n: int):
class TestPrime (line 493) | class TestPrime:
method test_prime (line 494) | def test_prime(self):
class TestClosure (line 498) | class TestClosure:
method test_closure_gen (line 499) | def test_closure_gen(self):
class TestScheduler (line 506) | class TestScheduler:
method test_round_robin (line 507) | def test_round_robin(self):
class TestCompiler (line 512) | class TestCompiler:
method test_tokenize (line 513) | def test_tokenize(self):
method test_evaluate_simple (line 519) | def test_evaluate_simple(self):
method test_evaluate_complex (line 525) | def test_evaluate_complex(self):
class TestAsyncGen (line 531) | class TestAsyncGen:
method test_async_iter (line 532) | def test_async_iter(self):
method test_async_iter_class (line 540) | def test_async_iter_class(self):
FILE: src/basic/heap.py
function heapify_list (line 12) | def heapify_list(items: list) -> list:
function heap_push (line 19) | def heap_push(h: list, item) -> list:
function heap_pop (line 25) | def heap_pop(h: list):
function heap_pushpop (line 30) | def heap_pushpop(h: list, item):
function heap_replace (line 35) | def heap_replace(h: list, item):
function heap_sort (line 41) | def heap_sort(items: list) -> list:
function max_heap_sort (line 49) | def max_heap_sort(items: list) -> list:
class MaxHeapItem (line 56) | class MaxHeapItem:
method __init__ (line 59) | def __init__(self, val):
method __lt__ (line 62) | def __lt__(self, other):
function priority_queue_example (line 67) | def priority_queue_example():
class Task (line 77) | class Task:
method __init__ (line 80) | def __init__(self, priority: int, name: str):
method __lt__ (line 84) | def __lt__(self, other):
method __repr__ (line 87) | def __repr__(self):
function task_queue (line 91) | def task_queue():
function k_smallest (line 101) | def k_smallest(items: list, k: int) -> list:
function k_largest (line 106) | def k_largest(items: list, k: int) -> list:
function k_largest_by_key (line 111) | def k_largest_by_key(items: list, k: int, key) -> list:
function merge_sorted (line 117) | def merge_sorted(*iterables) -> list:
function merge_sorted_reverse (line 122) | def merge_sorted_reverse(*iterables) -> list:
function top_k (line 128) | def top_k(items: list, k: int) -> list:
class IndexedHeap (line 140) | class IndexedHeap:
method __init__ (line 145) | def __init__(self):
method push (line 149) | def push(self, item, priority):
method remove (line 156) | def remove(self, item):
method pop (line 160) | def pop(self):
class TestBasicHeap (line 170) | class TestBasicHeap:
method test_heapify (line 171) | def test_heapify(self):
method test_push_pop (line 175) | def test_push_pop(self):
method test_pushpop (line 183) | def test_pushpop(self):
method test_replace (line 188) | def test_replace(self):
class TestHeapSort (line 195) | class TestHeapSort:
method test_sort (line 196) | def test_sort(self):
method test_max_sort (line 199) | def test_max_sort(self):
class TestPriorityQueue (line 203) | class TestPriorityQueue:
method test_priority_order (line 204) | def test_priority_order(self):
class TestCustomObjects (line 211) | class TestCustomObjects:
method test_max_heap_item (line 212) | def test_max_heap_item(self):
method test_task_queue (line 218) | def test_task_queue(self):
class TestKElements (line 225) | class TestKElements:
method test_k_smallest (line 226) | def test_k_smallest(self):
method test_k_largest (line 229) | def test_k_largest(self):
method test_k_largest_by_key (line 232) | def test_k_largest_by_key(self):
class TestMerge (line 239) | class TestMerge:
method test_merge_sorted (line 240) | def test_merge_sorted(self):
method test_merge_three (line 243) | def test_merge_three(self):
method test_merge_reverse (line 246) | def test_merge_reverse(self):
class TestTopK (line 250) | class TestTopK:
method test_top_k (line 251) | def test_top_k(self):
class TestIndexedHeap (line 255) | class TestIndexedHeap:
method test_push_pop (line 256) | def test_push_pop(self):
method test_update_priority (line 263) | def test_update_priority(self):
FILE: src/basic/list.py
function init_immutable (line 16) | def init_immutable(n: int) -> list:
function init_mutable (line 21) | def init_mutable(n: int) -> list:
function shallow_copy (line 27) | def shallow_copy(lst: list) -> list:
function deep_copy (line 32) | def deep_copy(lst: list) -> list:
function squares (line 38) | def squares(n: int) -> list:
function filter_even (line 43) | def filter_even(nums: list) -> list:
function flatten (line 48) | def flatten(nested: list) -> list:
function extended_unpack (line 54) | def extended_unpack(lst: list) -> tuple:
function enumerate_example (line 61) | def enumerate_example(items: list) -> list:
function zip_to_dict (line 66) | def zip_to_dict(keys: list, values: list) -> dict:
function unzip (line 71) | def unzip(pairs: list) -> tuple:
function sort_by_key (line 77) | def sort_by_key(items: list, key_func) -> list:
function sort_dicts (line 82) | def sort_dicts(dicts: list, key: str) -> list:
class Stack (line 88) | class Stack:
method __init__ (line 91) | def __init__(self):
method push (line 94) | def push(self, item):
method pop (line 97) | def pop(self):
method peek (line 100) | def peek(self):
method is_empty (line 103) | def is_empty(self):
method __len__ (line 106) | def __len__(self):
class TestInitialize (line 111) | class TestInitialize:
method test_immutable (line 112) | def test_immutable(self):
method test_mutable (line 117) | def test_mutable(self):
class TestCopy (line 123) | class TestCopy:
method test_shallow (line 124) | def test_shallow(self):
method test_deep (line 130) | def test_deep(self):
class TestComprehensions (line 137) | class TestComprehensions:
method test_squares (line 138) | def test_squares(self):
method test_filter_even (line 141) | def test_filter_even(self):
method test_flatten (line 144) | def test_flatten(self):
class TestUnpacking (line 148) | class TestUnpacking:
method test_extended (line 149) | def test_extended(self):
class TestEnumerateZip (line 156) | class TestEnumerateZip:
method test_enumerate (line 157) | def test_enumerate(self):
method test_zip_to_dict (line 160) | def test_zip_to_dict(self):
method test_unzip (line 163) | def test_unzip(self):
class TestSorting (line 169) | class TestSorting:
method test_sort_by_key (line 170) | def test_sort_by_key(self):
method test_sort_dicts (line 173) | def test_sort_dicts(self):
class TestStack (line 178) | class TestStack:
method test_push_pop (line 179) | def test_push_pop(self):
method test_peek (line 186) | def test_peek(self):
method test_is_empty (line 192) | def test_is_empty(self):
function bisect_insort (line 200) | def bisect_insort(items: list) -> list:
function bisect_left_example (line 208) | def bisect_left_example(lst: list, x) -> int:
function bisect_right_example (line 213) | def bisect_right_example(lst: list, x) -> int:
function binary_search (line 218) | def binary_search(arr: list, x, lo: int = 0, hi: int = None) -> int:
function create_2d_list (line 227) | def create_2d_list(rows: int, cols: int) -> list:
function circular_buffer (line 233) | def circular_buffer(items: list, maxlen: int) -> deque:
function chunk (line 242) | def chunk(lst: list, n: int) -> list:
function groupby_example (line 248) | def groupby_example(s: str) -> list:
function create_trie (line 254) | def create_trie(words: list) -> dict:
function trie_has_prefix (line 264) | def trie_has_prefix(trie: dict, prefix: str) -> bool:
class TestBisect (line 274) | class TestBisect:
method test_insort (line 275) | def test_insort(self):
method test_bisect_left (line 278) | def test_bisect_left(self):
method test_bisect_right (line 282) | def test_bisect_right(self):
method test_binary_search (line 286) | def test_binary_search(self):
class TestNestedLists (line 293) | class TestNestedLists:
method test_create_2d (line 294) | def test_create_2d(self):
class TestDeque (line 300) | class TestDeque:
method test_circular_buffer (line 301) | def test_circular_buffer(self):
class TestChunk (line 306) | class TestChunk:
method test_chunk (line 307) | def test_chunk(self):
class TestGroupby (line 315) | class TestGroupby:
method test_groupby (line 316) | def test_groupby(self):
class TestTrie (line 325) | class TestTrie:
method test_create_and_search (line 326) | def test_create_and_search(self):
FILE: src/basic/object.py
class Person (line 13) | class Person:
method __init__ (line 16) | def __init__(self, name: str, age: int):
method greet (line 20) | def greet(self) -> str:
class Counter (line 25) | class Counter:
method __init__ (line 30) | def __init__(self):
class Animal (line 36) | class Animal:
method __init__ (line 39) | def __init__(self, name: str):
method speak (line 42) | def speak(self) -> str:
class Dog (line 46) | class Dog(Animal):
method speak (line 47) | def speak(self) -> str:
class Cat (line 51) | class Cat(Animal):
method speak (line 52) | def speak(self) -> str:
class Vector (line 57) | class Vector:
method __init__ (line 60) | def __init__(self, x: int, y: int):
method __repr__ (line 63) | def __repr__(self):
method __str__ (line 66) | def __str__(self):
method __add__ (line 69) | def __add__(self, other):
method __mul__ (line 72) | def __mul__(self, scalar):
method __eq__ (line 75) | def __eq__(self, other):
class Number (line 81) | class Number:
method __init__ (line 84) | def __init__(self, val):
method __eq__ (line 87) | def __eq__(self, other):
method __lt__ (line 90) | def __lt__(self, other):
class Multiplier (line 95) | class Multiplier:
method __init__ (line 98) | def __init__(self, factor: int):
method __call__ (line 101) | def __call__(self, x: int) -> int:
class Circle (line 106) | class Circle:
method __init__ (line 109) | def __init__(self, radius: float):
method radius (line 113) | def radius(self) -> float:
method radius (line 117) | def radius(self, value: float):
method area (line 123) | def area(self) -> float:
class Positive (line 128) | class Positive:
method __init__ (line 131) | def __init__(self, name):
method __get__ (line 134) | def __get__(self, obj, objtype=None):
method __set__ (line 139) | def __set__(self, obj, value):
class DescriptorExample (line 145) | class DescriptorExample:
method __init__ (line 148) | def __init__(self, x):
class ManagedResource (line 153) | class ManagedResource:
method __init__ (line 156) | def __init__(self):
method __enter__ (line 160) | def __enter__(self):
method __exit__ (line 164) | def __exit__(self, exc_type, exc_val, exc_tb):
class Singleton (line 170) | class Singleton:
method __new__ (line 175) | def __new__(cls):
class Date (line 182) | class Date:
method __init__ (line 185) | def __init__(self, year: int, month: int, day: int):
method from_string (line 189) | def from_string(cls, date_string: str):
method is_valid (line 194) | def is_valid(date_string: str) -> bool:
class Shape (line 203) | class Shape(ABC):
method area (line 207) | def area(self) -> float:
class Rectangle (line 211) | class Rectangle(Shape):
method __init__ (line 212) | def __init__(self, width: float, height: float):
method area (line 215) | def area(self) -> float:
class A (line 220) | class A:
method method (line 221) | def method(self):
class B (line 225) | class B(A):
method method (line 226) | def method(self):
class C (line 230) | class C(A):
method method (line 231) | def method(self):
class D (line 235) | class D(B, C):
class PointWithSlots (line 240) | class PointWithSlots:
method __init__ (line 245) | def __init__(self, x, y):
class TestBasicClass (line 250) | class TestBasicClass:
method test_person (line 251) | def test_person(self):
class TestClassAttributes (line 257) | class TestClassAttributes:
method test_counter (line 258) | def test_counter(self):
class TestInheritance (line 266) | class TestInheritance:
method test_dog (line 267) | def test_dog(self):
method test_cat (line 270) | def test_cat(self):
class TestMagicMethods (line 274) | class TestMagicMethods:
method test_vector_add (line 275) | def test_vector_add(self):
method test_vector_mul (line 280) | def test_vector_mul(self):
method test_vector_repr (line 284) | def test_vector_repr(self):
method test_vector_str (line 287) | def test_vector_str(self):
class TestComparison (line 291) | class TestComparison:
method test_total_ordering (line 292) | def test_total_ordering(self):
class TestCallable (line 300) | class TestCallable:
method test_multiplier (line 301) | def test_multiplier(self):
class TestProperty (line 307) | class TestProperty:
method test_circle_area (line 308) | def test_circle_area(self):
method test_circle_setter (line 312) | def test_circle_setter(self):
method test_circle_invalid (line 317) | def test_circle_invalid(self):
class TestDescriptor (line 323) | class TestDescriptor:
method test_positive (line 324) | def test_positive(self):
method test_positive_invalid (line 328) | def test_positive_invalid(self):
class TestContextManager (line 333) | class TestContextManager:
method test_managed_resource (line 334) | def test_managed_resource(self):
class TestSingleton (line 340) | class TestSingleton:
method test_singleton (line 341) | def test_singleton(self):
class TestClassMethods (line 348) | class TestClassMethods:
method test_from_string (line 349) | def test_from_string(self):
method test_is_valid (line 354) | def test_is_valid(self):
class TestABC (line 359) | class TestABC:
method test_rectangle (line 360) | def test_rectangle(self):
method test_abstract_instantiation (line 364) | def test_abstract_instantiation(self):
class TestMRO (line 369) | class TestMRO:
method test_diamond (line 370) | def test_diamond(self):
method test_mro (line 373) | def test_mro(self):
class TestSlots (line 377) | class TestSlots:
method test_slots (line 378) | def test_slots(self):
method test_slots_no_dict (line 383) | def test_slots_no_dict(self):
FILE: src/basic/os_.py
class TestSystemInfo (line 17) | class TestSystemInfo:
method test_os_name (line 20) | def test_os_name(self):
method test_platform_system (line 24) | def test_platform_system(self):
method test_cpu_count (line 28) | def test_cpu_count(self):
method test_getpid (line 34) | def test_getpid(self):
method test_getcwd (line 39) | def test_getcwd(self):
class TestEnvironmentVariables (line 45) | class TestEnvironmentVariables:
method test_get_env (line 48) | def test_get_env(self):
method test_getenv_default (line 54) | def test_getenv_default(self):
method test_set_env (line 59) | def test_set_env(self):
method test_env_not_found (line 65) | def test_env_not_found(self):
class TestPathOperations (line 71) | class TestPathOperations:
method test_join (line 74) | def test_join(self):
method test_dirname_basename (line 81) | def test_dirname_basename(self):
method test_splitext (line 87) | def test_splitext(self):
method test_abspath (line 93) | def test_abspath(self):
method test_exists (line 98) | def test_exists(self):
method test_isfile_isdir (line 103) | def test_isfile_isdir(self):
class TestDirectoryOperations (line 111) | class TestDirectoryOperations:
method test_mkdir_rmdir (line 114) | def test_mkdir_rmdir(self):
method test_makedirs (line 123) | def test_makedirs(self):
method test_makedirs_exist_ok (line 130) | def test_makedirs_exist_ok(self):
method test_listdir (line 135) | def test_listdir(self):
method test_walk (line 145) | def test_walk(self):
class TestFileOperations (line 162) | class TestFileOperations:
method test_rename (line 165) | def test_rename(self):
method test_remove (line 175) | def test_remove(self):
method test_getsize (line 183) | def test_getsize(self):
class TestSubprocess (line 195) | class TestSubprocess:
method test_run_simple (line 198) | def test_run_simple(self):
method test_check_output (line 206) | def test_check_output(self):
method test_run_with_input (line 211) | def test_run_with_input(self):
class TestTempFiles (line 219) | class TestTempFiles:
method test_named_temp_file (line 222) | def test_named_temp_file(self):
method test_temp_directory (line 230) | def test_temp_directory(self):
method test_gettempdir (line 241) | def test_gettempdir(self):
class TestPathlib (line 247) | class TestPathlib:
method test_path_creation (line 250) | def test_path_creation(self):
method test_path_join (line 257) | def test_path_join(self):
method test_path_parent (line 262) | def test_path_parent(self):
method test_path_exists (line 267) | def test_path_exists(self):
method test_read_write_text (line 272) | def test_read_write_text(self):
method test_mkdir (line 279) | def test_mkdir(self):
method test_glob (line 286) | def test_glob(self):
method test_iterdir (line 296) | def test_iterdir(self):
class TestPsutil (line 306) | class TestPsutil:
method psutil_available (line 310) | def psutil_available(self):
method test_cpu_count (line 319) | def test_cpu_count(self, psutil_available):
method test_cpu_percent (line 328) | def test_cpu_percent(self, psutil_available):
method test_virtual_memory (line 334) | def test_virtual_memory(self, psutil_available):
method test_disk_usage (line 342) | def test_disk_usage(self, psutil_available):
method test_process (line 350) | def test_process(self, psutil_available):
method test_boot_time (line 358) | def test_boot_time(self, psutil_available):
FILE: src/basic/rexp.py
function search_pattern (line 13) | def search_pattern(pattern: str, text: str) -> str | None:
function match_start (line 19) | def match_start(pattern: str, text: str) -> bool:
function fullmatch (line 24) | def fullmatch(pattern: str, text: str) -> bool:
function find_all (line 30) | def find_all(pattern: str, text: str) -> list:
function find_all_groups (line 35) | def find_all_groups(pattern: str, text: str) -> list:
function split_pattern (line 41) | def split_pattern(pattern: str, text: str) -> list:
function parse_date (line 47) | def parse_date(text: str) -> dict | None:
function parse_date_named (line 55) | def parse_date_named(text: str) -> dict | None:
function parse_url (line 63) | def parse_url(url: str) -> tuple | None:
function has_repeated_char (line 70) | def has_repeated_char(text: str) -> bool:
function match_html_tag (line 75) | def match_html_tag(text: str) -> str | None:
function find_before_at (line 82) | def find_before_at(text: str) -> list:
function find_after_dollar (line 87) | def find_after_dollar(text: str) -> list:
function find_not_followed_by (line 92) | def find_not_followed_by(text: str, suffix: str) -> list:
function replace_pattern (line 98) | def replace_pattern(pattern: str, repl: str, text: str) -> str:
function double_numbers (line 103) | def double_numbers(text: str) -> str:
function camel_to_snake (line 108) | def camel_to_snake(s: str) -> str:
function is_valid_email (line 133) | def is_valid_email(text: str) -> bool:
function is_valid_ip (line 138) | def is_valid_ip(text: str) -> bool:
function is_valid_mac (line 143) | def is_valid_mac(text: str) -> bool:
function is_valid_url (line 148) | def is_valid_url(text: str) -> bool:
function is_valid_hex_color (line 153) | def is_valid_hex_color(text: str) -> bool:
function is_valid_phone (line 158) | def is_valid_phone(text: str) -> bool:
function is_strong_password (line 163) | def is_strong_password(text: str) -> bool:
function find_open_tags (line 169) | def find_open_tags(html: str) -> list:
function find_close_tags (line 174) | def find_close_tags(html: str) -> list:
function strip_html_tags (line 179) | def strip_html_tags(html: str) -> str:
function tokenize (line 188) | def tokenize(text: str) -> list:
function find_hashtags (line 208) | def find_hashtags(text: str) -> list:
function find_mentions (line 213) | def find_mentions(text: str) -> list:
function extract_domain (line 218) | def extract_domain(url: str) -> str | None:
class TestBasicMatching (line 225) | class TestBasicMatching:
method test_search (line 226) | def test_search(self):
method test_match_start (line 230) | def test_match_start(self):
method test_fullmatch (line 234) | def test_fullmatch(self):
class TestFindAll (line 239) | class TestFindAll:
method test_find_all (line 240) | def test_find_all(self):
method test_find_all_groups (line 243) | def test_find_all_groups(self):
class TestSplit (line 250) | class TestSplit:
method test_split (line 251) | def test_split(self):
class TestGroups (line 256) | class TestGroups:
method test_parse_date (line 257) | def test_parse_date(self):
method test_parse_date_named (line 261) | def test_parse_date_named(self):
method test_parse_url (line 265) | def test_parse_url(self):
class TestBackReference (line 269) | class TestBackReference:
method test_repeated_char (line 270) | def test_repeated_char(self):
method test_html_tag (line 274) | def test_html_tag(self):
class TestLookaround (line 279) | class TestLookaround:
method test_lookahead (line 280) | def test_lookahead(self):
method test_lookbehind (line 283) | def test_lookbehind(self):
method test_negative_lookahead (line 286) | def test_negative_lookahead(self):
class TestSubstitution (line 290) | class TestSubstitution:
method test_replace (line 291) | def test_replace(self):
method test_double_numbers (line 294) | def test_double_numbers(self):
method test_camel_to_snake (line 297) | def test_camel_to_snake(self):
class TestValidation (line 302) | class TestValidation:
method test_email (line 303) | def test_email(self):
method test_ip (line 308) | def test_ip(self):
method test_mac (line 313) | def test_mac(self):
method test_url (line 317) | def test_url(self):
method test_hex_color (line 321) | def test_hex_color(self):
method test_phone (line 326) | def test_phone(self):
method test_password (line 330) | def test_password(self):
class TestHtmlTags (line 335) | class TestHtmlTags:
method test_open_tags (line 336) | def test_open_tags(self):
method test_close_tags (line 339) | def test_close_tags(self):
method test_strip_tags (line 342) | def test_strip_tags(self):
class TestLexer (line 346) | class TestLexer:
method test_tokenize (line 347) | def test_tokenize(self):
class TestUtility (line 354) | class TestUtility:
method test_hashtags (line 355) | def test_hashtags(self):
method test_mentions (line 358) | def test_mentions(self):
method test_extract_domain (line 361) | def test_extract_domain(self):
FILE: src/basic/set.py
function create_set_literal (line 10) | def create_set_literal():
function create_set_from_list (line 15) | def create_set_from_list(items: list) -> set:
function create_empty_set (line 20) | def create_empty_set() -> set:
function set_comprehension_basic (line 26) | def set_comprehension_basic(items: list) -> set:
function set_comprehension_filter (line 31) | def set_comprehension_filter(items: list, threshold: int) -> set:
function set_comprehension_squares (line 36) | def set_comprehension_squares(n: int) -> set:
function uniquify_list (line 42) | def uniquify_list(items: list) -> list:
function uniquify_preserve_order (line 47) | def uniquify_preserve_order(items: list) -> list:
function add_single (line 53) | def add_single(s: set, item) -> set:
function add_multiple (line 59) | def add_multiple(s: set, items) -> set:
function remove_item (line 66) | def remove_item(s: set, item) -> set:
function discard_item (line 72) | def discard_item(s: set, item) -> set:
function pop_item (line 78) | def pop_item(s: set):
function union (line 84) | def union(a: set, b: set) -> set:
function intersection (line 89) | def intersection(a: set, b: set) -> set:
function difference (line 94) | def difference(a: set, b: set) -> set:
function symmetric_difference (line 99) | def symmetric_difference(a: set, b: set) -> set:
function is_subset (line 104) | def is_subset(a: set, b: set) -> bool:
function is_proper_subset (line 109) | def is_proper_subset(a: set, b: set) -> bool:
function is_superset (line 114) | def is_superset(a: set, b: set) -> bool:
function is_disjoint (line 119) | def is_disjoint(a: set, b: set) -> bool:
function membership_test (line 125) | def membership_test(s: set, item) -> bool:
function create_frozenset (line 131) | def create_frozenset(items: list) -> frozenset:
function frozenset_as_dict_key (line 136) | def frozenset_as_dict_key():
function frozenset_in_set (line 141) | def frozenset_in_set():
class TestSetCreation (line 147) | class TestSetCreation:
method test_literal (line 148) | def test_literal(self):
method test_from_list (line 151) | def test_from_list(self):
method test_empty (line 154) | def test_empty(self):
class TestSetComprehension (line 159) | class TestSetComprehension:
method test_basic (line 160) | def test_basic(self):
method test_filter (line 163) | def test_filter(self):
method test_squares (line 166) | def test_squares(self):
class TestUniquify (line 170) | class TestUniquify:
method test_uniquify (line 171) | def test_uniquify(self):
method test_preserve_order (line 175) | def test_preserve_order(self):
class TestAddRemove (line 179) | class TestAddRemove:
method test_add_single (line 180) | def test_add_single(self):
method test_add_multiple (line 184) | def test_add_multiple(self):
method test_remove (line 188) | def test_remove(self):
method test_remove_missing (line 192) | def test_remove_missing(self):
method test_discard (line 197) | def test_discard(self):
method test_pop (line 202) | def test_pop(self):
class TestSetOperations (line 209) | class TestSetOperations:
method test_union (line 210) | def test_union(self):
method test_intersection (line 213) | def test_intersection(self):
method test_difference (line 216) | def test_difference(self):
method test_symmetric_difference (line 219) | def test_symmetric_difference(self):
method test_subset (line 222) | def test_subset(self):
method test_proper_subset (line 227) | def test_proper_subset(self):
method test_superset (line 231) | def test_superset(self):
method test_disjoint (line 235) | def test_disjoint(self):
class TestMembership (line 240) | class TestMembership:
method test_in (line 241) | def test_in(self):
class TestFrozenset (line 246) | class TestFrozenset:
method test_create (line 247) | def test_create(self):
method test_immutable (line 251) | def test_immutable(self):
method test_as_dict_key (line 255) | def test_as_dict_key(self):
method test_in_set (line 259) | def test_in_set(self):
FILE: src/basic/socket_.py
class TestHostname (line 9) | class TestHostname:
method test_gethostname (line 12) | def test_gethostname(self):
method test_gethostbyname_localhost (line 17) | def test_gethostbyname_localhost(self):
method test_getaddrinfo (line 21) | def test_getaddrinfo(self):
class TestByteOrder (line 30) | class TestByteOrder:
method test_htons (line 33) | def test_htons(self):
method test_htonl (line 39) | def test_htonl(self):
method test_ntohs (line 44) | def test_ntohs(self):
method test_ntohl (line 49) | def test_ntohl(self):
class TestIPConversion (line 55) | class TestIPConversion:
method test_inet_aton (line 58) | def test_inet_aton(self):
method test_inet_ntoa (line 62) | def test_inet_ntoa(self):
method test_inet_pton_ipv4 (line 66) | def test_inet_pton_ipv4(self):
method test_inet_ntop_ipv4 (line 70) | def test_inet_ntop_ipv4(self):
method test_inet_pton_ipv6 (line 74) | def test_inet_pton_ipv6(self):
method test_inet_ntop_ipv6 (line 78) | def test_inet_ntop_ipv6(self):
class TestSocketOptions (line 83) | class TestSocketOptions:
method test_reuseaddr (line 86) | def test_reuseaddr(self):
method test_timeout (line 93) | def test_timeout(self):
method test_blocking (line 99) | def test_blocking(self):
class TestTCPEchoServer (line 108) | class TestTCPEchoServer:
method test_echo (line 111) | def test_echo(self):
class TestUDPEchoServer (line 145) | class TestUDPEchoServer:
method test_echo (line 148) | def test_echo(self):
class TestSocketPair (line 176) | class TestSocketPair:
method test_socketpair (line 179) | def test_socketpair(self):
class TestPortCheck (line 194) | class TestPortCheck:
method test_port_available (line 197) | def test_port_available(self):
method test_port_in_use (line 206) | def test_port_in_use(self):
class TestMACConversion (line 223) | class TestMACConversion:
method test_mac_to_bytes (line 226) | def test_mac_to_bytes(self):
method test_bytes_to_mac (line 233) | def test_bytes_to_mac(self):
class TestSelectorsEcho (line 241) | class TestSelectorsEcho:
method test_selectors_echo (line 244) | def test_selectors_echo(self):
FILE: src/basic/sqlalchemy_core.py
class TestEngine (line 44) | class TestEngine:
method test_create_sqlite_memory (line 47) | def test_create_sqlite_memory(self):
method test_create_sqlite_file (line 51) | def test_create_sqlite_file(self, tmp_path):
class TestRawSQL (line 57) | class TestRawSQL:
method test_execute_raw_sql (line 60) | def test_execute_raw_sql(self):
class TestTransaction (line 77) | class TestTransaction:
method test_begin_commit (line 80) | def test_begin_commit(self):
class TestMetadata (line 92) | class TestMetadata:
method test_define_table (line 95) | def test_define_table(self):
method test_reflect_table (line 108) | def test_reflect_table(self):
class TestInspect (line 121) | class TestInspect:
method test_get_table_names (line 124) | def test_get_table_names(self):
class TestCoreInsert (line 133) | class TestCoreInsert:
method test_single_insert (line 136) | def test_single_insert(self):
method test_bulk_insert (line 154) | def test_bulk_insert(self):
class TestCoreSelect (line 171) | class TestCoreSelect:
method test_select_all (line 174) | def test_select_all(self):
method test_select_where (line 194) | def test_select_where(self):
class TestCoreUpdate (line 217) | class TestCoreUpdate:
method test_update (line 220) | def test_update(self):
class TestCoreDelete (line 242) | class TestCoreDelete:
method test_delete (line 245) | def test_delete(self):
class TestExpressionLanguage (line 265) | class TestExpressionLanguage:
method test_and_condition (line 268) | def test_and_condition(self):
method test_or_condition (line 297) | def test_or_condition(self):
method test_in_clause (line 319) | def test_in_clause(self):
class TestCoreJoin (line 340) | class TestCoreJoin:
method test_join (line 343) | def test_join(self):
class TestAggregate (line 379) | class TestAggregate:
method test_count (line 382) | def test_count(self):
method test_sum_group_by (line 398) | def test_sum_group_by(self):
class TestDropTable (line 428) | class TestDropTable:
method test_drop_single (line 431) | def test_drop_single(self):
method test_drop_all (line 442) | def test_drop_all(self):
FILE: src/basic/sqlalchemy_orm.py
class TestDeclarativeBase (line 33) | class TestDeclarativeBase:
method test_define_model (line 36) | def test_define_model(self):
class TestSession (line 49) | class TestSession:
method test_add_commit (line 52) | def test_add_commit(self):
method test_add_all (line 72) | def test_add_all(self):
class TestORMQuery (line 93) | class TestORMQuery:
method test_select_all (line 96) | def test_select_all(self):
method test_filter_where (line 116) | def test_filter_where(self):
class TestORMFilter (line 144) | class TestORMFilter:
method test_and_filter (line 147) | def test_and_filter(self):
method test_or_filter (line 177) | def test_or_filter(self):
method test_in_filter (line 202) | def test_in_filter(self):
class TestORMUpdate (line 224) | class TestORMUpdate:
method test_update_object (line 227) | def test_update_object(self):
class TestORMDelete (line 251) | class TestORMDelete:
method test_delete_object (line 254) | def test_delete_object(self):
class TestOneToMany (line 283) | class TestOneToMany:
method test_relationship (line 286) | def test_relationship(self):
class TestManyToMany (line 318) | class TestManyToMany:
method test_relationship (line 321) | def test_relationship(self):
class TestSelfReferential (line 377) | class TestSelfReferential:
method test_hierarchy (line 380) | def test_hierarchy(self):
class TestCascade (line 419) | class TestCascade:
method test_delete_orphan (line 422) | def test_delete_orphan(self):
class TestEagerLoading (line 452) | class TestEagerLoading:
method test_joinedload (line 455) | def test_joinedload(self):
class TestHybridProperty (line 487) | class TestHybridProperty:
method test_full_name (line 490) | def test_full_name(self):
class TestEventHooks (line 516) | class TestEventHooks:
method test_before_insert (line 519) | def test_before_insert(self):
FILE: src/basic/sqlalchemy_query.py
class TestOrderBy (line 33) | class TestOrderBy:
method test_ascending (line 36) | def test_ascending(self):
method test_descending (line 64) | def test_descending(self):
class TestLimitOffset (line 86) | class TestLimitOffset:
method test_limit (line 89) | def test_limit(self):
method test_offset (line 110) | def test_offset(self):
class TestGroupBy (line 132) | class TestGroupBy:
method test_sum_group_by (line 135) | def test_sum_group_by(self):
method test_having (line 166) | def test_having(self):
class TestJoin (line 195) | class TestJoin:
method test_inner_join (line 198) | def test_inner_join(self):
method test_outer_join (line 233) | def test_outer_join(self):
class TestSubquery (line 267) | class TestSubquery:
method test_scalar_subquery (line 270) | def test_scalar_subquery(self):
class TestCTE (line 300) | class TestCTE:
method test_cte (line 303) | def test_cte(self):
class TestExists (line 338) | class TestExists:
method test_exists (line 341) | def test_exists(self):
method test_not_exists (line 376) | def test_not_exists(self):
class TestUnion (line 412) | class TestUnion:
method test_union_all (line 415) | def test_union_all(self):
class TestCase (line 444) | class TestCase:
method test_case (line 447) | def test_case(self):
class TestDistinct (line 481) | class TestDistinct:
method test_distinct (line 484) | def test_distinct(self):
method test_count_distinct (line 511) | def test_count_distinct(self):
class TestAliased (line 539) | class TestAliased:
method test_aliased (line 542) | def test_aliased(self):
class TestRawSQL (line 579) | class TestRawSQL:
method test_text (line 582) | def test_text(self):
FILE: src/basic/typing_.py
function greet (line 22) | def greet(name: str) -> str:
function add (line 27) | def add(a: int, b: int) -> int:
function sum_list (line 33) | def sum_list(numbers: list[int]) -> int:
function get_value (line 38) | def get_value(data: dict[str, int], key: str) -> int | None:
function find_user (line 44) | def find_user(user_id: int) -> Optional[str]:
function process (line 50) | def process(value: int | str) -> str:
function apply (line 56) | def apply(func: Callable[[int, int], int], a: int, b: int) -> int:
function first (line 68) | def first(items: list[T]) -> T:
function double_num (line 76) | def double_num(x: Number) -> Number:
class Stack (line 82) | class Stack(Generic[T]):
method __init__ (line 85) | def __init__(self) -> None:
method push (line 88) | def push(self, item: T) -> None:
method pop (line 91) | def pop(self) -> T:
method is_empty (line 94) | def is_empty(self) -> bool:
class Drawable (line 99) | class Drawable(Protocol):
method draw (line 102) | def draw(self) -> str: ...
class Circle (line 105) | class Circle:
method draw (line 108) | def draw(self) -> str:
class Square (line 112) | class Square:
method draw (line 115) | def draw(self) -> str:
function render (line 119) | def render(shape: Drawable) -> str:
class UserDict (line 125) | class UserDict(TypedDict):
function set_status (line 133) | def set_status(status: Literal["active", "inactive"]) -> str:
class Config (line 143) | class Config:
method __init__ (line 149) | def __init__(self, name: str) -> None:
class TestBasicTypes (line 154) | class TestBasicTypes:
method test_greet (line 155) | def test_greet(self):
method test_add (line 158) | def test_add(self):
class TestCollections (line 162) | class TestCollections:
method test_sum_list (line 163) | def test_sum_list(self):
method test_get_value (line 166) | def test_get_value(self):
class TestOptionalUnion (line 171) | class TestOptionalUnion:
method test_find_user (line 172) | def test_find_user(self):
method test_process (line 176) | def test_process(self):
class TestCallable (line 181) | class TestCallable:
method test_apply (line 182) | def test_apply(self):
method test_double (line 185) | def test_double(self):
class TestGenerics (line 189) | class TestGenerics:
method test_first (line 190) | def test_first(self):
method test_double_num (line 194) | def test_double_num(self):
class TestGenericClass (line 199) | class TestGenericClass:
method test_stack (line 200) | def test_stack(self):
class TestProtocol (line 209) | class TestProtocol:
method test_render (line 210) | def test_render(self):
class TestTypedDict (line 215) | class TestTypedDict:
method test_user_dict (line 216) | def test_user_dict(self):
class TestLiteral (line 222) | class TestLiteral:
method test_set_status (line 223) | def test_set_status(self):
class TestClassVar (line 227) | class TestClassVar:
method test_config (line 228) | def test_config(self):
FILE: src/basic/unicode_.py
function encode_utf8 (line 11) | def encode_utf8(s: str) -> bytes:
function decode_utf8 (line 16) | def decode_utf8(b: bytes) -> str:
function encode_with_errors (line 21) | def encode_with_errors(s: str, encoding: str, errors: str) -> bytes:
function get_code_point (line 27) | def get_code_point(char: str) -> int:
function get_char (line 32) | def get_char(code_point: int) -> str:
function format_code_points (line 37) | def format_code_points(s: str) -> list[str]:
function normalize_nfc (line 43) | def normalize_nfc(s: str) -> str:
function normalize_nfd (line 48) | def normalize_nfd(s: str) -> str:
function get_char_name (line 54) | def get_char_name(char: str) -> str:
function get_char_category (line 59) | def get_char_category(char: str) -> str:
function lookup_char (line 64) | def lookup_char(name: str) -> str:
function case_insensitive_equal (line 70) | def case_insensitive_equal(s1: str, s2: str) -> bool:
class TestEncodingDecoding (line 76) | class TestEncodingDecoding:
method test_encode_utf8 (line 77) | def test_encode_utf8(self):
method test_decode_utf8 (line 80) | def test_decode_utf8(self):
method test_roundtrip (line 83) | def test_roundtrip(self):
method test_encode_errors_ignore (line 87) | def test_encode_errors_ignore(self):
method test_encode_errors_replace (line 90) | def test_encode_errors_replace(self):
class TestCodePoints (line 94) | class TestCodePoints:
method test_get_code_point (line 95) | def test_get_code_point(self):
method test_get_char (line 100) | def test_get_char(self):
method test_format_code_points (line 105) | def test_format_code_points(self):
class TestNormalization (line 109) | class TestNormalization:
method test_nfc (line 110) | def test_nfc(self):
method test_nfd (line 116) | def test_nfd(self):
method test_normalization_equality (line 121) | def test_normalization_equality(self):
class TestCharInfo (line 128) | class TestCharInfo:
method test_get_char_name (line 129) | def test_get_char_name(self):
method test_get_char_category (line 133) | def test_get_char_category(self):
method test_lookup_char (line 138) | def test_lookup_char(self):
class TestStringOperations (line 143) | class TestStringOperations:
method test_case_insensitive (line 144) | def test_case_insensitive(self):
method test_unicode_upper_lower (line 148) | def test_unicode_upper_lower(self):
method test_unicode_isalpha (line 152) | def test_unicode_isalpha(self):
FILE: src/cext/capi/args.c
function PyObject (line 5) | static PyObject* no_args(PyObject* self) { Py_RETURN_NONE; }
function PyObject (line 8) | static PyObject* single_arg(PyObject* self, PyObject* arg) { return Py_B...
function PyObject (line 11) | static PyObject* pos_args(PyObject* self, PyObject* args) {
function PyObject (line 20) | static PyObject* kw_args(PyObject* self, PyObject* args, PyObject* kwarg...
function PyObject (line 31) | static PyObject* typed_args(PyObject* self, PyObject* args) {
type PyModuleDef (line 51) | struct PyModuleDef
function PyMODINIT_FUNC (line 53) | PyMODINIT_FUNC PyInit_args(void) { return PyModule_Create(&module); }
FILE: src/cext/capi/errors.c
function PyObject (line 7) | static PyObject* raise_value_error(PyObject* self) {
function PyObject (line 13) | static PyObject* raise_foo_error(PyObject* self) {
function PyObject (line 19) | static PyObject* raise_with_format(PyObject* self, PyObject* args) {
function PyObject (line 29) | static PyObject* divide(PyObject* self, PyObject* args) {
type PyModuleDef (line 49) | struct PyModuleDef
function PyMODINIT_FUNC (line 51) | PyMODINIT_FUNC PyInit_errors(void) {
FILE: src/cext/capi/gil.c
function PyObject (line 12) | static PyObject* sleep_with_gil(PyObject* self, PyObject* args) {
function PyObject (line 22) | static PyObject* sleep_no_gil(PyObject* self, PyObject* args) {
function fib_impl (line 35) | static unsigned long fib_impl(unsigned long n) {
function PyObject (line 40) | static PyObject* fib_no_gil(PyObject* self, PyObject* args) {
type PyModuleDef (line 59) | struct PyModuleDef
function PyMODINIT_FUNC (line 61) | PyMODINIT_FUNC PyInit_gil(void) { return PyModule_Create(&module); }
FILE: src/cext/capi/simple.c
function PyObject (line 9) | static PyObject* hello(PyObject* self) { return PyUnicode_FromString("He...
function PyObject (line 11) | static PyObject* add(PyObject* self, PyObject* args) {
function fib_impl (line 19) | static unsigned long fib_impl(unsigned long n) {
function PyObject (line 24) | static PyObject* fib(PyObject* self, PyObject* args) {
type PyModuleDef (line 39) | struct PyModuleDef
function PyMODINIT_FUNC (line 41) | PyMODINIT_FUNC PyInit_simple(void) { return PyModule_Create(&module); }
FILE: src/cext/capi/test_capi.py
class TestSimple (line 15) | class TestSimple:
method test_hello (line 18) | def test_hello(self):
method test_add (line 23) | def test_add(self):
method test_fib (line 29) | def test_fib(self):
class TestArgs (line 38) | class TestArgs:
method test_no_args (line 41) | def test_no_args(self):
method test_single_arg (line 46) | def test_single_arg(self):
method test_pos_args (line 52) | def test_pos_args(self):
method test_kw_args (line 58) | def test_kw_args(self):
method test_typed_args (line 65) | def test_typed_args(self):
class TestGil (line 72) | class TestGil:
method test_fib_no_gil (line 75) | def test_fib_no_gil(self):
class TestErrors (line 82) | class TestErrors:
method test_raise_value_error (line 85) | def test_raise_value_error(self):
method test_raise_foo_error (line 91) | def test_raise_foo_error(self):
method test_raise_with_format (line 97) | def test_raise_with_format(self):
method test_divide (line 103) | def test_divide(self):
class TestTypesDemo (line 111) | class TestTypesDemo:
method test_list_demo (line 114) | def test_list_demo(self):
method test_list_sum (line 119) | def test_list_sum(self):
method test_iter_list (line 124) | def test_iter_list(self):
method test_dict_demo (line 129) | def test_dict_demo(self):
method test_dict_get (line 134) | def test_dict_get(self):
method test_iter_dict (line 141) | def test_iter_dict(self):
method test_tuple_demo (line 148) | def test_tuple_demo(self):
method test_tuple_unpack (line 153) | def test_tuple_unpack(self):
method test_set_demo (line 159) | def test_set_demo(self):
method test_set_contains (line 164) | def test_set_contains(self):
method test_str_demo (line 171) | def test_str_demo(self):
method test_str_format (line 176) | def test_str_format(self):
method test_bytes_demo (line 181) | def test_bytes_demo(self):
method test_bytes_len (line 186) | def test_bytes_len(self):
FILE: src/cext/capi/types_demo.c
function PyObject (line 5) | static PyObject* list_demo(PyObject* self) {
function PyObject (line 13) | static PyObject* list_sum(PyObject* self, PyObject* args) {
function PyObject (line 28) | static PyObject* iter_list(PyObject* self, PyObject* args) {
function PyObject (line 48) | static PyObject* dict_demo(PyObject* self) {
function PyObject (line 55) | static PyObject* dict_get(PyObject* self, PyObject* args) {
function PyObject (line 70) | static PyObject* iter_dict(PyObject* self, PyObject* args) {
function PyObject (line 87) | static PyObject* tuple_demo(PyObject* self) { return Py_BuildValue("(isd...
function PyObject (line 89) | static PyObject* tuple_unpack(PyObject* self, PyObject* args) {
function PyObject (line 100) | static PyObject* set_demo(PyObject* self) {
function PyObject (line 109) | static PyObject* set_contains(PyObject* self, PyObject* args) {
function PyObject (line 120) | static PyObject* str_demo(PyObject* self) {
function PyObject (line 129) | static PyObject* str_format(PyObject* self, PyObject* args) {
function PyObject (line 139) | static PyObject* bytes_demo(PyObject* self) { return PyBytes_FromString(...
function PyObject (line 141) | static PyObject* bytes_len(PyObject* self, PyObject* args) {
type PyModuleDef (line 167) | struct PyModuleDef
function PyMODINIT_FUNC (line 169) | PyMODINIT_FUNC PyInit_types_demo(void) { return PyModule_Create(&module); }
FILE: src/cext/conftest.py
function pytest_configure (line 10) | def pytest_configure(config):
FILE: src/cext/example.cpp
function add (line 19) | int add(int a, int b) { return a + b; }
function fib (line 21) | unsigned long fib(unsigned long n) {
function fib_iter (line 27) | unsigned long fib_iter(unsigned long n) {
function PYBIND11_MODULE (line 38) | PYBIND11_MODULE(example, m) {
FILE: src/cext/fib.c
function fib (line 25) | unsigned long fib(unsigned long n) {
function fib_iter (line 30) | unsigned long fib_iter(unsigned long n) {
function add (line 41) | int add(int a, int b) { return a + b; }
function multiply (line 43) | double multiply(double a, double b) { return a * b; }
type Point (line 46) | typedef struct {
function point_distance (line 51) | double point_distance(Point* p1, Point* p2) {
function point_scale (line 57) | void point_scale(Point* p, double factor) {
FILE: src/cext/gil_example.cpp
function slow_operation (line 24) | void slow_operation(int seconds) {
function fib_nogil (line 31) | unsigned long fib_nogil(unsigned long n) {
function call_python_callback (line 44) | void call_python_callback(py::function callback, const std::string& msg) {
function PYBIND11_MODULE (line 55) | PYBIND11_MODULE(gil_example, m) {
FILE: src/cext/numpy_example.cpp
function multiply_inplace (line 26) | void multiply_inplace(py::array_t<double> arr, double factor) {
function add_arrays (line 34) | py::array_t<double> add_arrays(py::array_t<double> a, py::array_t<double...
function matrix_sum (line 52) | double matrix_sum(py::array_t<double> mat) {
function square (line 64) | py::array_t<double> square(py::array_t<double> arr) {
function PYBIND11_MODULE (line 75) | PYBIND11_MODULE(numpy_example, m) {
FILE: src/cext/test_cext.py
class TestExample (line 49) | class TestExample:
method test_add (line 52) | def test_add(self):
method test_fib (line 57) | def test_fib(self):
method test_fib_iter (line 63) | def test_fib_iter(self):
class TestVector (line 71) | class TestVector:
method test_constructor (line 74) | def test_constructor(self):
method test_length (line 83) | def test_length(self):
method test_dot (line 90) | def test_dot(self):
method test_normalized (line 95) | def test_normalized(self):
method test_add (line 100) | def test_add(self):
method test_sub (line 107) | def test_sub(self):
method test_mul (line 114) | def test_mul(self):
method test_eq (line 120) | def test_eq(self):
method test_repr (line 127) | def test_repr(self):
class TestNumPy (line 135) | class TestNumPy:
method test_multiply_inplace (line 138) | def test_multiply_inplace(self):
method test_add_arrays (line 143) | def test_add_arrays(self):
method test_add_arrays_length_mismatch (line 149) | def test_add_arrays_length_mismatch(self):
method test_matrix_sum (line 155) | def test_matrix_sum(self):
method test_square (line 159) | def test_square(self):
class TestGIL (line 166) | class TestGIL:
method test_fib_nogil (line 169) | def test_fib_nogil(self):
method test_slow_operation_parallel (line 173) | def test_slow_operation_parallel(self):
method test_callback (line 196) | def test_callback(self):
FILE: src/cext/vector.cpp
class Vector2D (line 28) | class Vector2D {
method Vector2D (line 32) | Vector2D(double x = 0, double y = 0) : x(x), y(y) {}
method length (line 34) | double length() const { return std::sqrt(x * x + y * y); }
method dot (line 36) | double dot(const Vector2D& other) const { return x * other.x + y * oth...
method Vector2D (line 38) | Vector2D normalized() const {
method Vector2D (line 44) | Vector2D operator+(const Vector2D& other) const { return Vector2D(x + ...
method Vector2D (line 46) | Vector2D operator-(const Vector2D& other) const { return Vector2D(x - ...
method Vector2D (line 48) | Vector2D operator*(double scalar) const { return Vector2D(x * scalar, ...
method repr (line 52) | std::string repr() const {
function PYBIND11_MODULE (line 59) | PYBIND11_MODULE(vector, m) {
FILE: src/cpp_from_python/cpp_from_py.cpp
function hello_world (line 29) | void hello_world() {
function variables (line 44) | void variables() {
function lists_and_vectors (line 62) | std::vector<int> lists_and_vectors() {
function array_slicing (line 83) | std::vector<int> array_slicing() {
function dictionaries_and_maps (line 111) | std::map<std::string, int> dictionaries_and_maps() {
function for_loop (line 129) | void for_loop() {
function add (line 150) | auto add(int a, int b) -> int {
function create_square_lambda (line 170) | std::function<int(int)> create_square_lambda() {
function create_multiply_lambda (line 183) | std::function<int(int)> create_multiply_lambda(int multiplier) {
function transform_with_lambda (line 196) | std::vector<int> transform_with_lambda(const std::vector<int>& numbers) {
function list_comprehension (line 213) | std::vector<int> list_comprehension() {
function string_operations (line 234) | std::string string_operations() {
class Person (line 257) | class Person {
function find_value (line 286) | std::optional<int> find_value(const std::string& key) {
function create_tuple (line 305) | std::tuple<int, int> create_tuple() {
function filter_evens (line 318) | std::vector<int> filter_evens(const std::vector<int>& numbers) {
function has_even (line 335) | bool has_even(const std::vector<int>& numbers) {
function all_positive (line 349) | bool all_positive(const std::vector<int>& numbers) {
function sort_vector (line 363) | std::vector<int> sort_vector(std::vector<int> numbers) {
function find_min (line 377) | int find_min(const std::vector<int>& numbers) {
function sum_vector (line 390) | int sum_vector(const std::vector<int>& numbers) {
function greet (line 406) | std::string greet(const std::string& name, const std::string& greeting =...
function TEST (line 410) | TEST(BasicTest, AddFunction) {
function TEST (line 416) | TEST(VectorTest, ListsAndVectors) {
function TEST (line 423) | TEST(VectorTest, ArraySlicing) {
function TEST (line 430) | TEST(MapTest, DictionariesAndMaps) {
function TEST (line 437) | TEST(LambdaTest, SquareLambda) {
function TEST (line 444) | TEST(LambdaTest, MultiplyLambda) {
function TEST (line 450) | TEST(LambdaTest, TransformWithLambda) {
function TEST (line 460) | TEST(VectorTest, ListComprehension) {
function TEST (line 467) | TEST(StringTest, StringOperations) {
function TEST (line 473) | TEST(ClassTest, PersonClass) {
function TEST (line 480) | TEST(OptionalTest, FindValue) {
function TEST (line 489) | TEST(TupleTest, CreateTuple) {
function TEST (line 495) | TEST(AlgorithmTest, FilterEvens) {
function TEST (line 503) | TEST(AlgorithmTest, HasEven) {
function TEST (line 510) | TEST(AlgorithmTest, AllPositive) {
function TEST (line 517) | TEST(AlgorithmTest, SortVector) {
function TEST (line 524) | TEST(AlgorithmTest, FindMin) {
function TEST (line 529) | TEST(AlgorithmTest, SumVector) {
function TEST (line 534) | TEST(FunctionTest, DefaultArguments) {
function main (line 539) | int main(int argc, char **argv) {
FILE: src/llm/vllm/offline_bench.py
function viztracer_profiler (line 35) | def viztracer_profiler(output_file, world_rank):
function cuda_profiler (line 64) | def cuda_profiler(enabled, world_rank):
function load_sharegpt_prompts (line 84) | def load_sharegpt_prompts(dataset_path: str, num_prompts: int) -> List[s...
function generate_random_prompts (line 97) | def generate_random_prompts(num_prompts: int, input_len: int, tokenizer)...
function generate_dummy_prompts (line 113) | def generate_dummy_prompts(num_prompts: int) -> List[str]:
function parse_args (line 126) | def parse_args():
function main (line 224) | def main():
function print_results (line 417) | def print_results(
FILE: src/megatron/entrypoint.py
function load_recipe (line 30) | def load_recipe(path, **kwargs):
function parse_cli_overrides (line 38) | def parse_cli_overrides(cfg, args):
function main (line 46) | def main() -> None:
FILE: src/megatron/recipes/deepseek_v2_lite_pretrain.py
function configure (line 6) | def configure(hf_path=None, moe_token_dispatcher_type=None):
FILE: src/megatron/viztracer_plugin.py
function _patched_handle_step (line 31) | def _patched_handle_step(config, iteration, rank, pytorch_prof):
function _patched_handle_stop (line 60) | def _patched_handle_stop(config, iteration, rank, pytorch_prof, nsys_nvt...
function install (line 77) | def install():
FILE: src/new_py3/py3.py
function dict_merge (line 15) | def dict_merge(a: dict, b: dict) -> dict:
function dict_update (line 20) | def dict_update(a: dict, b: dict) -> dict:
function positional_only (line 27) | def positional_only(a, b, /, c, d):
function walrus_example (line 33) | def walrus_example(data: list):
function walrus_fib (line 40) | def walrus_fib(count: int) -> list:
class Point (line 48) | class Point:
class FrozenPoint (line 56) | class FrozenPoint:
function fstring_basic (line 64) | def fstring_basic(name: str) -> str:
function fstring_format (line 69) | def fstring_format(value: float) -> str:
function async_greet (line 75) | async def async_greet() -> str:
function general_unpacking (line 82) | def general_unpacking() -> list:
function fib_gen (line 88) | def fib_gen(n: int):
function delegate_fib (line 96) | def delegate_fib(n: int):
function extended_unpacking (line 102) | def extended_unpacking() -> tuple:
function keyword_only (line 109) | def keyword_only(a, b, *, kw):
function nonlocal_example (line 115) | def nonlocal_example() -> str:
class TestPython312 (line 129) | class TestPython312:
method test_box_int (line 130) | def test_box_int(self):
method test_first (line 135) | def test_first(self):
method test_fstring_nested (line 140) | def test_fstring_nested(self):
class TestPython311 (line 147) | class TestPython311:
method test_exception_group (line 148) | def test_exception_group(self):
class TestPython310 (line 163) | class TestPython310:
method test_http_status (line 164) | def test_http_status(self):
method test_describe_point (line 178) | def test_describe_point(self):
class TestPython39 (line 192) | class TestPython39:
method test_dict_merge (line 193) | def test_dict_merge(self):
method test_dict_update (line 196) | def test_dict_update(self):
class TestPython38 (line 200) | class TestPython38:
method test_positional_only (line 201) | def test_positional_only(self):
method test_walrus (line 205) | def test_walrus(self):
method test_walrus_fib (line 209) | def test_walrus_fib(self):
class TestPython37 (line 213) | class TestPython37:
method test_dataclass (line 214) | def test_dataclass(self):
method test_frozen_dataclass (line 217) | def test_frozen_dataclass(self):
class TestPython36 (line 222) | class TestPython36:
method test_fstring_basic (line 223) | def test_fstring_basic(self):
method test_fstring_format (line 226) | def test_fstring_format(self):
class TestPython35 (line 230) | class TestPython35:
method test_async_greet (line 231) | def test_async_greet(self):
method test_general_unpacking (line 234) | def test_general_unpacking(self):
class TestPython33 (line 238) | class TestPython33:
method test_delegate_fib (line 239) | def test_delegate_fib(self):
class TestPython30 (line 243) | class TestPython30:
method test_extended_unpacking (line 244) | def test_extended_unpacking(self):
method test_keyword_only (line 247) | def test_keyword_only(self):
method test_nonlocal (line 250) | def test_nonlocal(self):
FILE: src/security/vulnerability_.py
class TestTimingAttack (line 9) | class TestTimingAttack:
method test_insecure_comparison (line 12) | def test_insecure_comparison(self):
method test_secure_comparison (line 30) | def test_secure_comparison(self):
class TestWeakRandom (line 39) | class TestWeakRandom:
method test_weak_random (line 42) | def test_weak_random(self):
method test_secure_random (line 55) | def test_secure_random(self):
class TestSQLInjection (line 66) | class TestSQLInjection:
method test_vulnerable_query_building (line 69) | def test_vulnerable_query_building(self):
method test_parameterized_query (line 86) | def test_parameterized_query(self):
class TestCommandInjection (line 98) | class TestCommandInjection:
method test_vulnerable_shell_command (line 101) | def test_vulnerable_shell_command(self):
method test_secure_command (line 114) | def test_secure_command(self):
class TestPickleVulnerability (line 133) | class TestPickleVulnerability:
method test_pickle_code_execution (line 136) | def test_pickle_code_execution(self):
method test_safe_json (line 154) | def test_safe_json(self):
class TestHardcodedSecrets (line 166) | class TestHardcodedSecrets:
method test_environment_variables (line 169) | def test_environment_variables(self):
class TestAESModes (line 186) | class TestAESModes:
method test_ecb_mode_pattern_leak (line 189) | def test_ecb_mode_pattern_leak(self):
method test_cbc_needs_authentication (line 212) | def test_cbc_needs_authentication(self):
class TestPasswordStorage (line 223) | class TestPasswordStorage:
method test_weak_hash (line 226) | def test_weak_hash(self):
method test_secure_password_hash (line 240) | def test_secure_password_hash(self):
Condensed preview — 178 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (1,479K chars).
[
{
"path": ".clang-format",
"chars": 147,
"preview": "BasedOnStyle: Google\nAlignAfterOpenBracket: BlockIndent\nAllowAllParametersOfDeclarationOnNextLine: false\nBinPackParamete"
},
{
"path": ".claude-plugin/marketplace.json",
"chars": 451,
"preview": "{\n \"name\": \"pysheeet\",\n \"owner\": {\n \"name\": \"crazyguitar\"\n },\n \"plugins\": [\n {\n \"name\": \"pysheeet\",\n "
},
{
"path": ".claude-plugin/plugin.json",
"chars": 361,
"preview": "{\n \"name\": \"pysheeet\",\n \"description\": \"Comprehensive Python programming reference covering syntax, concurrency, netwo"
},
{
"path": ".coveragerc",
"chars": 149,
"preview": "[report]\nomit =\n */python?.?/*\n */site-packages/*\n app_test.py\n\nexclude_lines =\n if __name__ == .__main__.:\n"
},
{
"path": ".gitattributes",
"chars": 145,
"preview": "*.png filter=lfs diff=lfs merge=lfs -text\ndocs/_static/appendix/*.png !filter !diff !merge\ndocs/_static/appendix/nixl/*."
},
{
"path": ".github/FUNDING.yml",
"chars": 20,
"preview": "github: crazyguitar\n"
},
{
"path": ".github/dependabot.yml",
"chars": 207,
"preview": "version: 2\nupdates:\n- package-ecosystem: pip\n directory: \"/\"\n schedule:\n interval: daily\n time: \"21:00\"\n open-p"
},
{
"path": ".github/workflows/pythonpackage.yml",
"chars": 546,
"preview": "name: Build\n\non: [push, pull_request]\n\njobs:\n build:\n runs-on: ubuntu-latest\n strategy:\n max-parallel: 4\n "
},
{
"path": ".gitignore",
"chars": 1456,
"preview": "_build/\n\n# Created by https://www.gitignore.io/api/vim,python\n\n### Python ###\n# Byte-compiled / optimized / DLL files\n__"
},
{
"path": "CITATION.cff",
"chars": 563,
"preview": "cff-version: 1.2.0\nmessage: \"If you use this software, please cite it as below.\"\ntitle: \"Python Cheatsheet\"\nauthors:\n- f"
},
{
"path": "LICENSE",
"chars": 1077,
"preview": "MIT License\n\nCopyright (c) 2015-2026 Chang Ning Tsai\n\nPermission is hereby granted, free of charge, to any person obtain"
},
{
"path": "Makefile",
"chars": 1675,
"preview": "REQUIREMENT = requirements.txt\n\nVER = $(word 2, $(shell python --version 2>&1))\nSRC = app.py app_test.py\nPY36 = $(shel"
},
{
"path": "Procfile",
"chars": 57,
"preview": "web: make clean && make && gunicorn app:app --log-file -\n"
},
{
"path": "README.rst",
"chars": 8475,
"preview": "\n.. raw:: html\n\n <h1 align=\"center\">\n <br>\n <a href=\"https://www.pythonsheets.com\"><img src=\"docs/_static/log"
},
{
"path": "app.py",
"chars": 3203,
"preview": "# -*- coding: utf-8 -*-\n\"\"\"This is a simple cheatsheet webapp.\"\"\"\n\nimport os\nfrom flask import Flask, abort, send_from_d"
},
{
"path": "app_test.py",
"chars": 4774,
"preview": "\"\"\"Test app.py.\"\"\"\n\nimport multiprocessing\nimport platform\nimport unittest\nimport requests\nimport os\n\nfrom pathlib impor"
},
{
"path": "docs/404.rst",
"chars": 137,
"preview": ":orphan:\n\n404 Page Not Found\n==================\n\nWhat you were looking for is just not there.\n\n`Click here to go back to"
},
{
"path": "docs/Makefile",
"chars": 7664,
"preview": "# Makefile for Sphinx documentation\n#\n\n# You can set these variables from the command line.\nSPHINXOPTS =\nSPHINXBUILD "
},
{
"path": "docs/_extra/robots.txt",
"chars": 73,
"preview": "User-agent: *\nAllow: /\nSitemap: https://www.pythonsheets.com/sitemap.xml\n"
},
{
"path": "docs/_static/.gitignore",
"chars": 130,
"preview": "# Ignore everything in this directory\n*\n# Except this file\n!.gitignore\n!guido.png\n!logo.svg\n!style.css\n!carbonad.css\n!fa"
},
{
"path": "docs/_static/carbonad.css",
"chars": 1527,
"preview": "#carbonads {\n display: block;\n overflow: hidden;\n padding: 1em;\n padding-bottom: 0.3em;\n line-height: 1.5;\n margin"
},
{
"path": "docs/_static/style.css",
"chars": 1007,
"preview": "nav#table-of-contents {\n\tdisplay: none;\n}\n\ndiv.highlight > pre {\n font-size: 14px;\n border-radius: 3px;\n backgr"
},
{
"path": "docs/_templates/carbonad.html",
"chars": 170,
"preview": "<script async type=\"text/javascript\" src=\"https://cdn.carbonads.com/carbon.js?serve={{ carbonad_serve }}&placement={{ ca"
},
{
"path": "docs/_templates/cheatsheets.html",
"chars": 106,
"preview": "<h3>Cheat Sheets</h3>\n<ul>\n <li><a href=\"https://www.cppcheatsheet.com\">C/C++ cheat sheet</a></li>\n</ul>\n"
},
{
"path": "docs/_templates/layout.html",
"chars": 2464,
"preview": "{% extends \"!layout.html\" %}\n{%- block extrahead %}\n<meta name=\"viewport\" content=\"width=device-width, initial-scale=1.0"
},
{
"path": "docs/_templates/link.html",
"chars": 379,
"preview": "<h3>Useful Links</h3>\n<ul>\n <li><a href=\"https://www.pythonsheets.com\">pysheeet website</a></li>\n <li><a href=\"https:/"
},
{
"path": "docs/_templates/sidebarintro.html",
"chars": 89,
"preview": "<p>This project tries to provide many snippets of Python code that make life easier.</p>\n"
},
{
"path": "docs/conf.py",
"chars": 12044,
"preview": "# -*- coding: utf-8 -*-\n#\n# python-cheatsheet documentation build configuration file, created by\n# sphinx-quickstart on "
},
{
"path": "docs/index.rst",
"chars": 3248,
"preview": ".. python-cheatsheet documentation master file, created by\n sphinx-quickstart on Sun Feb 28 09:26:04 2016.\n You can "
},
{
"path": "docs/notes/appendix/disaggregated-prefill-decode.rst",
"chars": 20420,
"preview": ".. meta::\n :description lang=en: Evaluating disaggregated prefill/decode for LLM serving with vLLM, NIXL, and EFA on "
},
{
"path": "docs/notes/appendix/index.rst",
"chars": 844,
"preview": ".. meta::\n :description lang=en: Python appendix covering advanced topics including the walrus operator (PEP 572) and"
},
{
"path": "docs/notes/appendix/megatron-efa-monitoring.rst",
"chars": 11907,
"preview": ".. meta::\n :description lang=en: Monitoring EFA network performance with NCCL GIN and Nsys during distributed LLM tra"
},
{
"path": "docs/notes/appendix/nccl-gin.rst",
"chars": 13591,
"preview": "\n.. meta::\n :description lang=en: Enabling GPU-Initiated Networking for NCCL with DeepEP on AWS using EFA\n :keywor"
},
{
"path": "docs/notes/appendix/python-gdb.rst",
"chars": 21422,
"preview": ".. meta::\n :description lang=en: Python interpreter in GNU Debugger (GDB)\n :keywords: Python, Python3, GDB\n\n======"
},
{
"path": "docs/notes/appendix/python-walrus.rst",
"chars": 9071,
"preview": ".. meta::\n :description lang=en: Design philosophy of pep 572, the walrus operator\n :keywords: Python3, PEP 572, w"
},
{
"path": "docs/notes/asyncio/index.rst",
"chars": 1202,
"preview": ".. meta::\n :description lang=en: Python asyncio tutorial covering coroutines, event loops, tasks, async/await syntax,"
},
{
"path": "docs/notes/asyncio/python-asyncio-advanced.rst",
"chars": 18055,
"preview": ".. meta::\n :description lang=en: Python asyncio advanced - synchronization, queues, subprocesses, debugging, patterns"
},
{
"path": "docs/notes/asyncio/python-asyncio-basic.rst",
"chars": 11560,
"preview": ".. meta::\n :description lang=en: Python asyncio basics - coroutines, tasks, event loop, async/await syntax\n :keywo"
},
{
"path": "docs/notes/asyncio/python-asyncio-guide.rst",
"chars": 17419,
"preview": ".. meta::\n :description lang=en: A comprehensive guide to understanding asynchronous programming in Python, from bloc"
},
{
"path": "docs/notes/asyncio/python-asyncio-server.rst",
"chars": 13637,
"preview": ".. meta::\n :description lang=en: Python asyncio networking - TCP/UDP servers, HTTP clients, SSL/TLS, protocols\n :k"
},
{
"path": "docs/notes/basic/index.rst",
"chars": 1178,
"preview": ".. meta::\n :description lang=en: Python basics cheat sheet covering syntax, data types, functions, classes, generator"
},
{
"path": "docs/notes/basic/python-basic.rst",
"chars": 15914,
"preview": ".. meta::\n :description lang=en: Python basics tutorial covering fundamental syntax, data types, control flow, string"
},
{
"path": "docs/notes/basic/python-dict.rst",
"chars": 7810,
"preview": ".. meta::\n :description lang=en: Python dictionary cheat sheet covering creation, manipulation, merging, comprehensio"
},
{
"path": "docs/notes/basic/python-func.rst",
"chars": 20054,
"preview": ".. meta::\n :description lang=en: Python function cheat sheet covering function definitions, arguments, decorators, la"
},
{
"path": "docs/notes/basic/python-future.rst",
"chars": 6066,
"preview": ".. meta::\n :description lang=en: Python __future__ module guide covering future statements, backward compatibility, a"
},
{
"path": "docs/notes/basic/python-generator.rst",
"chars": 26377,
"preview": ".. meta::\n :description lang=en: Python generator cheat sheet covering generator functions, generator expressions, yi"
},
{
"path": "docs/notes/basic/python-heap.rst",
"chars": 8064,
"preview": ".. meta::\n :description lang=en: Python heap and priority queue cheat sheet covering heapq module operations, heap so"
},
{
"path": "docs/notes/basic/python-list.rst",
"chars": 23815,
"preview": ".. meta::\n :description lang=en: Python list cheat sheet covering list operations, comprehensions, slicing, sorting, "
},
{
"path": "docs/notes/basic/python-object.rst",
"chars": 13362,
"preview": ".. meta::\n :description lang=en: Python class cheat sheet covering magic methods, property decorators, inheritance, c"
},
{
"path": "docs/notes/basic/python-rexp.rst",
"chars": 14496,
"preview": ".. meta::\n :description lang=en: Python regex cheat sheet covering re module, pattern matching, groups, lookahead, lo"
},
{
"path": "docs/notes/basic/python-set.rst",
"chars": 7919,
"preview": ".. meta::\n :description lang=en: Python set cheat sheet covering set comprehensions, set operations (union, intersect"
},
{
"path": "docs/notes/basic/python-typing.rst",
"chars": 26266,
"preview": ".. meta::\n :description lang=en: Python typing cheat sheet covering type hints, annotations, generics, protocols, Typ"
},
{
"path": "docs/notes/basic/python-unicode.rst",
"chars": 7510,
"preview": ".. meta::\n :description lang=en: Python Unicode tutorial covering string encoding, decoding, UTF-8, ASCII, bytes conv"
},
{
"path": "docs/notes/concurrency/index.rst",
"chars": 1193,
"preview": ".. meta::\n :description lang=en: Python concurrency tutorial covering threading, multiprocessing, locks, semaphores, "
},
{
"path": "docs/notes/concurrency/python-futures.rst",
"chars": 13806,
"preview": ".. meta::\n :description lang=en: Python concurrent.futures tutorial covering ThreadPoolExecutor, ProcessPoolExecutor,"
},
{
"path": "docs/notes/concurrency/python-multiprocessing.rst",
"chars": 12642,
"preview": ".. meta::\n :description lang=en: Python multiprocessing tutorial covering process creation, pools, shared memory, int"
},
{
"path": "docs/notes/concurrency/python-threading.rst",
"chars": 14585,
"preview": ".. meta::\n :description lang=en: Python threading tutorial covering thread creation, synchronization primitives, lock"
},
{
"path": "docs/notes/database/index.rst",
"chars": 992,
"preview": ".. meta::\n :description lang=en: Python SQLAlchemy tutorial covering database connections, ORM models, relationships,"
},
{
"path": "docs/notes/database/python-sqlalchemy-orm.rst",
"chars": 22995,
"preview": ".. meta::\n :description lang=en: SQLAlchemy ORM tutorial covering declarative models, sessions, relationships, and ob"
},
{
"path": "docs/notes/database/python-sqlalchemy-query.rst",
"chars": 21054,
"preview": ".. meta::\n :description lang=en: SQLAlchemy advanced query patterns including joins, subqueries, aggregations, window"
},
{
"path": "docs/notes/database/python-sqlalchemy.rst",
"chars": 19250,
"preview": ".. meta::\n :description lang=en: SQLAlchemy Core tutorial covering database connections, engine creation, metadata, t"
},
{
"path": "docs/notes/extension/cpp-from-python.rst",
"chars": 19708,
"preview": ".. meta::\n :description lang=en: Learn modern C++ syntax from Python - side-by-side comparison of Python and C++ code"
},
{
"path": "docs/notes/extension/index.rst",
"chars": 1231,
"preview": ".. meta::\n :description lang=en: Python C/C++ extension tutorial covering pybind11, ctypes, cffi, Cython, and the Pyt"
},
{
"path": "docs/notes/extension/python-capi.rst",
"chars": 30219,
"preview": ".. meta::\n :description lang=en: Comprehensive Python C API tutorial covering native C extension development, module "
},
{
"path": "docs/notes/extension/python-cext-modern.rst",
"chars": 21391,
"preview": ".. meta::\n :description lang=en: Comprehensive guide to modern Python C/C++ extensions covering pybind11, ctypes, cff"
},
{
"path": "docs/notes/extension/python-ctypes.rst",
"chars": 11926,
"preview": ".. meta::\n :description lang=en: Python ctypes tutorial for loading shared libraries, calling C functions, handling p"
},
{
"path": "docs/notes/hpc/index.rst",
"chars": 829,
"preview": ".. meta::\n :description lang=en: High-Performance Computing (HPC) cheat sheet covering Slurm job scheduling, cluster "
},
{
"path": "docs/notes/hpc/slurm.rst",
"chars": 24248,
"preview": ".. meta::\n :description lang=en: Slurm cheat sheet for HPC job scheduling, batch jobs, distributed training, MPI, Enr"
},
{
"path": "docs/notes/llm/index.rst",
"chars": 712,
"preview": ".. meta::\n :description lang=en: Large Language Models (LLM) cheat sheet — PyTorch, distributed training, vLLM/SGLang"
},
{
"path": "docs/notes/llm/llm-bench.rst",
"chars": 15819,
"preview": ".. meta::\n :description lang=en: LLM benchmark suite — measure throughput, TTFT, ITL, latency for vLLM, SGLang, and T"
},
{
"path": "docs/notes/llm/llm-serving.rst",
"chars": 9615,
"preview": ".. meta::\n :description lang=en: LLM serving guide — vLLM, SGLang, and TensorRT-LLM for single-node, multi-node SLURM"
},
{
"path": "docs/notes/llm/megatron.rst",
"chars": 7898,
"preview": ".. meta::\n :description lang=en: Megatron Bridge cheat sheet — pretrain recipes, Nsys profiling, and distributed trai"
},
{
"path": "docs/notes/llm/pytorch.rst",
"chars": 22052,
"preview": ".. meta::\n :description lang=en: PyTorch cheat sheet covering tensors, operations, gradients, GPU usage, neural netwo"
},
{
"path": "docs/notes/network/index.rst",
"chars": 1171,
"preview": ".. meta::\n :description lang=en: Python network programming tutorial covering sockets, TCP/UDP servers, async I/O, SS"
},
{
"path": "docs/notes/network/python-socket-async.rst",
"chars": 13028,
"preview": ".. meta::\n :description lang=en: Python asynchronous socket programming tutorial with select, poll, epoll, kqueue, an"
},
{
"path": "docs/notes/network/python-socket-server.rst",
"chars": 11762,
"preview": ".. meta::\n :description lang=en: Python TCP and UDP server tutorial with examples for echo servers, IPv6 dual-stack, "
},
{
"path": "docs/notes/network/python-socket-sniffer.rst",
"chars": 12954,
"preview": ".. meta::\n :description lang=en: Python packet sniffing and raw socket programming tutorial for capturing and parsing"
},
{
"path": "docs/notes/network/python-socket-ssl.rst",
"chars": 10927,
"preview": ".. meta::\n :description lang=en: Python TLS/SSL socket programming tutorial covering secure servers, certificate hand"
},
{
"path": "docs/notes/network/python-socket.rst",
"chars": 23301,
"preview": ".. meta::\n :description lang=en: Python socket programming tutorial with examples for DNS resolution, TCP/UDP clients"
},
{
"path": "docs/notes/network/python-ssh.rst",
"chars": 27302,
"preview": ".. meta::\n :description lang=en: Comprehensive SSH cheat sheet for Python developers covering Paramiko library, SSH t"
},
{
"path": "docs/notes/os/index.rst",
"chars": 1021,
"preview": ".. meta::\n :description lang=en: Python system programming tutorial covering file operations, datetime, process manag"
},
{
"path": "docs/notes/os/python-date.rst",
"chars": 15503,
"preview": ".. meta::\n :description lang=en: Python datetime tutorial covering timestamps, date formatting, parsing, timezones, t"
},
{
"path": "docs/notes/os/python-io.rst",
"chars": 26390,
"preview": ".. meta::\n :description lang=en: Python file I/O tutorial covering reading, writing, binary files, pathlib, context m"
},
{
"path": "docs/notes/os/python-os.rst",
"chars": 12536,
"preview": ".. meta::\n :description lang=en: Python operating system interface tutorial covering file operations, process managem"
},
{
"path": "docs/notes/python-new-py3.rst",
"chars": 29027,
"preview": ".. meta::\n :description lang=en: Python 3 new features guide covering f-strings, walrus operator, dataclasses, async/"
},
{
"path": "docs/notes/security/index.rst",
"chars": 1076,
"preview": ".. meta::\n :description lang=en: Python security and cryptography guide covering modern encryption, TLS/SSL, common v"
},
{
"path": "docs/notes/security/python-crypto.rst",
"chars": 34556,
"preview": ".. meta::\n :description lang=en: Modern Python cryptography guide covering symmetric encryption (AES-GCM), asymmetric"
},
{
"path": "docs/notes/security/python-tls.rst",
"chars": 16977,
"preview": ".. meta::\n :description lang=en: Python TLS/SSL and X.509 certificate guide covering secure HTTPS servers, certificat"
},
{
"path": "docs/notes/security/python-vulnerability.rst",
"chars": 13949,
"preview": ".. meta::\n :description lang=en: Common Python security vulnerabilities and why legacy cryptographic patterns are ins"
},
{
"path": "requirements.txt",
"chars": 744,
"preview": "coverage==7.13.4\ncryptography==46.0.5\nargon2-cffi==25.1.0\ncffi==2.0.0\nSQLAlchemy==2.0.48\nbandit==1.9.4\ncoveralls==4.1.0\n"
},
{
"path": "runtime.txt",
"chars": 12,
"preview": "python-3.12\n"
},
{
"path": "skills/py/SKILL.md",
"chars": 2955,
"preview": "---\nname: py\ndescription: Comprehensive Python programming reference covering syntax, concurrency, networking, databases"
},
{
"path": "skills/py/references/guidelines.md",
"chars": 2977,
"preview": "# Python Development Guidelines\n\nAlways fetch relevant examples from pythonsheets.com first to ensure correctness, then "
},
{
"path": "skills/py/references/structure.md",
"chars": 4450,
"preview": "# Python Topics Reference Map\n\nComplete reference guide organized by topic, with direct links to live documentation.\n\n##"
},
{
"path": "src/basic/asyncio_.py",
"chars": 10582,
"preview": "\"\"\"Tests for asyncio examples.\"\"\"\n\nimport asyncio\nimport pytest\n\n\nclass TestAsyncioBasics:\n \"\"\"Test basic asyncio ope"
},
{
"path": "src/basic/basic.py",
"chars": 5075,
"preview": "\"\"\"Python Basics Examples\n\nSource code for docs/notes/basic/python-basic.rst\n\"\"\"\n\nimport sys\nimport platform\nimport pyte"
},
{
"path": "src/basic/cext_.py",
"chars": 9200,
"preview": "\"\"\"\nTests for C extension examples (ctypes and cffi).\n\nThese tests demonstrate calling C code from Python without\nrequir"
},
{
"path": "src/basic/concurrency_.py",
"chars": 11990,
"preview": "\"\"\"Tests for concurrency examples.\"\"\"\n\nimport pytest\nimport time\nfrom threading import Thread, Lock, RLock, Semaphore, E"
},
{
"path": "src/basic/crypto_.py",
"chars": 11548,
"preview": "\"\"\"\nTests for modern cryptography examples.\n\"\"\"\n\nimport hashlib\nimport hmac\nimport os\nimport secrets\n\nimport pytest\nfrom"
},
{
"path": "src/basic/datetime_.py",
"chars": 6976,
"preview": "\"\"\"Tests for datetime operations.\"\"\"\n\nimport calendar\nimport time\nfrom datetime import date, datetime, time as dt_time, "
},
{
"path": "src/basic/dict.py",
"chars": 5505,
"preview": "\"\"\"Python Dictionary Examples\n\nSource code for docs/notes/basic/python-dict.rst\n\"\"\"\n\nimport pytest\nfrom collections impo"
},
{
"path": "src/basic/fileio_.py",
"chars": 11374,
"preview": "\"\"\"Tests for file I/O operations.\"\"\"\n\nimport csv\nimport gzip\nimport json\nimport tempfile\nimport zipfile\nfrom pathlib imp"
},
{
"path": "src/basic/func.py",
"chars": 7719,
"preview": "\"\"\"Python Function Examples\n\nSource code for docs/notes/basic/python-func.rst\n\"\"\"\n\nfrom functools import lru_cache, part"
},
{
"path": "src/basic/future_.py",
"chars": 3181,
"preview": "\"\"\"Python Future Examples\n\nSource code for docs/notes/basic/python-future.rst\n\"\"\"\n\nfrom __future__ import annotations\nim"
},
{
"path": "src/basic/generator.py",
"chars": 12028,
"preview": "\"\"\"Python Generator Examples\n\nSource code for docs/notes/basic/python-generator.rst\n\"\"\"\n\nimport inspect\nfrom contextlib "
},
{
"path": "src/basic/heap.py",
"chars": 6462,
"preview": "\"\"\"Python Heap Examples\n\nSource code for docs/notes/basic/python-heap.rst\n\"\"\"\n\nimport heapq\n\nimport pytest\n\n\n# Basic Hea"
},
{
"path": "src/basic/list.py",
"chars": 7570,
"preview": "\"\"\"Python List Examples\n\nSource code for docs/notes/basic/python-list.rst\n\"\"\"\n\nimport bisect\nimport copy\nimport itertool"
},
{
"path": "src/basic/object.py",
"chars": 8006,
"preview": "\"\"\"Python OOP Examples\n\nSource code for docs/notes/basic/python-object.rst\n\"\"\"\n\nfrom abc import ABC, abstractmethod\nfrom"
},
{
"path": "src/basic/os_.py",
"chars": 11346,
"preview": "\"\"\"\nTests for operating system operations.\n\nThese tests demonstrate Python's os module for file system operations,\nproce"
},
{
"path": "src/basic/rexp.py",
"chars": 10142,
"preview": "\"\"\"Python Regular Expression Examples\n\nSource code for docs/notes/basic/python-rexp.rst\n\"\"\"\n\nimport re\nfrom collections "
},
{
"path": "src/basic/set.py",
"chars": 6212,
"preview": "\"\"\"Python Set Examples\n\nSource code for docs/notes/basic/python-set.rst\n\"\"\"\n\nimport pytest\n\n\n# Create a Set\ndef create_s"
},
{
"path": "src/basic/socket_.py",
"chars": 8371,
"preview": "\"\"\"Network/socket examples and tests for pysheeet documentation.\"\"\"\n\nimport socket\nimport threading\nimport time\nimport p"
},
{
"path": "src/basic/sqlalchemy_core.py",
"chars": 14113,
"preview": "\"\"\"SQLAlchemy examples and tests for pysheeet documentation.\"\"\"\n\nfrom datetime import datetime\nimport pytest\nfrom sqlalc"
},
{
"path": "src/basic/sqlalchemy_orm.py",
"chars": 16419,
"preview": "\"\"\"SQLAlchemy ORM examples and tests for pysheeet documentation.\"\"\"\n\nimport pytest\nfrom sqlalchemy import (\n create_e"
},
{
"path": "src/basic/sqlalchemy_query.py",
"chars": 18903,
"preview": "\"\"\"SQLAlchemy query recipe examples and tests for pysheeet documentation.\"\"\"\n\nimport pytest\nfrom sqlalchemy import (\n "
},
{
"path": "src/basic/typing_.py",
"chars": 4415,
"preview": "\"\"\"Python Typing Examples\n\nSource code for docs/notes/basic/python-typing.rst\n\"\"\"\n\nimport pytest\nfrom typing import (\n "
},
{
"path": "src/basic/unicode_.py",
"chars": 4116,
"preview": "\"\"\"Python Unicode Examples\n\nSource code for docs/notes/basic/python-unicode.rst\n\"\"\"\n\nimport pytest\nimport unicodedata\n\n\n"
},
{
"path": "src/cext/CMakeLists.txt",
"chars": 1386,
"preview": "cmake_minimum_required(VERSION 3.15)\nproject(pysheeet_cext)\n\nset(CMAKE_CXX_STANDARD 17)\nset(CMAKE_CXX_STANDARD_REQUIRED "
},
{
"path": "src/cext/README.md",
"chars": 1486,
"preview": "# pybind11 C++ Extension Examples\n\nThis directory contains C++ source files demonstrating pybind11 bindings.\n\n## Prerequ"
},
{
"path": "src/cext/capi/args.c",
"chars": 1830,
"preview": "/* Demonstrate argument parsing in Python C API. */\n#include <Python.h>\n\n/* METH_NOARGS - no arguments */\nstatic PyObjec"
},
{
"path": "src/cext/capi/errors.c",
"chars": 1727,
"preview": "/* Demonstrate exception handling in Python C API. */\n#include <Python.h>\n\nstatic PyObject* FooError;\n\n/* Raise built-in"
},
{
"path": "src/cext/capi/gil.c",
"chars": 1685,
"preview": "/* Demonstrate GIL release and acquire in Python C API. */\n#include <Python.h>\n\n#ifdef _WIN32\n#include <windows.h>\n#defi"
},
{
"path": "src/cext/capi/setup.py",
"chars": 334,
"preview": "from setuptools import setup, Extension\n\nextensions = [\n Extension(\"simple\", [\"simple.c\"]),\n Extension(\"args\", [\"a"
},
{
"path": "src/cext/capi/simple.c",
"chars": 1320,
"preview": "/* Simple C extension module demonstrating Python C API basics. */\n#include <Python.h>\n\nPyDoc_STRVAR(doc_mod, \"Simple ex"
},
{
"path": "src/cext/capi/test_capi.py",
"chars": 4576,
"preview": "\"\"\"Tests for Python C API extension examples.\"\"\"\n\nimport pytest\nimport sys\nimport os\n\n# Add build directory to path\nbuil"
},
{
"path": "src/cext/capi/types_demo.c",
"chars": 5360,
"preview": "/* Demonstrate Python types manipulation in C API. */\n#include <Python.h>\n\n/* List operations */\nstatic PyObject* list_d"
},
{
"path": "src/cext/conftest.py",
"chars": 377,
"preview": "\"\"\"\npytest configuration for C extension tests.\nAdds build directory to sys.path before tests run.\n\"\"\"\n\nimport sys\nfrom "
},
{
"path": "src/cext/example.cpp",
"chars": 983,
"preview": "/**\n * example.cpp - Basic pybind11 example\n *\n * Build:\n * mkdir build && cd build\n * cmake .. && make\n *\n * Usage:"
},
{
"path": "src/cext/fib.c",
"chars": 1319,
"preview": "/**\n * fib.c - Pure C library for ctypes/cffi examples\n *\n * Compile:\n * gcc -shared -fPIC -o libfib.so fib.c # L"
},
{
"path": "src/cext/gil_example.cpp",
"chars": 1888,
"preview": "/**\n * gil_example.cpp - GIL release example\n *\n * Demonstrates:\n * - Releasing GIL for CPU-intensive work\n * - Allo"
},
{
"path": "src/cext/numpy_example.cpp",
"chars": 2313,
"preview": "/**\n * numpy_example.cpp - pybind11 NumPy integration\n *\n * Demonstrates:\n * - Accepting NumPy arrays\n * - Modifying"
},
{
"path": "src/cext/setup.py",
"chars": 1091,
"preview": "\"\"\"\nsetup.py for pybind11 examples\n\nBuild:\n pip install .\n # or\n python setup.py build_ext --inplace\n\"\"\"\n\nfrom "
},
{
"path": "src/cext/test_cext.py",
"chars": 5228,
"preview": "\"\"\"\nTests for pybind11 C++ extension modules.\n\nRun from src/cext directory:\n python -m pytest test_cext.py -v\n\nBuild "
},
{
"path": "src/cext/vector.cpp",
"chars": 2088,
"preview": "/**\n * vector.cpp - pybind11 class binding example\n *\n * Demonstrates:\n * - Class binding with constructor\n * - Read"
},
{
"path": "src/cpp_from_python/CMakeLists.txt",
"chars": 614,
"preview": "cmake_minimum_required(VERSION 3.15)\nproject(cpp_from_python)\n\nset(CMAKE_CXX_STANDARD 17)\nset(CMAKE_CXX_STANDARD_REQUIRE"
},
{
"path": "src/cpp_from_python/cpp_from_py.cpp",
"chars": 11681,
"preview": "/*\n * Learn C++ from Python - Modern C++ Examples with Tests\n *\n * Demonstrates modern C++ syntax with Python equivalent"
},
{
"path": "src/gin/Dockerfile",
"chars": 7686,
"preview": "# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.\n#\n# Modifications copyright (c) 2025 chang-ning\n# M"
},
{
"path": "src/gin/Makefile",
"chars": 770,
"preview": ".PHONY: help docker save clean\n\n.DEFAULT_GOAL := help\n\nIMAGE_NAME ?= nccl\nIMAGE_TAG ?= latest\n\nhelp:\n\t@echo \"NCCL GIN Te"
},
{
"path": "src/gin/run.enroot",
"chars": 1381,
"preview": "#!/bin/bash\n# Launch command inside enroot container via srun + pyxis\n# Usage: salloc -N 2 ./run.enroot <command...>\n\nse"
},
{
"path": "src/gin/run.sbatch",
"chars": 12535,
"preview": "#!/bin/bash\n\nset -euo pipefail\n\nGPUS=\"${GPUS:-all}\"\nDP_BACKEND=\"${DP_BACKEND:-rpc}\"\nDEEPEP_ENV=\"export DEEP_EP_BACKEND=n"
},
{
"path": "src/llm/sglang/Dockerfile",
"chars": 4848,
"preview": "ARG SGLANG_VERSION=0.5.8\nARG CUDA_VERSION=12.8.1\nARG GDRCOPY_VERSION=v2.5.1\nARG EFA_INSTALLER_VERSION=1.46.0\nARG NCCL_VE"
},
{
"path": "src/llm/sglang/Makefile",
"chars": 2345,
"preview": ".PHONY: help docker sqush save load serve test bench clean\n\n.DEFAULT_GOAL := help\n\nhelp:\n\t@echo \"SGLang Serving Makefile"
},
{
"path": "src/llm/sglang/README.rst",
"chars": 7402,
"preview": "=============\nSGLang Serving\n=============\n\n.. contents:: Table of Contents\n :backlinks: none\n\nThis cheat sheet provi"
},
{
"path": "src/llm/sglang/bench.sh",
"chars": 7027,
"preview": "#!/usr/bin/env bash\n# SGLang serving benchmark suite\n# Usage:\n# salloc -N1 bash bench.sh -H 10.0.128.193 -i /fsx/sglan"
},
{
"path": "src/llm/sglang/run.sbatch",
"chars": 6294,
"preview": "#!/bin/bash\n\nset -euo pipefail\n\nGPUS=\"${GPUS:-all}\"\n\ninfo() { echo -e \"[$(date +'%Y-%m-%dT%H:%M:%S%z')][info] $*\"; }\nerr"
},
{
"path": "src/llm/sglang/test.sh",
"chars": 3421,
"preview": "#!/usr/bin/env bash\n# SGLang API test script\n\nset -uo pipefail\n\nHOST=\"localhost\"\nPORT=\"30000\"\nMODEL=\"\"\n\nwhile (( \"$#\" ))"
},
{
"path": "src/llm/tensorrt-llm/Dockerfile",
"chars": 4996,
"preview": "ARG TRTLLM_VERSION=1.1.0\nARG CUDA_VERSION=13.0.0\nARG GDRCOPY_VERSION=v2.5.1\nARG EFA_INSTALLER_VERSION=1.46.0\nARG NCCL_VE"
},
{
"path": "src/llm/tensorrt-llm/Makefile",
"chars": 2328,
"preview": ".PHONY: help docker sqush save load serve test bench clean\n\n.DEFAULT_GOAL := help\n\nhelp:\n\t@echo \"TensorRT-LLM Serving Ma"
},
{
"path": "src/llm/tensorrt-llm/README.rst",
"chars": 9060,
"preview": "====================\nTensorRT-LLM Serving\n====================\n\n.. contents:: Table of Contents\n :backlinks: none\n\nTh"
},
{
"path": "src/llm/tensorrt-llm/bench.sh",
"chars": 7889,
"preview": "#!/usr/bin/env bash\n# TensorRT-LLM serving benchmark suite\n# Uses the official benchmark_serving.py for proper TTFT/TPOT"
},
{
"path": "src/llm/tensorrt-llm/run.sbatch",
"chars": 5372,
"preview": "#!/bin/bash\n\nset -euo pipefail\n\nGPUS=\"${GPUS:-all}\"\n\ninfo() { echo -e \"[$(date +'%Y-%m-%dT%H:%M:%S%z')][info] $*\"; }\nerr"
},
{
"path": "src/llm/tensorrt-llm/test.sh",
"chars": 2945,
"preview": "#!/usr/bin/env bash\n# TensorRT-LLM API test script\n\nset -uo pipefail\n\nHOST=\"localhost\"\nPORT=\"8000\"\nMODEL=\"\"\n\nwhile (( \"$"
},
{
"path": "src/llm/vllm/Dockerfile",
"chars": 6427,
"preview": "ARG VLLM_VERSION=0.15.1\nARG CUDA_VERSION=12.8.1\nARG GDRCOPY_VERSION=v2.5.1\nARG EFA_INSTALLER_VERSION=1.46.0\nARG NCCL_VER"
},
{
"path": "src/llm/vllm/Makefile",
"chars": 859,
"preview": ".PHONY: help docker save clean format\n\n.DEFAULT_GOAL := help\n\nIMAGE_NAME ?= vllm-serve\nIMAGE_TAG ?= latest\n\nhelp:\n\t@echo"
},
{
"path": "src/llm/vllm/README.rst",
"chars": 14737,
"preview": "============\nvLLM Serving\n============\n\n.. contents:: Table of Contents\n :backlinks: none\n\nThis cheat sheet provides "
},
{
"path": "src/llm/vllm/bench.sh",
"chars": 12535,
"preview": "#!/usr/bin/env bash\n# vLLM serving benchmark suite\n# Usage:\n# salloc -N1 bash bench.sh -H 10.0.128.193 -i /fsx/vllm-se"
},
{
"path": "src/llm/vllm/offline_bench.py",
"chars": 14621,
"preview": "#!/usr/bin/env python3\n\"\"\"\nOffline vLLM benchmark without API server overhead.\nBased on vllm torchrun_dp_example.py for "
},
{
"path": "src/llm/vllm/offline_bench.sh",
"chars": 7556,
"preview": "#!/usr/bin/env bash\n# Offline vLLM benchmark wrapper (no API server)\n# Usage:\n# salloc -N1 bash offline_bench.sh --mod"
},
{
"path": "src/llm/vllm/run.sbatch",
"chars": 12385,
"preview": "#!/bin/bash\n\nset -euo pipefail\n\nGPUS=\"${GPUS:-all}\"\nDP_BACKEND=\"${DP_BACKEND:-rpc}\"\n\ninfo() { echo -e \"[$(date +'%Y-%m-%"
},
{
"path": "src/llm/vllm/run.sh",
"chars": 4506,
"preview": "#!/bin/bash\n# vLLM server launcher with multiple modes\n\nset -uo pipefail\n\nPROGRAM=\"$0\"\nMODE=\"single\"\nMODEL=\"Qwen/Qwen2.5"
},
{
"path": "src/llm/vllm/sweep.sbatch",
"chars": 4454,
"preview": "#!/usr/bin/env bash\n# sweep.sbatch — Self-contained sweep benchmark with server lifecycle management.\n#\n# Wraps `vllm be"
},
{
"path": "src/llm/vllm/sweep.sh",
"chars": 4162,
"preview": "#!/usr/bin/env bash\n# vLLM sweep benchmark suite\n#\n# Runs predefined sweep configurations via sweep.sbatch. Each suite w"
},
{
"path": "src/llm/vllm/test.sh",
"chars": 3609,
"preview": "#!/usr/bin/env bash\n# vLLM API test script\n\nset -uo pipefail\n\nHOST=\"localhost\"\nPORT=\"8000\"\nMODEL=\"\"\n\nwhile (( \"$#\" )); d"
},
{
"path": "src/megatron/Dockerfile",
"chars": 9925,
"preview": "# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.\n#\n# Modifications copyright (c) 2025 chang-ning\n# M"
},
{
"path": "src/megatron/Makefile",
"chars": 238,
"preview": "IMAGE_NAME ?= megatron-lm\n\n.PHONY: build sqsh clean fmt\n\nbuild:\n\t./enroot.sh -n \"$(IMAGE_NAME)\" -f Dockerfile\n\nsqsh: bui"
},
{
"path": "src/megatron/README.md",
"chars": 1710,
"preview": "# Megatron\n\n```bash\nmake build\n\n# Launch a 2-node DeepSeek V2 Lite pretrain job:\nsalloc -N 2\n./srun.sh recipes/deepseek_"
},
{
"path": "src/megatron/enroot.sh",
"chars": 2308,
"preview": "#!/bin/bash\n\nFILE=\"${PWD}/Dockerfile\"\nIMAGE=\"\"\nINPUT=\"${PWD}\"\nPROGRAM=\"$0\"\nOUTPUT=\"${PWD}\"\n\nerr() {\n echo -e \"[$(date +"
},
{
"path": "src/megatron/entrypoint.py",
"chars": 2014,
"preview": "#!/usr/bin/env python3\n\"\"\"Generic entrypoint for Megatron Bridge recipes.\n\nUsage:\n ./srun.sh recipes/deepseek_v2_lite"
},
{
"path": "src/megatron/recipes/deepseek_v2_lite_pretrain.py",
"chars": 908,
"preview": "from megatron.bridge.recipes.deepseek.deepseek_v2 import (\n deepseek_v2_lite_pretrain_config,\n)\n\n\ndef configure(hf_pa"
},
{
"path": "src/megatron/srun.sh",
"chars": 2771,
"preview": "#!/bin/bash\n# Launch Megatron Bridge recipe inside enroot container via srun + pyxis\n# Usage: salloc -N 2 ./srun.sh reci"
},
{
"path": "src/megatron/viztracer_plugin.py",
"chars": 3305,
"preview": "\"\"\"VizTracer profiling plugin for Megatron Bridge.\n\nMonkey-patches megatron.bridge.training.profiling to add viztracer s"
},
{
"path": "src/new_py3/py3.py",
"chars": 6168,
"preview": "\"\"\"New features in Python 3 (3.12 → 3.0)\n\nSource code examples for docs/notes/python-new-py3.rst\n\"\"\"\n\nimport sys\nimport "
},
{
"path": "src/nixl/Dockerfile",
"chars": 7177,
"preview": "ARG CUDA_VERSION=12.8.1\nARG GDRCOPY_VERSION=v2.5.1\nARG EFA_INSTALLER_VERSION=1.47.0\nARG UCX_VERSION=v1.20.0\nARG NIXL_VER"
},
{
"path": "src/nixl/Makefile",
"chars": 618,
"preview": ".PHONY: help docker save clean\n\n.DEFAULT_GOAL := help\n\nIMAGE_NAME ?= nixl\nIMAGE_TAG ?= latest\n\nhelp:\n\t@echo \"NIXL Makefi"
},
{
"path": "src/nixl/bench.sh",
"chars": 3789,
"preview": "#!/usr/bin/env bash\n# vLLM serving benchmark suite\n# Usage:\n# salloc -N1 bash bench.sh -H 10.0.128.193 -i /fsx/nixl-la"
},
{
"path": "src/nixl/nixl.sbatch",
"chars": 4748,
"preview": "#!/bin/bash\n# Usage:\n# salloc -N 2 bash nixl.sbatch [--image <path>] [nixlbench args...]\n#\n# Examples:\n# salloc -N 2"
},
{
"path": "src/nixl/vllm.sbatch",
"chars": 13692,
"preview": "#!/bin/bash\n# Usage:\n# salloc -N <nodes> bash vllm.sbatch [--route R] [--prefill P] [--image <path>] [vllm serve args."
},
{
"path": "src/security/vulnerability_.py",
"chars": 8498,
"preview": "\"\"\"Tests demonstrating security vulnerabilities and secure alternatives.\"\"\"\n\nimport pytest\nimport hmac\nimport secrets\nim"
}
]
About this extraction
This page contains the full source code of the crazyguitar/pysheeet GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 178 files (1.3 MB), approximately 367.5k tokens, and a symbol index with 1343 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.