Showing preview only (266K chars total). Download the full file or copy to clipboard to get everything.
Repository: trinhminhtriet/spiko
Branch: master
Commit: d15162659b9d
Files: 36
Total size: 254.2 KB
Directory structure:
gitextract_ygc3eeo3/
├── .dockerignore
├── .editorconfig
├── .github/
│ ├── ISSUE_TEMPLATE/
│ │ └── bug_report.md
│ ├── dependabot.yml
│ └── workflows/
│ ├── docker.yml
│ ├── publish.yml
│ └── release.yml
├── .gitignore
├── .vscode/
│ └── settings.json
├── CHANGELOG.md
├── Cargo.toml
├── Cross.toml
├── Dockerfile
├── LICENSE
├── Makefile
├── README.md
├── demo.tape
├── pgo/
│ └── server/
│ ├── Cargo.toml
│ └── src/
│ └── main.rs
├── pgo.js
├── schema.json
├── scripts/
│ └── release-version.sh
├── src/
│ ├── aws_auth.rs
│ ├── client.rs
│ ├── db.rs
│ ├── histogram.rs
│ ├── lib.rs
│ ├── main.rs
│ ├── monitor.rs
│ ├── pcg64si.rs
│ ├── printer.rs
│ ├── result_data.rs
│ ├── timescale.rs
│ ├── tls_config.rs
│ └── url_generator.rs
└── tests/
└── tests.rs
================================================
FILE CONTENTS
================================================
================================================
FILE: .dockerignore
================================================
.git/
.github/
.vscode/
docs/
scripts/
target/
================================================
FILE: .editorconfig
================================================
root = true
[*]
charset = utf-8
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true
indent_style = space
indent_size = 2
max_line_length = off
[*.rs]
indent_size = 4
max_line_length = 500
[*.yml]
indent_size = 2
indent_style = space
[*.toml]
indent_size = 2
indent_style = space
[*.sh]
indent_size = 2
indent_style = space
[*.md]
trim_trailing_whitespace = false # Allow trailing spaces in markdown
max_line_length = off # Disable line length limits for prose
================================================
FILE: .github/ISSUE_TEMPLATE/bug_report.md
================================================
---
name: "Bug report"
about: Create a bug report for spiko.
title: ""
labels: bug
assignees: "trinhminhtriet"
---
<!---
1. Verify first that your issue/request is not already reported on GitHub.
2. PLEASE FILL OUT ALL REQUIRED INFORMATION BELOW! Otherwise it might take more time to properly handle this bug report.
-->
### ISSUE TYPE:
- Bug Report
#### OS / ENVIRONMENT:
1. [ ] Operating system:
2. [ ] spiko version:
#### STEPS TO REPRODUCE:
1.
2.
3.
#### EXPECTED BEHAVIOUR:
explanation
#### ACTUAL BEHAVIOUR:
explanation
#### Additional information (optional):
================================================
FILE: .github/dependabot.yml
================================================
version: 2
updates:
- package-ecosystem: "cargo"
directory: "/"
schedule:
interval: "monthly"
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "daily"
================================================
FILE: .github/workflows/docker.yml
================================================
name: Create and publish a Docker image
on:
push:
tags:
- "v*"
pull_request:
types: [closed]
env:
CARGO_TERM_COLOR: always
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
concurrency: build-docker-image
jobs:
build-and-push-image:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v5
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to the Container registry
if: github.event_name == 'tag' || (github.ref == 'refs/heads/master' && github.event.pull_request.merged == true && contains(github.event.pull_request.labels.*.name, 'docker'))
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.PERSONAL_GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
- name: Build and push Docker image
uses: docker/build-push-action@v6
with:
context: .
push: ${{ github.event_name == 'tag' || (github.ref == 'refs/heads/master' && github.event.pull_request.merged == true && contains(github.event.pull_request.labels.*.name, 'docker')) }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
platforms: |
linux/arm64
linux/amd64
linux/arm/v7
provenance: true
cache-from: type=gha
cache-to: type=gha,mode=max
================================================
FILE: .github/workflows/publish.yml
================================================
name: Publish to crates.io
on:
push:
tags:
- "v*"
env:
CARGO_TERM_COLOR: always
jobs:
publish:
name: Publish to crates.io
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v5
- name: Set up Rust toolchain
uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: stable
override: true
- name: Publish spiko to crates.io
run: cargo publish --manifest-path Cargo.toml --token ${{ secrets.CARGO_REGISTRY_TOKEN }}
================================================
FILE: .github/workflows/release.yml
================================================
name: Release
on:
push:
tags:
- "v*"
env:
CARGO_TERM_COLOR: always
jobs:
release:
name: "Release"
runs-on: ${{ matrix.os }}
strategy:
matrix:
include:
- os: ubuntu-latest
artifact_name: spiko
asset_name: spiko-linux-gnu-amd64
- os: windows-latest
artifact_name: spiko.exe
asset_name: spiko-windows-amd64.exe
- os: macos-latest
artifact_name: spiko
asset_name: spiko-darwin-amd64
steps:
- name: Checkout repository
uses: actions/checkout@v5
- name: Set up Rust toolchain
uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: stable
override: true
- name: Build release
run: cargo build --release --locked
- name: Set prerelease flag (non-Windows)
if: runner.os != 'Windows'
run: |
if [ $(echo ${{ github.ref }} | grep "rc") ]; then
echo "PRERELEASE=true" >> $GITHUB_ENV
echo "PRERELEASE=true"
else
echo "PRERELEASE=false" >> $GITHUB_ENV
echo "PRERELEASE=false"
fi
echo $PRERELEASE
VERSION=$(echo ${{ github.ref }} | sed 's/refs\/tags\///g')
echo "VERSION=$VERSION" >> $GITHUB_ENV
echo "VERSION=$VERSION"
- name: Set prerelease flag (Windows)
if: runner.os == 'Windows'
shell: powershell
run: |
$full = "${{ github.ref }}"
if ( $full -like '*rc*' ) {
echo "PRERELEASE=true" >> $env:GITHUB_ENV
echo "PRERELEASE=true"
} else {
echo "PRERELEASE=false" >> $env:GITHUB_ENV
echo "PRERELEASE=false"
}
$trimmed = $full -replace 'refs/tags/',''
echo "VERSION=$trimmed" >> $env:GITHUB_ENV
echo "VERSION=$trimmed"
- name: Upload release assets
uses: svenstaro/upload-release-action@v2
with:
repo_token: ${{ secrets.PERSONAL_GITHUB_TOKEN }}
file: target/release/${{ matrix.artifact_name }}
asset_name: ${{ matrix.asset_name }}
tag: ${{ github.ref }}
prerelease: ${{ env.PRERELEASE }}
release_name: "spiko ${{ env.VERSION }}"
body: "Please refer to **[CHANGELOG.md](https://github.com/trinhminhtriet/spiko/blob/master/CHANGELOG.md)** for information on this release."
================================================
FILE: .gitignore
================================================
.DS_Store
target/
================================================
FILE: .vscode/settings.json
================================================
{
"editor.formatOnSave": true
}
================================================
FILE: CHANGELOG.md
================================================
# Changelog
### [v0.1.5](https://github.com/trinhminhtriet/spiko/compare/v0.1.4...v0.1.5) (2024-12-02)
- chore(deps): bump byte-unit from 5.1.4 to 5.1.6
- chore(deps): bump hyper from 1.5.0 to 1.5.1
- chore(deps): bump rustls from 0.23.16 to 0.23.19
- chore(deps): bump jsonschema from 0.26.0 to 0.26.1
- chore(deps): bump aws-lc-rs from 1.10.0 to 1.11.1
### [v0.1.4](https://github.com/trinhminhtriet/spiko/compare/v0.1.3...v0.1.4) (2024-11-30)
### [v0.1.3](https://github.com/trinhminhtriet/spiko/compare/v0.1.2...v0.1.3) (2024-11-10)
### [v0.1.2](https://github.com/trinhminhtriet/spiko/compare/v0.1.1...v0.1.2) (2024-11-10)
### [v0.1.1](https://github.com/trinhminhtriet/spiko/compare/v0.1.0...v0.1.1) (2024-11-10)
#### Features
- add script to automate version bumping and publishing
([efb2f89](https://github.com/trinhminhtriet/spiko/commit/efb2f89832394ff89e018415d5abe8351528f3f1))
#### Fixes
- remove test execution from version bump script
([7976dbd](https://github.com/trinhminhtriet/spiko/commit/7976dbdad6ae762db03cb6d4a7049639b05431d9))
## v0.1.0 (2024-11-10)
================================================
FILE: Cargo.toml
================================================
[package]
name = "spiko"
version = "0.1.16"
authors = ["Triet Trinh <contact@trinhminhtriet.com>"]
edition = "2021"
license = "MIT"
description = "🚀 Spiko is a fast, Rust-based load testing tool with a beautiful TUI for real-time insights."
readme = "README.md"
repository = "https://github.com/trinhminhtriet/spiko"
homepage = "https://trinhminhtriet.com"
keywords = ["cli", "load-testing", "performance", "http"]
categories = [
"command-line-utilities",
"network-programming",
"web-programming::http-client",
"development-tools::profiling",
]
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[features]
default = ["rustls"]
native-tls = ["dep:native-tls", "dep:tokio-native-tls"]
rustls = [
"dep:rustls",
"dep:tokio-rustls",
"dep:rustls-native-certs",
"dep:rustls-pki-types",
]
vsock = ["dep:tokio-vsock"]
[dependencies]
anyhow = "1.0.100"
average = "0.16.0"
byte-unit = "5.1.4"
clap = { version = "4.5.42", features = ["derive"] }
float-ord = "0.3.2"
flume = "0.11"
humantime = "2.2.0"
libc = "0.2.155"
serde = { version = "1.0.204", features = ["derive"] }
serde_json = "1.0"
thiserror = "2.0.12"
tokio = { version = "1.46.1", features = ["full"] }
ratatui = { version = "0.29.0", default-features = false, features = [
"crossterm",
] }
aws-sign-v4 = "0.3"
chrono = "0.4"
bytes = "1"
hyper = { version = "1.7", features = ["client", "http1", "http2"] }
# native-tls
native-tls = { version = "0.2.12", features = ["alpn"], optional = true }
tokio-native-tls = { version = "0.3.1", optional = true }
rustls = { version = "0.23.26", optional = true }
rustls-native-certs = { version = "0.8.0", optional = true }
tokio-rustls = { version = "0.26.4", optional = true }
rustls-pki-types = { version = "1.7.0", optional = true }
base64 = "0.22.1"
rand = "0.9.1"
rand_core = "0.9.3"
hickory-resolver = "0.25.2"
rand_regex = "0.18.1"
regex-syntax = "0.8.5"
url = "2.5.2"
http-body-util = "0.1.2"
hyper-util = { version = "0.1.17", features = ["tokio"] }
tokio-vsock = { version = "0.7.1", optional = true }
rusqlite = { version = "0.37.0", features = ["bundled"] }
num_cpus = "1.17.0"
tokio-util = "0.7.16"
[target.'cfg(not(target_env = "msvc"))'.dependencies]
tikv-jemallocator = "0.6"
[target.'cfg(unix)'.dependencies]
rlimit = "0.10.1"
[dev-dependencies]
assert_cmd = "2.1.1"
axum = { version = "0.8.6", features = ["http2"] }
axum-server = { version = "0.7.2", features = ["tls-rustls"] }
bytes = "1.6"
float-cmp = "0.10.0"
http-mitm-proxy = { version = "0.16.0", default-features = false }
jsonschema = "0.30.0"
lazy_static = "1.5.0"
predicates = "3.1.0"
# features = ["aws_lc_rs"] is a workaround for mac & native-tls
# https://github.com/sfackler/rust-native-tls/issues/225
rcgen = { version = "0.13.1", features = ["aws_lc_rs"] }
regex = "1.11.3"
tempfile = "3.21.0"
rustls = "0.23.26"
[target.'cfg(unix)'.dev-dependencies]
actix-web = "4"
[profile.pgo]
inherits = "release"
# https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/Rust/faf/Cargo.toml + lto=true
opt-level = 3
panic = 'abort'
codegen-units = 1
lto = true
debug = false
incremental = false
overflow-checks = false
[profile.release-ci]
inherits = "pgo"
================================================
FILE: Cross.toml
================================================
# For Asahi linux
[target.aarch64-unknown-linux-gnu.env]
passthrough = ["JEMALLOC_SYS_WITH_LG_PAGE=16"]
[target.aarch64-unknown-linux-musl.env]
passthrough = ["JEMALLOC_SYS_WITH_LG_PAGE=16"]
================================================
FILE: Dockerfile
================================================
FROM rust:1.84.0-bookworm AS builder
# Install dependencies including LLVM 14 first
RUN apt-get update && apt-get install -y \
clang \
cmake \
libssl-dev \
pkg-config \
llvm-14 \
libclang-14-dev
# Set LIBCLANG_PATH after LLVM 14 is installed
ENV LIBCLANG_PATH=/usr/lib/llvm-14/lib
WORKDIR /app
COPY . /app
RUN cargo build --release
FROM debian:bookworm-slim
RUN apt-get update \
&& rm -rf /var/lib/apt/lists/*
COPY --link --from=builder /app/target/release/spiko /usr/local/bin/spiko
ENTRYPOINT ["spiko"]
================================================
FILE: LICENSE
================================================
MIT License
Copyright (c) 2020 hatoo
Copyright (c) 2024 trinhminhtriet.com
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
================================================
FILE: Makefile
================================================
NAME := spiko
AUTHOR := trinhminhtriet
DATE := $(shell date +%FT%T%Z)
GIT := $(shell [ -d .git ] && git rev-parse --short HEAD)
VERSION := $(shell git describe --tags)
default: build
clean:
@echo "Cleaning build dir"
@$(RM) -r target
@echo "Cleaning using cargo"
@cargo clean
check:
@echo "Checking $(NAME)"
@cargo check
test:
@echo "Running tests"
@cargo test
@echo "Running tests with coverage and open report in browser"
@cargo tarpaulin --out Html --open
build:
@echo "Building release: $(VERSION)"
@cargo build --release
ln -sf $(PWD)/target/release/$(NAME) $(HOME)/.local/bin/$(NAME)
which $(NAME)
$(NAME) --version
build_debug:
@echo "Building debug"
@cargo build
run:
@echo "Running debug"
@cargo run
release:
./scripts/release-version.sh
================================================
FILE: README.md
================================================
# 🚀 spiko
```text
_ _
___ _ __ (_)| | __ ___
/ __|| '_ \ | || |/ / / _ \
\__ \| |_) || || < | (_) |
|___/| .__/ |_||_|\_\ \___/
|_|
```
Spiko is a fast, lightweight load testing tool built with Rust and powered by Tokio. It offers a clean and interactive TUI (Text User Interface) to provide real-time insights into your web application’s performance. Inspired by `trinhminhtriet/blast`, Spiko helps you simulate load and monitor the results in an intuitive and easy-to-understand interface.
## ✨ Features
- 🚀 High-performance load testing with minimal overhead
- 🎨 Real-time TUI with beautiful, interactive graphs
- ⚡ Powered by Rust and Tokio for fast, reliable results
- 🧑💻 Simple configuration with easy-to-understand commands
- 📊 Visual feedback on request rates, latency, and more
## 🚀 Installation
To install **spiko**, simply clone the repository and follow the instructions below:
```bash
git clone git@github.com:trinhminhtriet/spiko.git
cd spiko
cargo build --release
cp ./target/release/spiko /usr/local/bin/
spiko --version
spiko --help
spiko -n 1000 https://github.com
```
Running the below command will globally install the `spiko` binary.
```bash
cargo install spiko
```
Optionally, you can add `~/.cargo/bin` to your PATH if it's not already there
```bash
echo 'export PATH="$HOME/.cargo/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc
```
## 💡 Usage
```
Usage: spiko [OPTIONS] <URL>
Arguments:
<URL> Target URL.
Options:
-n <N_REQUESTS>
Number of requests to run. [default: 200]
-c <N_CONNECTIONS>
Number of connections to run concurrently. You may should increase limit to number of open files for larger `-c`. [default: 50]
-p <N_HTTP2_PARALLEL>
Number of parallel requests to send on HTTP/2. `spiko` will run c * p concurrent workers in total. [default: 1]
-z <DURATION>
Duration of application to send requests. If duration is specified, n is ignored.
On HTTP/1, When the duration is reached, ongoing requests are aborted and counted as "aborted due to deadline"
You can change this behavior with `-w` option.
Currently, on HTTP/2, When the duration is reached, ongoing requests are waited. `-w` option is ignored.
Examples: -z 10s -z 3m.
-w, --wait-ongoing-requests-after-deadline
When the duration is reached, ongoing requests are waited
-q <QUERY_PER_SECOND>
Rate limit for all, in queries per second (QPS)
--burst-delay <BURST_DURATION>
Introduce delay between a predefined number of requests.
Note: If qps is specified, burst will be ignored
--burst-rate <BURST_REQUESTS>
Rates of requests for burst. Default is 1
Note: If qps is specified, burst will be ignored
--rand-regex-url
Generate URL by rand_regex crate but dot is disabled for each query e.g. http://127.0.0.1/[a-z][a-z][0-9]. Currently dynamic scheme, host and port with keep-alive are not works well. See https://docs.rs/rand_regex/latest/rand_regex/struct.Regex.html for details of syntax.
--max-repeat <MAX_REPEAT>
A parameter for the '--rand-regex-url'. The max_repeat parameter gives the maximum extra repeat counts the x*, x+ and x{n,} operators will become. [default: 4]
--dump-urls <DUMP_URLS>
Dump target Urls <DUMP_URLS> times to debug --rand-regex-url
--latency-correction
Correct latency to avoid coordinated omission problem. It's ignored if -q is not set.
--no-tui
No realtime tui
-j, --json
Print results as JSON
--fps <FPS>
Frame per second for tui. [default: 16]
-m, --method <METHOD>
HTTP method [default: GET]
-H <HEADERS>
Custom HTTP header. Examples: -H "foo: bar"
-t <TIMEOUT>
Timeout for each request. Default to infinite.
-A <ACCEPT_HEADER>
HTTP Accept Header.
-d <BODY_STRING>
HTTP request body.
-D <BODY_PATH>
HTTP request body from file.
-T <CONTENT_TYPE>
Content-Type.
-a <BASIC_AUTH>
Basic authentication, username:password
--http-version <HTTP_VERSION>
HTTP version. Available values 0.9, 1.0, 1.1.
--http2
Use HTTP/2. Shorthand for --http-version=2
--host <HOST>
HTTP Host header
--disable-compression
Disable compression.
-r, --redirect <REDIRECT>
Limit for number of Redirect. Set 0 for no redirection. Redirection isn't supported for HTTP/2. [default: 10]
--disable-keepalive
Disable keep-alive, prevents re-use of TCP connections between different HTTP requests. This isn't supported for HTTP/2.
--no-pre-lookup
*Not* perform a DNS lookup at beginning to cache it
--ipv6
Lookup only ipv6.
--ipv4
Lookup only ipv4.
--insecure
Accept invalid certs.
--connect-to <CONNECT_TO>
Override DNS resolution and default port numbers with strings like 'example.org:443:localhost:8443'
--disable-color
Disable the color scheme.
--unix-socket <UNIX_SOCKET>
Connect to a unix socket instead of the domain in the URL. Only for non-HTTPS URLs.
--stats-success-breakdown
Include a response status code successful or not successful breakdown for the time histogram and distribution statistics
--db-url <DB_URL>
Write succeeded requests to sqlite database url E.G test.db
--debug
Perform a single request and dump the request and response
-h, --help
Print help
-V, --version
Print version
```
## 🗑️ Uninstallation
Running the below command will globally uninstall the `spiko` binary.
```bash
cargo uninstall spiko
```
Remove the project repo
```bash
rm -rf /path/to/git/clone/spiko
```
## 🤝 How to contribute
We welcome contributions!
- Fork this repository;
- Create a branch with your feature: `git checkout -b my-feature`;
- Commit your changes: `git commit -m "feat: my new feature"`;
- Push to your branch: `git push origin my-feature`.
Once your pull request has been merged, you can delete your branch.
## 🙏 Acknowledgements
- [hatoo/oha](https://github.com/hatoo/oha)
## 📝 License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
================================================
FILE: demo.tape
================================================
Output media/demo.gif
Set FontSize 16
Set Width 1440
Set Height 768
Set TypingSpeed 400ms
Type@100ms "echo 'https://trinhminhtriet.com'"
Sleep 50ms
Enter
Type@50ms "spiko --version"
Sleep 50ms
Enter
Sleep 100ms
Type@50ms "spiko --help"
Sleep 50ms
Enter
Sleep 100ms
Type@20ms "spiko -n 1000 https://github.com"
Sleep 100ms
Enter
Sleep 20s
================================================
FILE: pgo/server/Cargo.toml
================================================
[package]
name = "server"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
axum = "0.8.1"
tokio = { version = "1", features = ["full"] }
================================================
FILE: pgo/server/src/main.rs
================================================
use std::net::SocketAddr;
use tokio::net::TcpListener;
use axum::{routing::get, Router};
#[tokio::main]
async fn main() {
// build our application with a route
let app = Router::new()
// `GET /` goes to `root`
.route("/", get(root));
// run our app with hyper
// `axum::Server` is a re-export of `hyper::Server`
let addr = SocketAddr::from(([127, 0, 0, 1], 8888));
let listener = TcpListener::bind(&addr).await.unwrap();
axum::serve(listener, app).await.unwrap();
}
async fn root() -> &'static str {
"Hello, World!"
}
================================================
FILE: pgo.js
================================================
import { $ } from "bun";
let additional = [];
if (Bun.argv.length >= 3) {
additional = Bun.argv.slice(2);
}
let server = null;
try {
server = Bun.spawn(['cargo', 'run', '--release', '--manifest-path', 'pgo/server/Cargo.toml']);
await $`cargo pgo run -- --profile pgo ${additional} -- -z 3m -c 900 --no-tui http://localhost:8888`;
await $`cargo pgo optimize build -- --profile pgo ${additional}`
} finally {
if (server !== null) {
server.kill();
}
}
================================================
FILE: schema.json
================================================
{
"$schema": "http://json-schema.org/draft-04/schema#",
"description": "JSON schema for the output of the `spiko -j`",
"type": "object",
"properties": {
"summary": {
"description": "Important statistics",
"type": "object",
"properties": {
"successRate": {
"description": "The number of success requests / All requests which isn't includes deadline",
"type": "number"
},
"total": {
"description": "Total duration in seconds",
"type": "number"
},
"slowest": {
"description": "The slowest request duration in seconds",
"type": "number"
},
"fastest": {
"description": "The fastest request duration in seconds",
"type": "number"
},
"average": {
"description": "The average request duration in seconds",
"type": "number"
},
"requestsPerSec": {
"description": "The number of requests per second",
"type": "number"
},
"totalData": {
"description": "Total data of HTTP bodies in bytes",
"type": "integer"
},
"sizePerRequest": {
"description": "The average size of HTTP bodies in bytes",
"type": "integer"
},
"sizePerSec": {
"description": "The average size of HTTP bodies per second in bytes",
"type": "number"
}
},
"required": [
"successRate",
"total",
"slowest",
"fastest",
"average",
"requestsPerSec",
"totalData",
"sizePerRequest",
"sizePerSec"
]
},
"responseTimeHistogram": {
"description": "The histogram of response time in seconds. The key is the response time in seconds and the value is the number of requests",
"type": "object",
"additionalProperties": {
"string": "integer"
}
},
"latencyPercentiles": {
"description": "The latency percentiles in seconds",
"type": "object",
"properties": {
"p10": {
"type": "number"
},
"p25": {
"type": "number"
},
"p50": {
"type": "number"
},
"p75": {
"type": "number"
},
"p90": {
"type": "number"
},
"p95": {
"type": "number"
},
"p99": {
"type": "number"
},
"p99.9": {
"type": "number"
},
"p99.99": {
"type": "number"
}
},
"required": [
"p10",
"p25",
"p50",
"p75",
"p90",
"p95",
"p99",
"p99.9",
"p99.99"
]
},
"responseTimeHistogramSuccessful": {
"description": "Only present if `--stats-success-breakdown` argument is passed. The histogram of response time in seconds for successful requests. The key is the response time in seconds and the value is the number of requests",
"type": "object",
"additionalProperties": {
"string": "integer"
}
},
"latencyPercentileSuccessful": {
"description": "Only present if `--stats-success-breakdown` argument is passed. The latency percentiles in seconds for successful requests",
"type": "object",
"properties": {
"p10": {
"type": "number"
},
"p25": {
"type": "number"
},
"p50": {
"type": "number"
},
"p75": {
"type": "number"
},
"p90": {
"type": "number"
},
"p95": {
"type": "number"
},
"p99": {
"type": "number"
},
"p99.9": {
"type": "number"
},
"p99.99": {
"type": "number"
}
},
"required": [
"p10",
"p25",
"p50",
"p75",
"p90",
"p95",
"p99",
"p99.9",
"p99.99"
]
},
"responseTimeHistogramNotSuccessful": {
"description": "Only present if `--stats-success-breakdown` argument is passed. The histogram of response time in seconds for not successful requests. The key is the response time in seconds and the value is the number of requests",
"type": "object",
"additionalProperties": {
"string": "integer"
}
},
"latencyPercentileNotSuccessful": {
"description": "Only present if `--stats-success-breakdown` argument is passed. The latency percentiles in seconds for not successful requests",
"type": "object",
"properties": {
"p10": {
"type": "number"
},
"p25": {
"type": "number"
},
"p50": {
"type": "number"
},
"p75": {
"type": "number"
},
"p90": {
"type": "number"
},
"p95": {
"type": "number"
},
"p99": {
"type": "number"
},
"p99.9": {
"type": "number"
},
"p99.99": {
"type": "number"
}
},
"required": [
"p10",
"p25",
"p50",
"p75",
"p90",
"p95",
"p99",
"p99.9",
"p99.99"
]
},
"rps": {
"description": "The statistics for requests per second. Note: the way of calculating rps over time isn't obvious, see source code for details.",
"type": "object",
"properties": {
"mean": {
"type": "number"
},
"stddev": {
"type": ["number", "null"]
},
"max": {
"type": "number"
},
"min": {
"type": "number"
},
"percentiles": {
"type": "object",
"properties": {
"p10": {
"type": "number"
},
"p25": {
"type": "number"
},
"p50": {
"type": "number"
},
"p75": {
"type": "number"
},
"p90": {
"type": "number"
},
"p95": {
"type": "number"
},
"p99": {
"type": "number"
},
"p99.9": {
"type": "number"
},
"p99.99": {
"type": "number"
}
},
"required": [
"p10",
"p25",
"p50",
"p75",
"p90",
"p95",
"p99",
"p99.9",
"p99.99"
]
}
},
"required": ["mean", "stddev", "max", "min", "percentiles"]
},
"details": {
"description": "The details of connection time. Note: `spiko` uses keep-alive connections in default. So, the connection time may added only for the first request.",
"type": "object",
"properties": {
"DNSDialup": {
"description": "The time of DNS resolution + TCP handshake in seconds",
"type": "object",
"properties": {
"average": {
"type": "number"
},
"fastest": {
"type": "number"
},
"slowest": {
"type": "number"
}
},
"required": ["average", "fastest", "slowest"]
},
"DNSLookup": {
"description": "The time of DNS resolution in seconds",
"type": "object",
"properties": {
"average": {
"type": "number"
},
"fastest": {
"type": "number"
},
"slowest": {
"type": "number"
}
},
"required": ["average", "fastest", "slowest"]
}
},
"required": ["DNSDialup", "DNSLookup"]
},
"statusCodeDistribution": {
"description": "The distribution of status codes. The key is the status code and the value is the number of requests",
"type": "object",
"additionalProperties": {
"string": "integer"
}
},
"errorDistribution": {
"description": "The distribution of errors. The key is the error message and the value is the number of errors. Note: the error message is from internal libraries so the detail may change in future.",
"type": "object",
"additionalProperties": {
"string": "integer"
}
}
},
"required": [
"summary",
"responseTimeHistogram",
"latencyPercentiles",
"rps",
"details",
"statusCodeDistribution",
"errorDistribution"
]
}
================================================
FILE: scripts/release-version.sh
================================================
#!/bin/bash
set -xe
[ -z "$(git status --porcelain)" ] || (echo "dirty working directory" && exit 1)
current_version="$(grep '^version = ' Cargo.toml | head -1 | cut -d '"' -f2)"
IFS='.' read -r major minor patch <<<"$current_version"
new_patch=$((patch + 1))
new_version="$major.$minor.$new_patch"
tag_name="v$new_version"
if [ -z "$new_version" ]; then
echo "New version required as argument"
exit 1
fi
echo ">>> Bumping version"
sed -i.bak "s/version = \"$current_version\"/version = \"$new_version\"/" Cargo.toml
rm Cargo.toml.bak
sleep 10
echo ">>> Commit"
git add Cargo.toml Cargo.lock
git commit -am "version $new_version"
git tag $tag_name
echo ">>> Publish"
git push
git push origin $tag_name
echo ">>> Done"
================================================
FILE: src/aws_auth.rs
================================================
use crate::client::ClientError;
use anyhow::Result;
use bytes::Bytes;
use hyper::{
HeaderMap,
header::{self, HeaderName},
};
use url::Url;
pub struct AwsSignatureConfig {
pub access_key: String,
pub secret_key: String,
pub session_token: Option<String>,
pub service: String,
pub region: String,
}
// Initialize unsignable headers as a static constant
static UNSIGNABLE_HEADERS: [HeaderName; 8] = [
header::ACCEPT,
header::ACCEPT_ENCODING,
header::USER_AGENT,
header::EXPECT,
header::RANGE,
header::CONNECTION,
HeaderName::from_static("presigned-expires"),
HeaderName::from_static("x-amzn-trace-id"),
];
impl AwsSignatureConfig {
pub fn sign_request(
&self,
method: &str,
headers: &mut HeaderMap,
url: &Url,
body: Option<Bytes>,
) -> Result<(), ClientError> {
let datetime = chrono::Utc::now();
let header_amz_date = datetime
.format("%Y%m%dT%H%M%SZ")
.to_string()
.parse()
.unwrap();
if !headers.contains_key(header::HOST) {
let host = url
.host_str()
.ok_or_else(|| ClientError::SigV4Error("URL must contain a host"))?;
headers.insert(
header::HOST,
host.parse()
.map_err(|_| ClientError::SigV4Error("Invalid host header name"))?,
);
}
headers.insert("x-amz-date", header_amz_date);
if let Some(session_token) = &self.session_token {
headers.insert("x-amz-security-token", session_token.parse().unwrap());
}
headers.remove(header::AUTHORIZATION);
//remove and store headers in a vec from unsignable_headers
let removed_headers: Vec<(header::HeaderName, header::HeaderValue)> = UNSIGNABLE_HEADERS
.iter()
.filter_map(|k| headers.remove(k).map(|v| (k.clone(), v)))
.collect();
let body = body.as_deref().unwrap_or_default();
headers.insert(
header::CONTENT_LENGTH,
body.len().to_string().parse().unwrap(),
);
let aws_sign = aws_sign_v4::AwsSign::new(
method,
url.as_str(),
&datetime,
headers,
&self.region,
&self.access_key,
&self.secret_key,
&self.service,
body,
);
let signature = aws_sign.sign();
//insert headers
for (key, value) in removed_headers {
headers.insert(key, value);
}
headers.insert(
header::AUTHORIZATION,
signature
.parse()
.map_err(|_| ClientError::SigV4Error("Invalid authorization header name"))?,
);
Ok(())
}
pub fn new(
access_key: &str,
secret_key: &str,
signing_params: &str,
session_token: Option<String>,
) -> Result<Self, anyhow::Error> {
let parts: Vec<&str> = signing_params
.strip_prefix("aws:amz:")
.unwrap_or_default()
.split(':')
.collect();
if parts.len() != 2 {
anyhow::bail!("Invalid AWS signing params format. Expected aws:amz:region:service");
}
Ok(Self {
access_key: access_key.into(),
secret_key: secret_key.into(),
session_token,
region: parts[0].to_string(),
service: parts[1].to_string(),
})
}
}
================================================
FILE: src/client.rs
================================================
use bytes::Bytes;
use http_body_util::{BodyExt, Full};
use hyper::{Method, http};
use hyper_util::rt::{TokioExecutor, TokioIo};
use rand::prelude::*;
use std::{
borrow::Cow,
io::Write,
sync::{
Arc,
atomic::{AtomicBool, Ordering::Relaxed},
},
time::Instant,
};
use thiserror::Error;
use tokio::{
io::{AsyncRead, AsyncWrite},
net::TcpStream,
};
use url::{ParseError, Url};
use crate::{
ConnectToEntry,
aws_auth::AwsSignatureConfig,
pcg64si::Pcg64Si,
url_generator::{UrlGenerator, UrlGeneratorError},
};
type SendRequestHttp1 = hyper::client::conn::http1::SendRequest<Full<Bytes>>;
type SendRequestHttp2 = hyper::client::conn::http2::SendRequest<Full<Bytes>>;
#[derive(Debug, Clone, Copy)]
pub struct ConnectionTime {
pub dns_lookup: std::time::Instant,
pub dialup: std::time::Instant,
}
#[derive(Debug, Clone)]
/// a result for a request
pub struct RequestResult {
pub rng: Pcg64Si,
// When the query should started
pub start_latency_correction: Option<std::time::Instant>,
/// When the query started
pub start: std::time::Instant,
/// DNS + dialup
/// None when reuse connection
pub connection_time: Option<ConnectionTime>,
/// When the query ends
pub end: std::time::Instant,
/// HTTP status
pub status: http::StatusCode,
/// Length of body
pub len_bytes: usize,
}
impl RequestResult {
/// Duration the request takes.
pub fn duration(&self) -> std::time::Duration {
self.end - self.start_latency_correction.unwrap_or(self.start)
}
}
pub struct Dns {
pub connect_to: Vec<ConnectToEntry>,
pub resolver:
hickory_resolver::AsyncResolver<hickory_resolver::name_server::TokioConnectionProvider>,
}
impl Dns {
/// Perform a DNS lookup for a given url and returns (ip_addr, port)
async fn lookup<R: Rng>(
&self,
url: &Url,
rng: &mut R,
) -> Result<(std::net::IpAddr, u16), ClientError> {
let host = url.host_str().ok_or(ClientError::HostNotFound)?;
let port = url
.port_or_known_default()
.ok_or(ClientError::PortNotFound)?;
// Try to find an override (passed via `--connect-to`) that applies to this (host, port),
// choosing one randomly if several match.
let (host, port) = if let Some(entry) = self
.connect_to
.iter()
.filter(|entry| entry.requested_port == port && entry.requested_host == host)
.collect::<Vec<_>>()
.choose(rng)
{
(entry.target_host.as_str(), entry.target_port)
} else {
(host, port)
};
let host = if host.starts_with('[') && host.ends_with(']') {
// host is [ipv6] format
// remove first [ and last ]
&host[1..host.len() - 1]
} else {
host
};
// Perform actual DNS lookup, either on the original (host, port), or
// on the (host, port) specified with `--connect-to`.
let addrs = self
.resolver
.lookup_ip(host)
.await
.map_err(Box::new)?
.iter()
.collect::<Vec<_>>();
let addr = *addrs.choose(rng).ok_or(ClientError::DNSNoRecord)?;
Ok((addr, port))
}
}
#[derive(Error, Debug)]
pub enum ClientError {
#[error("failed to get port from URL")]
PortNotFound,
#[error("failed to get host from URL")]
HostNotFound,
#[error("No record returned from DNS")]
DNSNoRecord,
#[error("Redirection limit has reached")]
TooManyRedirect,
#[error(transparent)]
// Use Box here because ResolveError is big.
ResolveError(#[from] Box<hickory_resolver::error::ResolveError>),
#[cfg(feature = "native-tls")]
#[error(transparent)]
NativeTlsError(#[from] native_tls::Error),
#[cfg(feature = "rustls")]
#[error(transparent)]
RustlsError(#[from] rustls::Error),
#[cfg(feature = "rustls")]
#[error(transparent)]
InvalidDnsName(#[from] rustls_pki_types::InvalidDnsNameError),
#[error(transparent)]
IoError(#[from] std::io::Error),
#[error(transparent)]
HttpError(#[from] http::Error),
#[error(transparent)]
HyperError(#[from] hyper::Error),
#[error(transparent)]
InvalidUriParts(#[from] http::uri::InvalidUriParts),
#[error(transparent)]
InvalidHeaderValue(#[from] http::header::InvalidHeaderValue),
#[error("Failed to get header from builder")]
GetHeaderFromBuilderError,
#[error(transparent)]
HeaderToStrError(#[from] http::header::ToStrError),
#[error(transparent)]
InvalidUri(#[from] http::uri::InvalidUri),
#[error("timeout")]
Timeout,
#[error("aborted due to deadline")]
Deadline,
#[error(transparent)]
UrlGeneratorError(#[from] UrlGeneratorError),
#[error(transparent)]
UrlParseError(#[from] ParseError),
#[error("AWS SigV4 signature error: {0}")]
SigV4Error(&'static str),
}
pub struct Client {
pub http_version: http::Version,
pub proxy_http_version: http::Version,
pub url_generator: UrlGenerator,
pub method: http::Method,
pub headers: http::header::HeaderMap,
pub proxy_headers: http::header::HeaderMap,
pub body: Option<&'static [u8]>,
pub dns: Dns,
pub timeout: Option<std::time::Duration>,
pub redirect_limit: usize,
pub disable_keepalive: bool,
pub proxy_url: Option<Url>,
pub aws_config: Option<AwsSignatureConfig>,
#[cfg(unix)]
pub unix_socket: Option<std::path::PathBuf>,
#[cfg(feature = "vsock")]
pub vsock_addr: Option<tokio_vsock::VsockAddr>,
#[cfg(feature = "rustls")]
pub rustls_configs: crate::tls_config::RuslsConfigs,
#[cfg(all(feature = "native-tls", not(feature = "rustls")))]
pub native_tls_connectors: crate::tls_config::NativeTlsConnectors,
}
impl Default for Client {
fn default() -> Self {
Self {
http_version: http::Version::HTTP_11,
proxy_http_version: http::Version::HTTP_11,
url_generator: UrlGenerator::new_static("http://example.com".parse().unwrap()),
method: http::Method::GET,
headers: http::header::HeaderMap::new(),
proxy_headers: http::header::HeaderMap::new(),
body: None,
dns: Dns {
resolver: hickory_resolver::AsyncResolver::tokio_from_system_conf().unwrap(),
connect_to: Vec::new(),
},
timeout: None,
redirect_limit: 0,
disable_keepalive: false,
proxy_url: None,
aws_config: None,
#[cfg(unix)]
unix_socket: None,
#[cfg(feature = "vsock")]
vsock_addr: None,
#[cfg(feature = "rustls")]
rustls_configs: crate::tls_config::RuslsConfigs::new(false, None, None),
#[cfg(all(feature = "native-tls", not(feature = "rustls")))]
native_tls_connectors: crate::tls_config::NativeTlsConnectors::new(false, None, None),
}
}
}
struct ClientStateHttp1 {
rng: Pcg64Si,
send_request: Option<SendRequestHttp1>,
}
impl Default for ClientStateHttp1 {
fn default() -> Self {
Self {
rng: SeedableRng::from_os_rng(),
send_request: None,
}
}
}
struct ClientStateHttp2 {
rng: Pcg64Si,
send_request: SendRequestHttp2,
}
pub enum QueryLimit {
Qps(usize),
Burst(std::time::Duration, usize),
}
// To avoid dynamic dispatch
// I'm not sure how much this is effective
enum Stream {
Tcp(TcpStream),
#[cfg(all(feature = "native-tls", not(feature = "rustls")))]
Tls(tokio_native_tls::TlsStream<TcpStream>),
#[cfg(feature = "rustls")]
// Box for large variant
Tls(Box<tokio_rustls::client::TlsStream<TcpStream>>),
#[cfg(unix)]
Unix(tokio::net::UnixStream),
#[cfg(feature = "vsock")]
Vsock(tokio_vsock::VsockStream),
}
impl Stream {
async fn handshake_http1(self, with_upgrade: bool) -> Result<SendRequestHttp1, ClientError> {
match self {
Stream::Tcp(stream) => {
let (send_request, conn) =
hyper::client::conn::http1::handshake(TokioIo::new(stream)).await?;
if with_upgrade {
tokio::spawn(conn.with_upgrades());
} else {
tokio::spawn(conn);
}
Ok(send_request)
}
Stream::Tls(stream) => {
let (send_request, conn) =
hyper::client::conn::http1::handshake(TokioIo::new(stream)).await?;
if with_upgrade {
tokio::spawn(conn.with_upgrades());
} else {
tokio::spawn(conn);
}
Ok(send_request)
}
#[cfg(unix)]
Stream::Unix(stream) => {
let (send_request, conn) =
hyper::client::conn::http1::handshake(TokioIo::new(stream)).await?;
if with_upgrade {
tokio::spawn(conn.with_upgrades());
} else {
tokio::spawn(conn);
}
Ok(send_request)
}
#[cfg(feature = "vsock")]
Stream::Vsock(stream) => {
let (send_request, conn) =
hyper::client::conn::http1::handshake(TokioIo::new(stream)).await?;
if with_upgrade {
tokio::spawn(conn.with_upgrades());
} else {
tokio::spawn(conn);
}
Ok(send_request)
}
}
}
async fn handshake_http2(self) -> Result<SendRequestHttp2, ClientError> {
let mut builder = hyper::client::conn::http2::Builder::new(TokioExecutor::new());
builder
// from nghttp2's default
.initial_stream_window_size((1 << 30) - 1)
.initial_connection_window_size((1 << 30) - 1);
match self {
Stream::Tcp(stream) => {
let (send_request, conn) = builder.handshake(TokioIo::new(stream)).await?;
tokio::spawn(conn);
Ok(send_request)
}
Stream::Tls(stream) => {
let (send_request, conn) = builder.handshake(TokioIo::new(stream)).await?;
tokio::spawn(conn);
Ok(send_request)
}
#[cfg(unix)]
Stream::Unix(stream) => {
let (send_request, conn) = builder.handshake(TokioIo::new(stream)).await?;
tokio::spawn(conn);
Ok(send_request)
}
#[cfg(feature = "vsock")]
Stream::Vsock(stream) => {
let (send_request, conn) = builder.handshake(TokioIo::new(stream)).await?;
tokio::spawn(conn);
Ok(send_request)
}
}
}
}
impl Client {
#[inline]
fn is_http2(&self) -> bool {
self.http_version == http::Version::HTTP_2
}
#[inline]
fn is_proxy_http2(&self) -> bool {
self.proxy_http_version == http::Version::HTTP_2
}
pub fn is_work_http2(&self) -> bool {
if self.proxy_url.is_some() {
let url = self
.url_generator
.generate(&mut Pcg64Si::from_seed([0, 0, 0, 0, 0, 0, 0, 0]))
.unwrap();
if url.scheme() == "https" {
self.is_http2()
} else {
self.is_proxy_http2()
}
} else {
self.is_http2()
}
}
/// Perform a DNS lookup to cache it
/// This is useful to avoid DNS lookup latency at the first concurrent requests
pub async fn pre_lookup(&self) -> Result<(), ClientError> {
// If the client is using a unix socket, we don't need to do a DNS lookup
#[cfg(unix)]
if self.unix_socket.is_some() {
return Ok(());
}
// If the client is using a vsock address, we don't need to do a DNS lookup
#[cfg(feature = "vsock")]
if self.vsock_addr.is_some() {
return Ok(());
}
let mut rng = StdRng::from_os_rng();
let url = self.url_generator.generate(&mut rng)?;
// It automatically caches the result
self.dns.lookup(&url, &mut rng).await?;
Ok(())
}
pub fn generate_url(&self, rng: &mut Pcg64Si) -> Result<(Cow<Url>, Pcg64Si), ClientError> {
let snapshot = *rng;
Ok((self.url_generator.generate(rng)?, snapshot))
}
async fn client<R: Rng>(
&self,
url: &Url,
rng: &mut R,
is_http2: bool,
) -> Result<(Instant, Stream), ClientError> {
// TODO: Allow the connect timeout to be configured
let timeout_duration = tokio::time::Duration::from_secs(5);
if url.scheme() == "https" {
let addr = self.dns.lookup(url, rng).await?;
let dns_lookup = Instant::now();
// If we do not put a timeout here then the connections attempts will
// linger long past the configured timeout
let stream =
tokio::time::timeout(timeout_duration, self.tls_client(addr, url, is_http2)).await;
return match stream {
Ok(Ok(stream)) => Ok((dns_lookup, stream)),
Ok(Err(err)) => Err(err),
Err(_) => Err(ClientError::Timeout),
};
}
#[cfg(unix)]
if let Some(socket_path) = &self.unix_socket {
let dns_lookup = Instant::now();
let stream = tokio::time::timeout(
timeout_duration,
tokio::net::UnixStream::connect(socket_path),
)
.await;
return match stream {
Ok(Ok(stream)) => Ok((dns_lookup, Stream::Unix(stream))),
Ok(Err(err)) => Err(ClientError::IoError(err)),
Err(_) => Err(ClientError::Timeout),
};
}
#[cfg(feature = "vsock")]
if let Some(addr) = self.vsock_addr {
let dns_lookup = Instant::now();
let stream =
tokio::time::timeout(timeout_duration, tokio_vsock::VsockStream::connect(addr))
.await;
return match stream {
Ok(Ok(stream)) => Ok((dns_lookup, Stream::Vsock(stream))),
Ok(Err(err)) => Err(ClientError::IoError(err)),
Err(_) => Err(ClientError::Timeout),
};
}
// HTTP
let addr = self.dns.lookup(url, rng).await?;
let dns_lookup = Instant::now();
let stream =
tokio::time::timeout(timeout_duration, tokio::net::TcpStream::connect(addr)).await;
match stream {
Ok(Ok(stream)) => {
stream.set_nodelay(true)?;
Ok((dns_lookup, Stream::Tcp(stream)))
}
Ok(Err(err)) => Err(ClientError::IoError(err)),
Err(_) => Err(ClientError::Timeout),
}
}
async fn tls_client(
&self,
addr: (std::net::IpAddr, u16),
url: &Url,
is_http2: bool,
) -> Result<Stream, ClientError> {
let stream = tokio::net::TcpStream::connect(addr).await?;
stream.set_nodelay(true)?;
let stream = self.connect_tls(stream, url, is_http2).await?;
Ok(Stream::Tls(stream))
}
#[cfg(all(feature = "native-tls", not(feature = "rustls")))]
async fn connect_tls<S>(
&self,
stream: S,
url: &Url,
is_http2: bool,
) -> Result<tokio_native_tls::TlsStream<S>, ClientError>
where
S: AsyncRead + AsyncWrite + Unpin,
{
let connector = self.native_tls_connectors.connector(is_http2);
let stream = connector
.connect(url.host_str().ok_or(ClientError::HostNotFound)?, stream)
.await?;
Ok(stream)
}
#[cfg(feature = "rustls")]
async fn connect_tls<S>(
&self,
stream: S,
url: &Url,
is_http2: bool,
) -> Result<Box<tokio_rustls::client::TlsStream<S>>, ClientError>
where
S: AsyncRead + AsyncWrite + Unpin,
{
let connector =
tokio_rustls::TlsConnector::from(self.rustls_configs.config(is_http2).clone());
let domain = rustls_pki_types::ServerName::try_from(
url.host_str().ok_or(ClientError::HostNotFound)?,
)?;
let stream = connector.connect(domain.to_owned(), stream).await?;
Ok(Box::new(stream))
}
async fn client_http1<R: Rng>(
&self,
url: &Url,
rng: &mut R,
) -> Result<(Instant, SendRequestHttp1), ClientError> {
if let Some(proxy_url) = &self.proxy_url {
let (dns_lookup, stream) = self.client(proxy_url, rng, self.is_proxy_http2()).await?;
if url.scheme() == "https" {
// Do CONNECT request to proxy
let req = {
let mut builder =
http::Request::builder()
.method(Method::CONNECT)
.uri(format!(
"{}:{}",
url.host_str().unwrap(),
url.port_or_known_default().unwrap()
));
*builder
.headers_mut()
.ok_or(ClientError::GetHeaderFromBuilderError)? =
self.proxy_headers.clone();
builder.body(http_body_util::Full::default())?
};
let res = if self.proxy_http_version == http::Version::HTTP_2 {
let mut send_request = stream.handshake_http2().await?;
send_request.send_request(req).await?
} else {
let mut send_request = stream.handshake_http1(true).await?;
send_request.send_request(req).await?
};
let stream = hyper::upgrade::on(res).await?;
let stream = self.connect_tls(TokioIo::new(stream), url, false).await?;
let (send_request, conn) =
hyper::client::conn::http1::handshake(TokioIo::new(stream)).await?;
tokio::spawn(conn);
Ok((dns_lookup, send_request))
} else {
// Send full URL in request() for HTTP proxy
Ok((dns_lookup, stream.handshake_http1(false).await?))
}
} else {
let (dns_lookup, stream) = self.client(url, rng, false).await?;
Ok((dns_lookup, stream.handshake_http1(false).await?))
}
}
#[inline]
fn request(&self, url: &Url) -> Result<http::Request<Full<Bytes>>, ClientError> {
let use_proxy = self.proxy_url.is_some() && url.scheme() == "http";
let mut builder = http::Request::builder()
.uri(if self.is_http2() || use_proxy {
&url[..]
} else {
&url[url::Position::BeforePath..]
})
.method(self.method.clone())
.version(if use_proxy {
self.proxy_http_version
} else {
self.http_version
});
let bytes = self.body.map(Bytes::from_static);
let body = if let Some(body) = &bytes {
Full::new(body.clone())
} else {
Full::default()
};
let mut headers = self.headers.clone();
// Apply AWS SigV4 if configured
if let Some(aws_config) = &self.aws_config {
aws_config.sign_request(self.method.as_str(), &mut headers, url, bytes)?
}
if use_proxy {
for (key, value) in self.proxy_headers.iter() {
headers.insert(key, value.clone());
}
}
*builder
.headers_mut()
.ok_or(ClientError::GetHeaderFromBuilderError)? = headers;
let request = builder.body(body)?;
Ok(request)
}
async fn work_http1(
&self,
client_state: &mut ClientStateHttp1,
) -> Result<RequestResult, ClientError> {
let do_req = async {
let (url, rng) = self.generate_url(&mut client_state.rng)?;
let mut start = std::time::Instant::now();
let mut connection_time: Option<ConnectionTime> = None;
let mut send_request = if let Some(send_request) = client_state.send_request.take() {
send_request
} else {
let (dns_lookup, send_request) =
self.client_http1(&url, &mut client_state.rng).await?;
let dialup = std::time::Instant::now();
connection_time = Some(ConnectionTime { dns_lookup, dialup });
send_request
};
while send_request.ready().await.is_err() {
// This gets hit when the connection for HTTP/1.1 faults
// This re-connects
start = std::time::Instant::now();
let (dns_lookup, send_request_) =
self.client_http1(&url, &mut client_state.rng).await?;
send_request = send_request_;
let dialup = std::time::Instant::now();
connection_time = Some(ConnectionTime { dns_lookup, dialup });
}
let request = self.request(&url)?;
match send_request.send_request(request).await {
Ok(res) => {
let (parts, mut stream) = res.into_parts();
let mut status = parts.status;
let mut len_bytes = 0;
while let Some(chunk) = stream.frame().await {
len_bytes += chunk?.data_ref().map(|d| d.len()).unwrap_or_default();
}
if self.redirect_limit != 0 {
if let Some(location) = parts.headers.get("Location") {
let (send_request_redirect, new_status, len) = self
.redirect(
send_request,
&url,
location,
self.redirect_limit,
&mut client_state.rng,
)
.await?;
send_request = send_request_redirect;
status = new_status;
len_bytes = len;
}
}
let end = std::time::Instant::now();
let result = RequestResult {
rng,
start_latency_correction: None,
start,
end,
status,
len_bytes,
connection_time,
};
if !self.disable_keepalive {
client_state.send_request = Some(send_request);
}
Ok::<_, ClientError>(result)
}
Err(e) => {
client_state.send_request = Some(send_request);
Err(e.into())
}
}
};
if let Some(timeout) = self.timeout {
tokio::select! {
res = do_req => {
res
}
_ = tokio::time::sleep(timeout) => {
Err(ClientError::Timeout)
}
}
} else {
do_req.await
}
}
async fn connect_http2<R: Rng>(
&self,
url: &Url,
rng: &mut R,
) -> Result<(ConnectionTime, SendRequestHttp2), ClientError> {
if let Some(proxy_url) = &self.proxy_url {
let (dns_lookup, stream) = self.client(proxy_url, rng, self.is_proxy_http2()).await?;
if url.scheme() == "https" {
let req = {
let mut builder =
http::Request::builder()
.method(Method::CONNECT)
.uri(format!(
"{}:{}",
url.host_str().unwrap(),
url.port_or_known_default().unwrap()
));
*builder
.headers_mut()
.ok_or(ClientError::GetHeaderFromBuilderError)? =
self.proxy_headers.clone();
builder.body(http_body_util::Full::default())?
};
let res = if self.proxy_http_version == http::Version::HTTP_2 {
let mut send_request = stream.handshake_http2().await?;
send_request.send_request(req).await?
} else {
let mut send_request = stream.handshake_http1(true).await?;
send_request.send_request(req).await?
};
let stream = hyper::upgrade::on(res).await?;
let stream = self.connect_tls(TokioIo::new(stream), url, true).await?;
let (send_request, conn) =
hyper::client::conn::http2::Builder::new(TokioExecutor::new())
// from nghttp2's default
.initial_stream_window_size((1 << 30) - 1)
.initial_connection_window_size((1 << 30) - 1)
.handshake(TokioIo::new(stream))
.await?;
tokio::spawn(conn);
let dialup = std::time::Instant::now();
Ok((ConnectionTime { dns_lookup, dialup }, send_request))
} else {
let send_request = stream.handshake_http2().await?;
let dialup = std::time::Instant::now();
Ok((ConnectionTime { dns_lookup, dialup }, send_request))
}
} else {
let (dns_lookup, stream) = self.client(url, rng, true).await?;
let send_request = stream.handshake_http2().await?;
let dialup = std::time::Instant::now();
Ok((ConnectionTime { dns_lookup, dialup }, send_request))
}
}
async fn work_http2(
&self,
client_state: &mut ClientStateHttp2,
) -> Result<RequestResult, ClientError> {
let do_req = async {
let (url, rng) = self.generate_url(&mut client_state.rng)?;
let start = std::time::Instant::now();
let connection_time: Option<ConnectionTime> = None;
let request = self.request(&url)?;
match client_state.send_request.send_request(request).await {
Ok(res) => {
let (parts, mut stream) = res.into_parts();
let status = parts.status;
let mut len_bytes = 0;
while let Some(chunk) = stream.frame().await {
len_bytes += chunk?.data_ref().map(|d| d.len()).unwrap_or_default();
}
let end = std::time::Instant::now();
let result = RequestResult {
rng,
start_latency_correction: None,
start,
end,
status,
len_bytes,
connection_time,
};
Ok::<_, ClientError>(result)
}
Err(e) => Err(e.into()),
}
};
if let Some(timeout) = self.timeout {
tokio::select! {
res = do_req => {
res
}
_ = tokio::time::sleep(timeout) => {
Err(ClientError::Timeout)
}
}
} else {
do_req.await
}
}
#[allow(clippy::type_complexity)]
async fn redirect<R: Rng + Send>(
&self,
send_request: SendRequestHttp1,
base_url: &Url,
location: &http::header::HeaderValue,
limit: usize,
rng: &mut R,
) -> Result<(SendRequestHttp1, http::StatusCode, usize), ClientError> {
if limit == 0 {
return Err(ClientError::TooManyRedirect);
}
let url = match Url::parse(location.to_str()?) {
Ok(url) => url,
Err(ParseError::RelativeUrlWithoutBase) => Url::options()
.base_url(Some(base_url))
.parse(location.to_str()?)?,
Err(err) => Err(err)?,
};
let (mut send_request, send_request_base) =
if base_url.authority() == url.authority() && !self.disable_keepalive {
// reuse connection
(send_request, None)
} else {
let (_dns_lookup, stream) = self.client_http1(&url, rng).await?;
(stream, Some(send_request))
};
while send_request.ready().await.is_err() {
let (_dns_lookup, stream) = self.client_http1(&url, rng).await?;
send_request = stream;
}
let mut request = self.request(&url)?;
if url.authority() != base_url.authority() {
request.headers_mut().insert(
http::header::HOST,
http::HeaderValue::from_str(url.authority())?,
);
}
let res = send_request.send_request(request).await?;
let (parts, mut stream) = res.into_parts();
let mut status = parts.status;
let mut len_bytes = 0;
while let Some(chunk) = stream.frame().await {
len_bytes += chunk?.data_ref().map(|d| d.len()).unwrap_or_default();
}
if let Some(location) = parts.headers.get("Location") {
let (send_request_redirect, new_status, len) =
Box::pin(self.redirect(send_request, &url, location, limit - 1, rng)).await?;
send_request = send_request_redirect;
status = new_status;
len_bytes = len;
}
if let Some(send_request_base) = send_request_base {
Ok((send_request_base, status, len_bytes))
} else {
Ok((send_request, status, len_bytes))
}
}
}
/// Check error and decide whether to cancel the connection
fn is_cancel_error(res: &Result<RequestResult, ClientError>) -> bool {
matches!(res, Err(ClientError::Deadline)) || is_too_many_open_files(res)
}
/// Check error was "Too many open file"
fn is_too_many_open_files(res: &Result<RequestResult, ClientError>) -> bool {
res.as_ref()
.err()
.map(|err| match err {
ClientError::IoError(io_error) => io_error.raw_os_error() == Some(libc::EMFILE),
_ => false,
})
.unwrap_or(false)
}
/// Check error was any Hyper error (primarily for HTTP2 connection errors)
fn is_hyper_error(res: &Result<RequestResult, ClientError>) -> bool {
res.as_ref()
.err()
.map(|err| match err {
// REVIEW: IoErrors, if indicating the underlying connection has failed,
// should also cause a stop of HTTP2 requests
ClientError::IoError(_) => true,
ClientError::HyperError(_) => true,
_ => false,
})
.unwrap_or(false)
}
async fn setup_http2(client: &Client) -> Result<(ConnectionTime, SendRequestHttp2), ClientError> {
// Whatever rng state, all urls should have the same authority
let mut rng: Pcg64Si = SeedableRng::from_seed([0, 0, 0, 0, 0, 0, 0, 0]);
let url = client.url_generator.generate(&mut rng)?;
let (connection_time, send_request) = client.connect_http2(&url, &mut rng).await?;
Ok((connection_time, send_request))
}
async fn work_http2_once(
client: &Client,
client_state: &mut ClientStateHttp2,
report_tx: &flume::Sender<Result<RequestResult, ClientError>>,
connection_time: ConnectionTime,
start_latency_correction: Option<Instant>,
) -> (bool, bool) {
let mut res = client.work_http2(client_state).await;
let is_cancel = is_cancel_error(&res);
let is_reconnect = is_hyper_error(&res);
set_connection_time(&mut res, connection_time);
if let Some(start_latency_correction) = start_latency_correction {
set_start_latency_correction(&mut res, start_latency_correction);
}
report_tx.send(res).unwrap();
(is_cancel, is_reconnect)
}
fn set_connection_time<E>(res: &mut Result<RequestResult, E>, connection_time: ConnectionTime) {
if let Ok(res) = res {
res.connection_time = Some(connection_time);
}
}
fn set_start_latency_correction<E>(
res: &mut Result<RequestResult, E>,
start_latency_correction: std::time::Instant,
) {
if let Ok(res) = res {
res.start_latency_correction = Some(start_latency_correction);
}
}
pub async fn work_debug<W: Write>(w: &mut W, client: Arc<Client>) -> Result<(), ClientError> {
let mut rng = StdRng::from_os_rng();
let url = client.url_generator.generate(&mut rng)?;
writeln!(w, "URL: {}", url)?;
let request = client.request(&url)?;
writeln!(w, "{:#?}", request)?;
let response = if client.is_work_http2() {
let (_, mut client_state) = client.connect_http2(&url, &mut rng).await?;
client_state.send_request(request).await?
} else {
let (_dns_lookup, mut send_request) = client.client_http1(&url, &mut rng).await?;
send_request.send_request(request).await?
};
let (parts, body) = response.into_parts();
let body = body.collect().await.unwrap().to_bytes();
let response = http::Response::from_parts(parts, body);
writeln!(w, "{:#?}", response)?;
Ok(())
}
/// Run n tasks by m workers
pub async fn work(
client: Arc<Client>,
report_tx: flume::Sender<Result<RequestResult, ClientError>>,
n_tasks: usize,
n_connections: usize,
n_http2_parallel: usize,
) {
use std::sync::atomic::{AtomicUsize, Ordering};
let counter = Arc::new(AtomicUsize::new(0));
if client.is_work_http2() {
let futures = (0..n_connections)
.map(|_| {
let report_tx = report_tx.clone();
let counter = counter.clone();
let client = client.clone();
tokio::spawn(async move {
loop {
match setup_http2(&client).await {
Ok((connection_time, send_request)) => {
let futures = (0..n_http2_parallel)
.map(|_| {
let report_tx = report_tx.clone();
let counter = counter.clone();
let client = client.clone();
let mut client_state = ClientStateHttp2 {
rng: SeedableRng::from_os_rng(),
send_request: send_request.clone(),
};
tokio::spawn(async move {
while counter.fetch_add(1, Ordering::Relaxed) < n_tasks
{
let (is_cancel, is_reconnect) = work_http2_once(
&client,
&mut client_state,
&report_tx,
connection_time,
None,
)
.await;
if is_cancel || is_reconnect {
return is_cancel;
}
}
true
})
})
.collect::<Vec<_>>();
let mut connection_gone = false;
for f in futures {
match f.await {
Ok(true) => {
// All works done
connection_gone = true;
}
Err(_) => {
// Unexpected
connection_gone = true;
}
_ => {}
}
}
if connection_gone {
return;
}
}
Err(err) => {
if counter.fetch_add(1, Ordering::Relaxed) < n_tasks {
report_tx.send(Err(err)).unwrap();
} else {
return;
}
}
}
}
})
})
.collect::<Vec<_>>();
for f in futures {
let _ = f.await;
}
} else {
let futures = (0..n_connections)
.map(|_| {
let report_tx = report_tx.clone();
let counter = counter.clone();
let client = client.clone();
tokio::spawn(async move {
let mut client_state = ClientStateHttp1::default();
while counter.fetch_add(1, Ordering::Relaxed) < n_tasks {
let res = client.work_http1(&mut client_state).await;
let is_cancel = is_cancel_error(&res);
report_tx.send(res).unwrap();
if is_cancel {
break;
}
}
})
})
.collect::<Vec<_>>();
for f in futures {
let _ = f.await;
}
};
}
/// n tasks by m workers limit to qps works in a second
pub async fn work_with_qps(
client: Arc<Client>,
report_tx: flume::Sender<Result<RequestResult, ClientError>>,
query_limit: QueryLimit,
n_tasks: usize,
n_connections: usize,
n_http2_parallel: usize,
) {
let (tx, rx) = flume::unbounded();
let work_queue = async move {
match query_limit {
QueryLimit::Qps(qps) => {
let start = std::time::Instant::now();
for i in 0..n_tasks {
tokio::time::sleep_until(
(start + i as u32 * std::time::Duration::from_secs(1) / qps as u32).into(),
)
.await;
tx.send(())?;
}
}
QueryLimit::Burst(duration, rate) => {
let mut n = 0;
// Handle via rate till n_tasks out of bound
while n + rate < n_tasks {
tokio::time::sleep(duration).await;
for _ in 0..rate {
tx.send(())?;
}
n += rate;
}
// Handle the remaining tasks
if n_tasks > n {
tokio::time::sleep(duration).await;
for _ in 0..n_tasks - n {
tx.send(())?;
}
}
}
}
// tx gone
drop(tx);
Ok::<(), flume::SendError<_>>(())
};
if client.is_work_http2() {
let futures = (0..n_connections)
.map(|_| {
let report_tx = report_tx.clone();
let rx = rx.clone();
let client = client.clone();
tokio::spawn(async move {
loop {
match setup_http2(&client).await {
Ok((connection_time, send_request)) => {
let futures = (0..n_http2_parallel)
.map(|_| {
let report_tx = report_tx.clone();
let rx = rx.clone();
let client = client.clone();
let mut client_state = ClientStateHttp2 {
rng: SeedableRng::from_os_rng(),
send_request: send_request.clone(),
};
tokio::spawn(async move {
while let Ok(()) = rx.recv_async().await {
let (is_cancel, is_reconnect) = work_http2_once(
&client,
&mut client_state,
&report_tx,
connection_time,
None,
)
.await;
if is_cancel || is_reconnect {
return is_cancel;
}
}
true
})
})
.collect::<Vec<_>>();
let mut connection_gone = false;
for f in futures {
match f.await {
Ok(true) => {
// All works done
connection_gone = true;
}
Err(_) => {
// Unexpected
connection_gone = true;
}
_ => {}
}
}
if connection_gone {
return;
}
}
Err(err) => {
// Consume a task
if let Ok(()) = rx.recv_async().await {
report_tx.send(Err(err)).unwrap();
} else {
return;
}
}
}
}
})
})
.collect::<Vec<_>>();
work_queue.await.unwrap();
for f in futures {
let _ = f.await;
}
} else {
let futures = (0..n_connections)
.map(|_| {
let report_tx = report_tx.clone();
let rx = rx.clone();
let client = client.clone();
tokio::spawn(async move {
let mut client_state = ClientStateHttp1::default();
while let Ok(()) = rx.recv_async().await {
let res = client.work_http1(&mut client_state).await;
let is_cancel = is_cancel_error(&res);
report_tx.send(res).unwrap();
if is_cancel {
break;
}
}
})
})
.collect::<Vec<_>>();
work_queue.await.unwrap();
for f in futures {
let _ = f.await;
}
};
}
/// n tasks by m workers limit to qps works in a second with latency correction
pub async fn work_with_qps_latency_correction(
client: Arc<Client>,
report_tx: flume::Sender<Result<RequestResult, ClientError>>,
query_limit: QueryLimit,
n_tasks: usize,
n_connections: usize,
n_http2_parallel: usize,
) {
let (tx, rx) = flume::unbounded();
let work_queue = async move {
match query_limit {
QueryLimit::Qps(qps) => {
let start = std::time::Instant::now();
for i in 0..n_tasks {
tokio::time::sleep_until(
(start + i as u32 * std::time::Duration::from_secs(1) / qps as u32).into(),
)
.await;
tx.send(std::time::Instant::now())?;
}
}
QueryLimit::Burst(duration, rate) => {
let mut n = 0;
// Handle via rate till n_tasks out of bound
while n + rate < n_tasks {
tokio::time::sleep(duration).await;
let now = std::time::Instant::now();
for _ in 0..rate {
tx.send(now)?;
}
n += rate;
}
// Handle the remaining tasks
if n_tasks > n {
tokio::time::sleep(duration).await;
let now = std::time::Instant::now();
for _ in 0..n_tasks - n {
tx.send(now)?;
}
}
}
}
// tx gone
drop(tx);
Ok::<(), flume::SendError<_>>(())
};
if client.is_work_http2() {
let futures = (0..n_connections)
.map(|_| {
let report_tx = report_tx.clone();
let rx = rx.clone();
let client = client.clone();
tokio::spawn(async move {
loop {
match setup_http2(&client).await {
Ok((connection_time, send_request)) => {
let futures = (0..n_http2_parallel)
.map(|_| {
let report_tx = report_tx.clone();
let rx = rx.clone();
let client = client.clone();
let mut client_state = ClientStateHttp2 {
rng: SeedableRng::from_os_rng(),
send_request: send_request.clone(),
};
tokio::spawn(async move {
while let Ok(start) = rx.recv_async().await {
let (is_cancel, is_reconnect) = work_http2_once(
&client,
&mut client_state,
&report_tx,
connection_time,
Some(start),
)
.await;
if is_cancel || is_reconnect {
return is_cancel;
}
}
true
})
})
.collect::<Vec<_>>();
let mut connection_gone = false;
for f in futures {
match f.await {
Ok(true) => {
// All works done
connection_gone = true;
}
Err(_) => {
// Unexpected
connection_gone = true;
}
_ => {}
}
}
if connection_gone {
return;
}
}
Err(err) => {
// Consume a task
if rx.recv_async().await.is_ok() {
report_tx.send(Err(err)).unwrap();
} else {
return;
}
}
}
}
})
})
.collect::<Vec<_>>();
work_queue.await.unwrap();
for f in futures {
let _ = f.await;
}
} else {
let futures = (0..n_connections)
.map(|_| {
let client = client.clone();
let mut client_state = ClientStateHttp1::default();
let report_tx = report_tx.clone();
let rx = rx.clone();
tokio::spawn(async move {
while let Ok(start) = rx.recv_async().await {
let mut res = client.work_http1(&mut client_state).await;
set_start_latency_correction(&mut res, start);
let is_cancel = is_cancel_error(&res);
report_tx.send(res).unwrap();
if is_cancel {
break;
}
}
})
})
.collect::<Vec<_>>();
work_queue.await.unwrap();
for f in futures {
let _ = f.await;
}
};
}
/// Run until dead_line by n workers
pub async fn work_until(
client: Arc<Client>,
report_tx: flume::Sender<Result<RequestResult, ClientError>>,
dead_line: std::time::Instant,
n_connections: usize,
n_http2_parallel: usize,
wait_ongoing_requests_after_deadline: bool,
) {
if client.is_work_http2() {
// Using semaphore to control the deadline
// Maybe there is a better concurrent primitive to do this
let s = Arc::new(tokio::sync::Semaphore::new(0));
let futures = (0..n_connections)
.map(|_| {
let client = client.clone();
let report_tx = report_tx.clone();
let s = s.clone();
tokio::spawn(async move {
let s = s.clone();
// Keep trying to establish or re-establish connections up to the deadline
loop {
match setup_http2(&client).await {
Ok((connection_time, send_request)) => {
// Setup the parallel workers for each HTTP2 connection
let futures = (0..n_http2_parallel)
.map(|_| {
let client = client.clone();
let report_tx = report_tx.clone();
let mut client_state = ClientStateHttp2 {
rng: SeedableRng::from_os_rng(),
send_request: send_request.clone(),
};
let s = s.clone();
tokio::spawn(async move {
// This is where HTTP2 loops to make all the requests for a given client and worker
loop {
let (is_cancel, is_reconnect) = work_http2_once(
&client,
&mut client_state,
&report_tx,
connection_time,
None,
)
.await;
let is_cancel = is_cancel || s.is_closed();
if is_cancel || is_reconnect {
break is_cancel;
}
}
})
})
.collect::<Vec<_>>();
let mut connection_gone = false;
for f in futures {
tokio::select! {
r = f => {
match r {
Ok(true) => {
// All works done
connection_gone = true;
}
Err(_) => {
// Unexpected
connection_gone = true;
}
_ => {}
}
}
_ = s.acquire() => {
report_tx.send(Err(ClientError::Deadline)).unwrap();
connection_gone = true;
}
}
}
if connection_gone {
return;
}
}
Err(err) => {
report_tx.send(Err(err)).unwrap();
if s.is_closed() {
break;
}
}
}
}
})
})
.collect::<Vec<_>>();
tokio::time::sleep_until(dead_line.into()).await;
s.close();
for f in futures {
let _ = f.await;
}
} else {
let is_end = Arc::new(AtomicBool::new(false));
let futures = (0..n_connections)
.map(|_| {
let client = client.clone();
let report_tx = report_tx.clone();
let mut client_state = ClientStateHttp1::default();
let is_end = is_end.clone();
tokio::spawn(async move {
loop {
let res = client.work_http1(&mut client_state).await;
let is_cancel = is_cancel_error(&res);
report_tx.send(res).unwrap();
if is_cancel || is_end.load(Relaxed) {
break;
}
}
})
})
.collect::<Vec<_>>();
tokio::time::sleep_until(dead_line.into()).await;
is_end.store(true, Relaxed);
if wait_ongoing_requests_after_deadline {
for f in futures {
let _ = f.await;
}
} else {
for f in futures {
f.abort();
if let Err(e) = f.await {
if e.is_cancelled() {
report_tx.send(Err(ClientError::Deadline)).unwrap();
}
}
}
}
};
}
/// Run until dead_line by n workers limit to qps works in a second
#[allow(clippy::too_many_arguments)]
pub async fn work_until_with_qps(
client: Arc<Client>,
report_tx: flume::Sender<Result<RequestResult, ClientError>>,
query_limit: QueryLimit,
start: std::time::Instant,
dead_line: std::time::Instant,
n_connections: usize,
n_http2_parallel: usize,
wait_ongoing_requests_after_deadline: bool,
) {
let rx = match query_limit {
QueryLimit::Qps(qps) => {
let (tx, rx) = flume::unbounded();
tokio::spawn(async move {
for i in 0.. {
if std::time::Instant::now() > dead_line {
break;
}
tokio::time::sleep_until(
(start + i as u32 * std::time::Duration::from_secs(1) / qps as u32).into(),
)
.await;
let _ = tx.send(());
}
// tx gone
});
rx
}
QueryLimit::Burst(duration, rate) => {
let (tx, rx) = flume::unbounded();
tokio::spawn(async move {
// Handle via rate till deadline is reached
for _ in 0.. {
if std::time::Instant::now() > dead_line {
break;
}
tokio::time::sleep(duration).await;
for _ in 0..rate {
let _ = tx.send(());
}
}
// tx gone
});
rx
}
};
if client.is_work_http2() {
let s = Arc::new(tokio::sync::Semaphore::new(0));
let futures = (0..n_connections)
.map(|_| {
let client = client.clone();
let report_tx = report_tx.clone();
let rx = rx.clone();
let s = s.clone();
tokio::spawn(async move {
loop {
match setup_http2(&client).await {
Ok((connection_time, send_request)) => {
let futures = (0..n_http2_parallel)
.map(|_| {
let client = client.clone();
let report_tx = report_tx.clone();
let rx = rx.clone();
let mut client_state = ClientStateHttp2 {
rng: SeedableRng::from_os_rng(),
send_request: send_request.clone(),
};
let s = s.clone();
tokio::spawn(async move {
while let Ok(()) = rx.recv_async().await {
let (is_cancel, is_reconnect) = work_http2_once(
&client,
&mut client_state,
&report_tx,
connection_time,
None,
)
.await;
let is_cancel = is_cancel || s.is_closed();
if is_cancel || is_reconnect {
return is_cancel;
}
}
true
})
})
.collect::<Vec<_>>();
let mut connection_gone = false;
for f in futures {
tokio::select! {
r = f => {
match r {
Ok(true) => {
// All works done
connection_gone = true;
}
Err(_) => {
// Unexpected
connection_gone = true;
}
_ => {}
}
}
_ = s.acquire() => {
report_tx.send(Err(ClientError::Deadline)).unwrap();
connection_gone = true;
}
}
}
if connection_gone {
return;
}
}
Err(err) => {
// Consume a task
if rx.recv_async().await.is_ok() {
report_tx.send(Err(err)).unwrap();
} else {
return;
}
if s.is_closed() {
return;
}
}
}
}
})
})
.collect::<Vec<_>>();
tokio::time::sleep_until(dead_line.into()).await;
s.close();
for f in futures {
let _ = f.await;
}
} else {
let is_end = Arc::new(AtomicBool::new(false));
let futures = (0..n_connections)
.map(|_| {
let client = client.clone();
let mut client_state = ClientStateHttp1::default();
let report_tx = report_tx.clone();
let rx = rx.clone();
let is_end = is_end.clone();
tokio::spawn(async move {
while let Ok(()) = rx.recv_async().await {
let res = client.work_http1(&mut client_state).await;
let is_cancel = is_cancel_error(&res);
report_tx.send(res).unwrap();
if is_cancel || is_end.load(Relaxed) {
break;
}
}
})
})
.collect::<Vec<_>>();
tokio::time::sleep_until(dead_line.into()).await;
is_end.store(true, Relaxed);
if wait_ongoing_requests_after_deadline {
for f in futures {
let _ = f.await;
}
} else {
for f in futures {
f.abort();
if let Err(e) = f.await {
if e.is_cancelled() {
report_tx.send(Err(ClientError::Deadline)).unwrap();
}
}
}
}
}
}
/// Run until dead_line by n workers limit to qps works in a second with latency correction
#[allow(clippy::too_many_arguments)]
pub async fn work_until_with_qps_latency_correction(
client: Arc<Client>,
report_tx: flume::Sender<Result<RequestResult, ClientError>>,
query_limit: QueryLimit,
start: std::time::Instant,
dead_line: std::time::Instant,
n_connections: usize,
n_http2_parallel: usize,
wait_ongoing_requests_after_deadline: bool,
) {
let (tx, rx) = flume::unbounded();
match query_limit {
QueryLimit::Qps(qps) => {
tokio::spawn(async move {
for i in 0.. {
tokio::time::sleep_until(
(start + i as u32 * std::time::Duration::from_secs(1) / qps as u32).into(),
)
.await;
let now = std::time::Instant::now();
if now > dead_line {
break;
}
let _ = tx.send(now);
}
// tx gone
});
}
QueryLimit::Burst(duration, rate) => {
tokio::spawn(async move {
// Handle via rate till deadline is reached
loop {
tokio::time::sleep(duration).await;
let now = std::time::Instant::now();
if now > dead_line {
break;
}
for _ in 0..rate {
let _ = tx.send(now);
}
}
// tx gone
});
}
};
if client.is_work_http2() {
let s = Arc::new(tokio::sync::Semaphore::new(0));
let futures = (0..n_connections)
.map(|_| {
let client = client.clone();
let report_tx = report_tx.clone();
let rx = rx.clone();
let s = s.clone();
tokio::spawn(async move {
loop {
match setup_http2(&client).await {
Ok((connection_time, send_request)) => {
let futures = (0..n_http2_parallel)
.map(|_| {
let client = client.clone();
let report_tx = report_tx.clone();
let rx = rx.clone();
let mut client_state = ClientStateHttp2 {
rng: SeedableRng::from_os_rng(),
send_request: send_request.clone(),
};
let s = s.clone();
tokio::spawn(async move {
while let Ok(start) = rx.recv_async().await {
let (is_cancel, is_reconnect) = work_http2_once(
&client,
&mut client_state,
&report_tx,
connection_time,
Some(start),
)
.await;
let is_cancel = is_cancel || s.is_closed();
if is_cancel || is_reconnect {
return is_cancel;
}
}
true
})
})
.collect::<Vec<_>>();
let mut connection_gone = false;
for f in futures {
tokio::select! {
r = f => {
match r {
Ok(true) => {
// All works done
connection_gone = true;
}
Err(_) => {
// Unexpected
connection_gone = true;
}
_ => {}
}
}
_ = s.acquire() => {
report_tx.send(Err(ClientError::Deadline)).unwrap();
connection_gone = true;
}
}
}
if connection_gone {
return;
}
}
Err(err) => {
if rx.recv_async().await.is_ok() {
report_tx.send(Err(err)).unwrap();
} else {
return;
}
if s.is_closed() {
return;
}
}
}
}
})
})
.collect::<Vec<_>>();
tokio::time::sleep_until(dead_line.into()).await;
s.close();
for f in futures {
let _ = f.await;
}
} else {
let is_end = Arc::new(AtomicBool::new(false));
let futures = (0..n_connections)
.map(|_| {
let client = client.clone();
let mut client_state = ClientStateHttp1::default();
let report_tx = report_tx.clone();
let rx = rx.clone();
let is_end = is_end.clone();
tokio::spawn(async move {
while let Ok(start) = rx.recv_async().await {
let mut res = client.work_http1(&mut client_state).await;
set_start_latency_correction(&mut res, start);
let is_cancel = is_cancel_error(&res);
report_tx.send(res).unwrap();
if is_cancel || is_end.load(Relaxed) {
break;
}
}
})
})
.collect::<Vec<_>>();
tokio::time::sleep_until(dead_line.into()).await;
is_end.store(true, Relaxed);
if wait_ongoing_requests_after_deadline {
for f in futures {
let _ = f.await;
}
} else {
for f in futures {
f.abort();
if let Err(e) = f.await {
if e.is_cancelled() {
report_tx.send(Err(ClientError::Deadline)).unwrap();
}
}
}
}
}
}
/// Optimized workers for `--no-tui` mode
pub mod fast {
use std::sync::Arc;
use rand::SeedableRng;
use crate::{
client::{
ClientError, ClientStateHttp1, ClientStateHttp2, is_cancel_error, is_hyper_error,
set_connection_time, setup_http2,
},
result_data::ResultData,
};
use super::Client;
/// Run n tasks by m workers
pub async fn work(
client: Arc<Client>,
report_tx: flume::Sender<ResultData>,
n_tasks: usize,
n_connections: usize,
n_http2_parallel: usize,
) {
use std::sync::atomic::{AtomicUsize, Ordering};
let counter = Arc::new(AtomicUsize::new(0));
let num_threads = num_cpus::get_physical();
let connections = (0..num_threads).filter_map(|i| {
let num_connection = n_connections / num_threads
+ (if (n_connections % num_threads) > i {
1
} else {
0
});
if num_connection > 0 {
Some(num_connection)
} else {
None
}
});
let token = tokio_util::sync::CancellationToken::new();
let handles = if client.is_work_http2() {
connections
.map(|num_connections| {
let report_tx = report_tx.clone();
let counter = counter.clone();
let client = client.clone();
let rt = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.unwrap();
let token = token.clone();
std::thread::spawn(move || {
let client = client.clone();
let local = tokio::task::LocalSet::new();
for _ in 0..num_connections {
let report_tx = report_tx.clone();
let counter = counter.clone();
let client = client.clone();
let token = token.clone();
local.spawn_local(Box::pin(async move {
let mut has_err = false;
let mut result_data_err = ResultData::default();
loop {
let client = client.clone();
match setup_http2(&client).await {
Ok((connection_time, send_request)) => {
let futures = (0..n_http2_parallel)
.map(|_| {
let mut client_state = ClientStateHttp2 {
rng: SeedableRng::from_os_rng(),
send_request: send_request.clone(),
};
let counter = counter.clone();
let client = client.clone();
let report_tx = report_tx.clone();
let token = token.clone();
tokio::task::spawn_local(async move {
let mut result_data = ResultData::default();
let work = async {
while counter
.fetch_add(1, Ordering::Relaxed)
< n_tasks
{
let mut res = client
.work_http2(&mut client_state)
.await;
let is_cancel =
is_cancel_error(&res);
let is_reconnect =
is_hyper_error(&res);
set_connection_time(
&mut res,
connection_time,
);
result_data.push(res);
if is_cancel || is_reconnect {
return is_cancel;
}
}
true
};
let is_cancel = tokio::select! {
is_cancel = work => {
is_cancel
}
_ = token.cancelled() => {
true
}
};
report_tx.send(result_data).unwrap();
is_cancel
})
})
.collect::<Vec<_>>();
let mut connection_gone = false;
for f in futures {
match f.await {
Ok(true) => {
// All works done
connection_gone = true;
}
Err(_) => {
// Unexpected
connection_gone = true;
}
_ => {}
}
}
if connection_gone {
break;
}
}
Err(err) => {
if counter.fetch_add(1, Ordering::Relaxed) < n_tasks {
has_err = true;
result_data_err.push(Err(err));
} else {
break;
}
}
}
}
if has_err {
report_tx.send(result_data_err).unwrap();
}
}));
}
rt.block_on(local);
})
})
.collect::<Vec<_>>()
} else {
connections
.map(|num_connection| {
let report_tx = report_tx.clone();
let counter = counter.clone();
let client = client.clone();
let rt = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.unwrap();
let token = token.clone();
std::thread::spawn(move || {
let local = tokio::task::LocalSet::new();
for _ in 0..num_connection {
let report_tx = report_tx.clone();
let counter = counter.clone();
let client = client.clone();
let token = token.clone();
local.spawn_local(Box::pin(async move {
let mut result_data = ResultData::default();
tokio::select! {
_ = token.cancelled() => {}
_ = async {
let mut client_state = ClientStateHttp1::default();
while counter.fetch_add(1, Ordering::Relaxed) < n_tasks {
let res = client.work_http1(&mut client_state).await;
let is_cancel = is_cancel_error(&res);
result_data.push(res);
if is_cancel {
break;
}
}
} => {}
}
report_tx.send(result_data).unwrap();
}));
}
rt.block_on(local);
})
})
.collect::<Vec<_>>()
};
tokio::spawn(async move {
tokio::signal::ctrl_c().await.unwrap();
token.cancel();
});
tokio::task::block_in_place(|| {
for handle in handles {
let _ = handle.join();
}
});
}
/// Run until dead_line by n workers
pub async fn work_until(
client: Arc<Client>,
report_tx: flume::Sender<ResultData>,
dead_line: std::time::Instant,
n_connections: usize,
n_http2_parallel: usize,
wait_ongoing_requests_after_deadline: bool,
) {
use std::sync::atomic::{AtomicBool, Ordering};
let num_threads = num_cpus::get_physical();
let is_end = Arc::new(AtomicBool::new(false));
let connections = (0..num_threads).filter_map(|i| {
let num_connection = n_connections / num_threads
+ (if (n_connections % num_threads) > i {
1
} else {
0
});
if num_connection > 0 {
Some(num_connection)
} else {
None
}
});
let token = tokio_util::sync::CancellationToken::new();
let handles = if client.is_work_http2() {
connections
.map(|num_connections| {
let report_tx = report_tx.clone();
let client = client.clone();
let rt = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.unwrap();
let token = token.clone();
let is_end = is_end.clone();
std::thread::spawn(move || {
let client = client.clone();
let local = tokio::task::LocalSet::new();
for _ in 0..num_connections {
let report_tx = report_tx.clone();
let client = client.clone();
let token = token.clone();
let is_end = is_end.clone();
local.spawn_local(Box::pin(async move {
let mut has_err = false;
let mut result_data_err = ResultData::default();
loop {
let client = client.clone();
match setup_http2(&client).await {
Ok((connection_time, send_request)) => {
let futures = (0..n_http2_parallel)
.map(|_| {
let mut client_state = ClientStateHttp2 {
rng: SeedableRng::from_os_rng(),
send_request: send_request.clone(),
};
let client = client.clone();
let report_tx = report_tx.clone();
let token = token.clone();
let is_end = is_end.clone();
tokio::task::spawn_local(async move {
let mut result_data = ResultData::default();
let work = async {
loop {
let mut res = client
.work_http2(&mut client_state)
.await;
let is_cancel = is_cancel_error(&res) || is_end.load(Ordering::Relaxed);
let is_reconnect = is_hyper_error(&res);
set_connection_time(
&mut res,
connection_time,
);
result_data.push(res);
if is_cancel || is_reconnect {
return is_cancel;
}
}
};
let is_cancel = tokio::select! {
is_cancel = work => {
is_cancel
}
_ = token.cancelled() => {
result_data.push(Err(ClientError::Deadline));
true
}
};
report_tx.send(result_data).unwrap();
is_cancel
})
})
.collect::<Vec<_>>();
let mut connection_gone = false;
for f in futures {
match f.await {
Ok(true) => {
// All works done
connection_gone = true;
}
Err(_) => {
// Unexpected
connection_gone = true;
}
_ => {}
}
}
if connection_gone {
break;
}
}
Err(err) => {
has_err = true;
result_data_err.push(Err(err));
if is_end.load(Ordering::Relaxed) {
break;
}
}
}
}
if has_err {
report_tx.send(result_data_err).unwrap();
}
}));
}
rt.block_on(local);
})
})
.collect::<Vec<_>>()
} else {
connections
.map(|num_connection| {
let report_tx = report_tx.clone();
let is_end = is_end.clone();
let client = client.clone();
let rt = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.unwrap();
let token = token.clone();
std::thread::spawn(move || {
let local = tokio::task::LocalSet::new();
for _ in 0..num_connection {
let report_tx = report_tx.clone();
let is_end = is_end.clone();
let client = client.clone();
let token = token.clone();
local.spawn_local(Box::pin(async move {
let mut result_data = ResultData::default();
let work = async {
let mut client_state = ClientStateHttp1::default();
loop {
let res = client.work_http1(&mut client_state).await;
let is_cancel = is_cancel_error(&res);
result_data.push(res);
if is_cancel || is_end.load(Ordering::Relaxed) {
break;
}
}
};
tokio::select! {
_ = work => {
}
_ = token.cancelled() => {
result_data.push(Err(ClientError::Deadline));
}
}
report_tx.send(result_data).unwrap();
}));
}
rt.block_on(local);
})
})
.collect::<Vec<_>>()
};
tokio::select! {
_ = tokio::time::sleep_until(dead_line.into()) => {
}
_ = tokio::signal::ctrl_c() => {
}
}
is_end.store(true, Ordering::Relaxed);
if !wait_ongoing_requests_after_deadline {
token.cancel();
}
tokio::task::block_in_place(|| {
for handle in handles {
let _ = handle.join();
}
});
}
}
================================================
FILE: src/db.rs
================================================
use rusqlite::Connection;
use crate::client::{Client, RequestResult};
fn create_db(conn: &Connection) -> Result<usize, rusqlite::Error> {
conn.execute(
"CREATE TABLE spiko (
url TEXT NOT NULL,
start REAL NOT NULL,
start_latency_correction REAL,
end REAL NOT NULL,
duration REAL NOT NULL,
status INTEGER NOT NULL,
len_bytes INTEGER NOT NULL
)",
(),
)
}
pub fn store(
client: &Client,
db_url: &str,
start: std::time::Instant,
request_records: &[RequestResult],
) -> Result<usize, rusqlite::Error> {
let mut conn = Connection::open(db_url)?;
create_db(&conn)?;
let t = conn.transaction()?;
let mut affected_rows = 0;
for request in request_records {
let url = client.generate_url(&mut request.rng.clone()).unwrap().0;
affected_rows += t.execute(
"INSERT INTO spiko (url, start, start_latency_correction, end, duration, status, len_bytes) VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7)",
(
url.to_string(),
(request.start - start).as_secs_f64(),
request.start_latency_correction.map(|d| (d - start).as_secs_f64()),
(request.end - start).as_secs_f64(),
request.duration().as_secs_f64(),
request.status.as_u16() as i64,
request.len_bytes,
),
)?;
}
t.commit()?;
Ok(affected_rows)
}
#[cfg(test)]
mod test_db {
use rand::SeedableRng;
use super::*;
#[test]
fn test_store() {
let start = std::time::Instant::now();
let test_val = RequestResult {
rng: SeedableRng::seed_from_u64(0),
status: hyper::StatusCode::OK,
len_bytes: 100,
start_latency_correction: None,
start: std::time::Instant::now(),
connection_time: None,
end: std::time::Instant::now(),
};
let test_vec = vec![test_val.clone(), test_val.clone()];
let client = Client::default();
let result = store(&client, ":memory:", start, &test_vec);
assert_eq!(result.unwrap(), 2);
}
}
================================================
FILE: src/histogram.rs
================================================
pub fn histogram(values: &[f64], bins: usize) -> Vec<(f64, usize)> {
assert!(bins >= 2);
let mut bucket: Vec<usize> = vec![0; bins];
let min = values.iter().collect::<average::Min>().min();
let max = values.iter().collect::<average::Max>().max();
let step = (max - min) / (bins - 1) as f64;
for &v in values {
let i = std::cmp::min(((v - min) / step).ceil() as usize, bins - 1);
bucket[i] += 1;
}
bucket
.into_iter()
.enumerate()
.map(|(i, v)| (min + step * i as f64, v))
.collect()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_histogram() {
let values1: [f64; 10] = [1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0];
assert_eq!(
histogram(&values1, 10),
vec![
(1.0, 1),
(2.0, 1),
(3.0, 1),
(4.0, 1),
(5.0, 1),
(6.0, 1),
(7.0, 1),
(8.0, 1),
(9.0, 1),
(10.0, 1)
]
);
assert_eq!(
histogram(&values1, 4),
vec![(1.0, 1), (4.0, 3), (7.0, 3), (10.0, 3)]
);
assert_eq!(
histogram(&values1, 17),
vec![
(1.0, 1),
(1.5625, 0),
(2.125, 1),
(2.6875, 0),
(3.25, 1),
(3.8125, 0),
(4.375, 1),
(4.9375, 0),
(5.5, 1),
(6.0625, 1),
(6.625, 0),
(7.1875, 1),
(7.75, 0),
(8.3125, 1),
(8.875, 0),
(9.4375, 1),
(10.0, 1)
]
);
let values2: [f64; 10] = [1.0, 1.0, 1.0, 1.0, 1.0, 10.0, 10.0, 10.0, 10.0, 10.0];
assert_eq!(
histogram(&values2, 10),
vec![
(1.0, 5),
(2.0, 0),
(3.0, 0),
(4.0, 0),
(5.0, 0),
(6.0, 0),
(7.0, 0),
(8.0, 0),
(9.0, 0),
(10.0, 5)
]
);
assert_eq!(histogram(&values2, 2), vec![(1.0, 5), (10.0, 5)]);
}
}
================================================
FILE: src/lib.rs
================================================
use anyhow::Context;
use aws_auth::AwsSignatureConfig;
use clap::Parser;
use crossterm::tty::IsTty;
use hickory_resolver::config::{ResolverConfig, ResolverOpts};
use humantime::Duration;
use hyper::{
http::{
self,
header::{HeaderName, HeaderValue},
},
HeaderMap,
};
use printer::{PrintConfig, PrintMode};
use rand_regex::Regex;
use ratatui::crossterm;
use result_data::ResultData;
use std::{
env,
fs::File,
io::{BufRead, Read},
path::{Path, PathBuf},
pin::Pin,
str::FromStr,
sync::Arc,
};
use url::Url;
use url_generator::UrlGenerator;
mod aws_auth;
mod client;
mod db;
mod histogram;
mod monitor;
mod pcg64si;
mod printer;
mod result_data;
mod timescale;
mod tls_config;
mod url_generator;
#[cfg(not(target_env = "msvc"))]
use tikv_jemallocator::Jemalloc;
#[cfg(not(target_env = "msvc"))]
#[global_allocator]
static GLOBAL: Jemalloc = Jemalloc;
#[derive(Parser)]
#[command(version, about, long_about = None)]
#[command(arg_required_else_help(true))]
pub struct Opts {
#[arg(help = "Target URL or file with multiple URLs.")]
url: String,
#[arg(
help = "Number of requests to run.",
short = 'n',
default_value = "200"
)]
n_requests: usize,
#[arg(
help = "Number of connections to run concurrently. You may should increase limit to number of open files for larger `-c`.",
short = 'c',
default_value = "50"
)]
n_connections: usize,
#[arg(
help = "Number of parallel requests to send on HTTP/2. `spiko` will run c * p concurrent workers in total.",
short = 'p',
default_value = "1"
)]
n_http2_parallel: usize,
#[arg(
help = "Duration of application to send requests. If duration is specified, n is ignored.
On HTTP/1, When the duration is reached, ongoing requests are aborted and counted as \"aborted due to deadline\"
You can change this behavior with `-w` option.
Currently, on HTTP/2, When the duration is reached, ongoing requests are waited. `-w` option is ignored.
Examples: -z 10s -z 3m.",
short = 'z'
)]
duration: Option<Duration>,
#[arg(
help = "When the duration is reached, ongoing requests are waited",
short,
long,
default_value = "false"
)]
wait_ongoing_requests_after_deadline: bool,
#[arg(help = "Rate limit for all, in queries per second (QPS)", short = 'q')]
query_per_second: Option<usize>,
#[arg(
help = "Introduce delay between a predefined number of requests.
Note: If qps is specified, burst will be ignored",
long = "burst-delay"
)]
burst_duration: Option<Duration>,
#[arg(
help = "Rates of requests for burst. Default is 1
Note: If qps is specified, burst will be ignored",
long = "burst-rate"
)]
burst_requests: Option<usize>,
#[arg(
help = "Generate URL by rand_regex crate but dot is disabled for each query e.g. http://127.0.0.1/[a-z][a-z][0-9]. Currently dynamic scheme, host and port with keep-alive do not work well. See https://docs.rs/rand_regex/latest/rand_regex/struct.Regex.html for details of syntax.",
default_value = "false",
long
)]
rand_regex_url: bool,
#[arg(
help = "Read the URLs to query from a file",
default_value = "false",
long
)]
urls_from_file: bool,
#[arg(
help = "A parameter for the '--rand-regex-url'. The max_repeat parameter gives the maximum extra repeat counts the x*, x+ and x{n,} operators will become.",
default_value = "4",
long
)]
max_repeat: u32,
#[arg(
help = "Dump target Urls <DUMP_URLS> times to debug --rand-regex-url",
long
)]
dump_urls: Option<usize>,
#[arg(
help = "Correct latency to avoid coordinated omission problem. It's ignored if -q is not set.",
long = "latency-correction"
)]
latency_correction: bool,
#[arg(help = "No realtime tui", long = "no-tui")]
no_tui: bool,
#[arg(help = "Print results as JSON", short, long)]
json: bool,
#[arg(help = "Frame per second for tui.", default_value = "16", long = "fps")]
fps: usize,
#[arg(
help = "HTTP method",
short = 'm',
long = "method",
default_value = "GET"
)]
method: http::Method,
#[arg(help = "Custom HTTP header. Examples: -H \"foo: bar\"", short = 'H')]
headers: Vec<String>,
#[arg(
help = "Custom Proxy HTTP header. Examples: --proxy-header \"foo: bar\"",
long = "proxy-header"
)]
proxy_headers: Vec<String>,
#[arg(help = "Timeout for each request. Default to infinite.", short = 't')]
timeout: Option<humantime::Duration>,
#[arg(help = "HTTP Accept Header.", short = 'A')]
accept_header: Option<String>,
#[arg(help = "HTTP request body.", short = 'd')]
body_string: Option<String>,
#[arg(help = "HTTP request body from file.", short = 'D')]
body_path: Option<std::path::PathBuf>,
#[arg(help = "Content-Type.", short = 'T')]
content_type: Option<String>,
#[arg(
help = "Basic authentication (username:password), or AWS credentials (access_key:secret_key)",
short = 'a'
)]
basic_auth: Option<String>,
#[arg(help = "AWS session token", long = "aws-session")]
aws_session: Option<String>,
#[arg(
help = "AWS SigV4 signing params (format: aws:amz:region:service)",
long = "aws-sigv4"
)]
aws_sigv4: Option<String>,
#[arg(help = "HTTP proxy", short = 'x')]
proxy: Option<Url>,
#[arg(
help = "HTTP version to connect to proxy. Available values 0.9, 1.0, 1.1, 2.",
long = "proxy-http-version"
)]
proxy_http_version: Option<String>,
#[arg(
help = "Use HTTP/2 to connect to proxy. Shorthand for --proxy-http-version=2",
long = "proxy-http2"
)]
proxy_http2: bool,
#[arg(
help = "HTTP version. Available values 0.9, 1.0, 1.1, 2.",
long = "http-version"
)]
http_version: Option<String>,
#[arg(help = "Use HTTP/2. Shorthand for --http-version=2", long = "http2")]
http2: bool,
#[arg(help = "HTTP Host header", long = "host")]
host: Option<String>,
#[arg(help = "Disable compression.", long = "disable-compression")]
disable_compression: bool,
#[arg(
help = "Limit for number of Redirect. Set 0 for no redirection. Redirection isn't supported for HTTP/2.",
default_value = "10",
short = 'r',
long = "redirect"
)]
redirect: usize,
#[arg(
help = "Disable keep-alive, prevents re-use of TCP connections between different HTTP requests. This isn't supported for HTTP/2.",
long = "disable-keepalive"
)]
disable_keepalive: bool,
#[arg(
help = "*Not* perform a DNS lookup at beginning to cache it",
long = "no-pre-lookup",
default_value = "false"
)]
no_pre_lookup: bool,
#[arg(help = "Lookup only ipv6.", long = "ipv6")]
ipv6: bool,
#[arg(help = "Lookup only ipv4.", long = "ipv4")]
ipv4: bool,
#[arg(
help = "(TLS) Use the specified certificate file to verify the peer. Native certificate store is used even if this argument is specified.",
long
)]
cacert: Option<PathBuf>,
#[arg(
help = "(TLS) Use the specified client certificate file. --key must be also specified",
long
)]
cert: Option<PathBuf>,
#[arg(
help = "(TLS) Use the specified client key file. --cert must be also specified",
long
)]
key: Option<PathBuf>,
#[arg(help = "Accept invalid certs.", long = "insecure")]
insecure: bool,
#[arg(
help = "Override DNS resolution and default port numbers with strings like 'example.org:443:localhost:8443'
Note: if used several times for the same host:port:target_host:target_port, a random choice is made",
long = "connect-to"
)]
connect_to: Vec<ConnectToEntry>,
#[arg(help = "Disable the color scheme.", long = "disable-color")]
disable_color: bool,
#[cfg(unix)]
#[arg(
help = "Connect to a unix socket instead of the domain in the URL. Only for non-HTTPS URLs.",
long = "unix-socket",
group = "socket-type"
)]
unix_socket: Option<std::path::PathBuf>,
#[cfg(feature = "vsock")]
#[arg(
help = "Connect to a VSOCK socket using 'cid:port' instead of the domain in the URL. Only for non-HTTPS URLs.",
long = "vsock-addr",
group = "socket-type"
)]
vsock_addr: Option<VsockAddr>,
#[arg(
help = "Include a response status code successful or not successful breakdown for the time histogram and distribution statistics",
long = "stats-success-breakdown"
)]
stats_success_breakdown: bool,
#[arg(
help = "Write succeeded requests to sqlite database url E.G test.db",
long = "db-url"
)]
db_url: Option<String>,
#[arg(
long,
help = "Perform a single request and dump the request and response"
)]
debug: bool,
#[arg(
help = "Output file to write the results to. If not specified, results are written to stdout.",
long,
short
)]
output: Option<PathBuf>,
}
/// An entry specified by `connect-to` to override DNS resolution and default
/// port numbers. For example, `example.org:80:localhost:5000` will connect to
/// `localhost:5000` whenever `http://example.org` is requested.
#[derive(Clone, Debug)]
pub struct ConnectToEntry {
pub requested_host: String,
pub requested_port: u16,
pub target_host: String,
pub target_port: u16,
}
impl FromStr for ConnectToEntry {
type Err = String;
fn from_str(s: &str) -> Result<Self, Self::Err> {
let expected_syntax: &str = "syntax for --connect-to is host:port:target_host:target_port";
let (s, target_port) = s.rsplit_once(':').ok_or(expected_syntax)?;
let (s, target_host) = if s.ends_with(']') {
// ipv6
let i = s.rfind(":[").ok_or(expected_syntax)?;
(&s[..i], &s[i + 1..])
} else {
s.rsplit_once(':').ok_or(expected_syntax)?
};
let (requested_host, requested_port) = s.rsplit_once(':').ok_or(expected_syntax)?;
Ok(ConnectToEntry {
requested_host: requested_host.into(),
requested_port: requested_port.parse().map_err(|err| {
format!("requested port must be an u16, but got {requested_port}: {err}")
})?,
target_host: target_host.into(),
target_port: target_port.parse().map_err(|err| {
format!("target port must be an u16, but got {target_port}: {err}")
})?,
})
}
}
/// A wrapper around a [`tokio_vsock::VsockAddr`] that provides a parser for clap
#[derive(Debug, Clone)]
#[repr(transparent)]
#[cfg(feature = "vsock")]
struct VsockAddr(tokio_vsock::VsockAddr);
#[cfg(feature = "vsock")]
impl FromStr for VsockAddr {
type Err = String;
fn from_str(s: &str) -> Result<Self, Self::Err> {
let (cid, port) = s
.split_once(':')
.ok_or("syntax for --vsock-addr is cid:port")?;
Ok(Self(tokio_vsock::VsockAddr::new(
cid.parse()
.map_err(|err| format!("cid must be a u32, but got {cid}: {err}"))?,
port.parse()
.map_err(|err| format!("port must be a u32, but got {port}: {err}"))?,
)))
}
}
pub async fn run(mut opts: Opts) -> anyhow::Result<()> {
let work_mode = opts.work_mode();
// Parse AWS credentials from basic auth if AWS signing is requested
let aws_config = if let Some(signing_params) = opts.aws_sigv4 {
if let Some(auth) = &opts.basic_auth {
let parts: Vec<&str> = auth.split(':').collect();
if parts.len() != 2 {
anyhow::bail!("Invalid AWS credentials format. Expected access_key:secret_key");
}
let access_key = parts[0];
let secret_key = parts[1];
let session_token = opts.aws_session.take();
Some(AwsSignatureConfig::new(
access_key,
secret_key,
&signing_params,
session_token,
)?)
} else {
anyhow::bail!("AWS credentials (--auth) required when using --aws-sigv4");
}
} else {
None
};
let parse_http_version = |is_http2: bool, version: Option<&str>| match (is_http2, version) {
(true, Some(_)) => anyhow::bail!("--http2 and --http-version are exclusive"),
(true, None) => Ok(http::Version::HTTP_2),
(false, Some(http_version)) => match http_version.trim() {
"0.9" => Ok(http::Version::HTTP_09),
"1.0" => Ok(http::Version::HTTP_10),
"1.1" => Ok(http::Version::HTTP_11),
"2.0" | "2" => Ok(http::Version::HTTP_2),
"3.0" | "3" => anyhow::bail!("HTTP/3 is not supported yet."),
_ => anyhow::bail!("Unknown HTTP version. Valid versions are 0.9, 1.0, 1.1, 2."),
},
(false, None) => Ok(http::Version::HTTP_11),
};
let http_version: http::Version = parse_http_version(opts.http2, opts.http_version.as_deref())?;
let proxy_http_version: http::Version =
parse_http_version(opts.proxy_http2, opts.proxy_http_version.as_deref())?;
let url_generator = if opts.rand_regex_url {
// Almost URL has dot in domain, so disable dot in regex for convenience.
let dot_disabled: String = opts
.url
.chars()
.map(|c| {
if c == '.' {
regex_syntax::escape(".")
} else {
c.to_string()
}
})
.collect();
UrlGenerator::new_dynamic(Regex::compile(&dot_disabled, opts.max_repeat)?)
} else if opts.urls_from_file {
let path = Path::new(opts.url.as_str());
let file = File::open(path)?;
let reader = std::io::BufReader::new(file);
let urls: Vec<Url> = reader
.lines()
.map_while(Result::ok)
.filter(|line| !line.trim().is_empty())
.map(|url_str| Url::parse(&url_str))
.collect::<Result<Vec<_>, _>>()?;
UrlGenerator::new_multi_static(urls)
} else {
UrlGenerator::new_static(Url::parse(&opts.url)?)
};
if let Some(n) = opts.dump_urls {
let mut rng = rand::rng();
for _ in 0..n {
let url = url_generator.generate(&mut rng)?;
println!("{}", url);
}
return Ok(());
}
let url = url_generator.generate(&mut rand::rng())?;
let headers = {
let mut headers: http::header::HeaderMap = Default::default();
// Accept all
headers.insert(
http::header::ACCEPT,
http::header::HeaderValue::from_static("*/*"),
);
// https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Encoding
if !opts.disable_compression {
headers.insert(
http::header::ACCEPT_ENCODING,
http::header::HeaderValue::from_static("gzip, compress, deflate, br"),
);
}
// User agent
headers
.entry(http::header::USER_AGENT)
.or_insert(HeaderValue::from_static(concat!(
"spiko/",
env!("CARGO_PKG_VERSION")
)));
if let Some(h) = opts.accept_header {
headers.insert(http::header::ACCEPT, HeaderValue::from_bytes(h.as_bytes())?);
}
if let Some(h) = opts.content_type {
headers.insert(
http::header::CONTENT_TYPE,
HeaderValue::from_bytes(h.as_bytes())?,
);
}
if let Some(h) = opts.host {
headers.insert(http::header::HOST, HeaderValue::from_bytes(h.as_bytes())?);
} else if http_version != http::Version::HTTP_2 {
headers.insert(
http::header::HOST,
http::header::HeaderValue::from_str(url.authority())?,
);
}
if let Some(auth) = opts.basic_auth {
let u_p = auth.splitn(2, ':').collect::<Vec<_>>();
anyhow::ensure!(u_p.len() == 2, anyhow::anyhow!("Parse auth"));
let mut header_value = b"Basic ".to_vec();
{
use std::io::Write;
let username = u_p[0];
let password = if u_p[1].is_empty() {
None
} else {
Some(u_p[1])
};
let mut encoder = base64::write::EncoderWriter::new(
&mut header_value,
&base64::engine::general_purpose::STANDARD,
);
// The unwraps here are fine because Vec::write* is infallible.
write!(encoder, "{username}:").unwrap();
if let Some(password) = password {
write!(encoder, "{password}").unwrap();
}
}
headers.insert(
http::header::AUTHORIZATION,
HeaderValue::from_bytes(&header_value)?,
);
}
if opts.disable_keepalive && http_version == http::Version::HTTP_11 {
headers.insert(http::header::CONNECTION, HeaderValue::from_static("close"));
}
for (k, v) in opts
.headers
.into_iter()
.map(|s| parse_header(s.as_str()))
.collect::<anyhow::Result<Vec<_>>>()?
{
headers.insert(k, v);
}
headers
};
let proxy_headers = {
opts.proxy_headers
.into_iter()
.map(|s| parse_header(s.as_str()))
.collect::<anyhow::Result<HeaderMap<_>>>()?
};
let body: Option<&'static [u8]> = match (opts.body_string, opts.body_path) {
(Some(body), _) => Some(Box::leak(body.into_boxed_str().into_boxed_bytes())),
(_, Some(path)) => {
let mut buf = Vec::new();
std::fs::File::open(path)?.read_to_end(&mut buf)?;
Some(Box::leak(buf.into_boxed_slice()))
}
_ => None,
};
let ip_strategy = match (opts.ipv4, opts.ipv6) {
(false, false) => Default::default(),
(true, false) => hickory_resolver::config::LookupIpStrategy::Ipv4Only,
(false, true) => hickory_resolver::config::LookupIpStrategy::Ipv6Only,
(true, true) => hickory_resolver::config::LookupIpStrategy::Ipv4AndIpv6,
};
let (config, mut resolver_opts) = system_resolv_conf()?;
resolver_opts.ip_strategy = ip_strategy;
let resolver = hickory_resolver::AsyncResolver::tokio(config, resolver_opts);
let cacert = opts.cacert.as_deref().map(std::fs::read).transpose()?;
let client_auth = match (opts.cert, opts.key) {
(Some(cert), Some(key)) => Some((std::fs::read(cert)?, std::fs::read(key)?)),
(None, None) => None,
// TODO: Ensure it on clap
_ => anyhow::bail!("Both --cert and --key must be specified"),
};
let client = Arc::new(client::Client {
aws_config,
http_version,
proxy_http_version,
url_generator,
method: opts.method,
headers,
proxy_headers,
body,
dns: client::Dns {
resolver,
connect_to: opts.connect_to,
},
timeout: opts.timeout.map(|d| d.into()),
redirect_limit: opts.redirect,
disable_keepalive: opts.disable_keepalive,
proxy_url: opts.proxy,
#[cfg(unix)]
unix_socket: opts.unix_socket,
#[cfg(feature = "vsock")]
vsock_addr: opts.vsock_addr.map(|v| v.0),
#[cfg(feature = "rustls")]
rustls_configs: tls_config::RuslsConfigs::new(
opts.insecure,
cacert.as_deref(),
client_auth
.as_ref()
.map(|(cert, key)| (cert.as_slice(), key.as_slice())),
),
#[cfg(all(feature = "native-tls", not(feature = "rustls")))]
native_tls_connectors: tls_config::NativeTlsConnectors::new(
opts.insecure,
cacert.as_deref(),
client_auth
.as_ref()
.map(|(cert, key)| (cert.as_slice(), key.as_slice())),
),
});
if !opts.no_pre_lookup {
client.pre_lookup().await?;
}
let no_tui = opts.no_tui || !std::io::stdout().is_tty() || opts.debug;
let print_config = {
let mode = if opts.json {
PrintMode::Json
} else {
PrintMode::Text
};
let disable_style =
opts.disable_color || !std::io::stdout().is_tty() || opts.output.is_some();
let output: Box<dyn std::io::Write + Send + 'static> = if let Some(output) = opts.output {
Box::new(File::create(output)?)
} else {
Box::new(std::io::stdout())
};
PrintConfig {
mode,
output,
disable_style,
stats_success_breakdown: opts.stats_success_breakdown,
}
};
let start = std::time::Instant::now();
let data_collect_future: Pin<Box<dyn std::future::Future<Output = (ResultData, PrintConfig)>>> =
match work_mode {
WorkMode::Debug => {
let mut print_config = print_config;
client::work_debug(&mut print_config.output, client).await?;
return Ok(());
}
WorkMode::FixedNumber {
n_requests,
n_connections,
n_http2_parallel,
query_limit: None,
latency_correction: _,
} if no_tui => {
// Use optimized worker of no_tui mode.
let (result_tx, result_rx) = flume::unbounded();
client::fast::work(
client.clone(),
result_tx,
n_requests,
n_connections,
n_http2_parallel,
)
.await;
Box::pin(async move {
let mut res = ResultData::default();
while let Ok(r) = result_rx.recv() {
res.merge(r);
}
(res, print_config)
})
}
WorkMode::Until {
duration,
n_connections,
n_http2_parallel,
query_limit: None,
latency_correction: _,
wait_ongoing_requests_after_deadline,
} if no_tui => {
// Use optimized worker of no_tui mode.
let (result_tx, result_rx) = flume::unbounded();
client::fast::work_until(
client.clone(),
result_tx,
start + duration,
n_connections,
n_http2_parallel,
wait_ongoing_requests_after_deadline,
)
.await;
Box::pin(async move {
let mut res = ResultData::default();
while let Ok(r) = result_rx.recv() {
res.merge(r);
}
(res, print_config)
})
}
mode => {
let (result_tx, result_rx) = flume::unbounded();
let data_collector = if no_tui {
// When `--no-tui` is enabled, just collect all data.
let token = tokio_util::sync::CancellationToken::new();
let result_rx_ctrl_c = result_rx.clone();
let token_ctrl_c = token.clone();
let ctrl_c = tokio::spawn(async move {
tokio::select! {
_ = tokio::signal::ctrl_c() => {
let mut all: ResultData = Default::default();
for report in result_rx_ctrl_c.drain() {
all.push(report);
}
let _ = printer::print_result(print_config, start, &all, start.elapsed());
std::process::exit(libc::EXIT_SUCCESS);
}
_ = token_ctrl_c.cancelled() => {
print_config
}
}
});
Box::pin(async move {
token.cancel();
let config = ctrl_c.await.unwrap();
let mut all = ResultData::default();
while let Ok(res) = result_rx.recv() {
all.push(res);
}
(all, config)
})
as Pin<Box<dyn std::future::Future<Output = (ResultData, PrintConfig)>>>
} else {
// Spawn monitor future which draws realtime tui
let join_handle = tokio::spawn(
monitor::Monitor {
print_config,
end_line: opts
.duration
.map(|d| monitor::EndLine::Duration(d.into()))
.unwrap_or(monitor::EndLine::NumQuery(opts.n_requests)),
report_receiver: result_rx,
start,
fps: opts.fps,
disable_color: opts.disable_color,
}
.monitor(),
);
Box::pin(async { join_handle.await.unwrap().unwrap() })
as Pin<Box<dyn std::future::Future<Output = (ResultData, PrintConfig)>>>
};
match mode {
WorkMode::Debug => unreachable!("Must be already handled"),
WorkMode::FixedNumber {
n_requests,
n_connections,
n_http2_parallel,
query_limit,
latency_correction,
} => {
if let Some(query_limit) = query_limit {
if latency_correction {
client::work_with_qps(
client.clone(),
result_tx,
query_limit,
n_requests,
n_connections,
n_http2_parallel,
)
.await;
} else {
client::work_with_qps_latency_correction(
client.clone(),
result_tx,
query_limit,
n_requests,
n_connections,
n_http2_parallel,
)
.await;
}
} else {
client::work(
client.clone(),
result_tx,
n_requests,
n_connections,
n_http2_parallel,
)
.await;
}
}
WorkMode::Until {
duration,
n_connections,
n_http2_parallel,
query_limit,
latency_correction,
wait_ongoing_requests_after_deadline,
} => {
if let Some(query_limit) = query_limit {
if latency_correction {
client::work_until_with_qps_latency_correction(
client.clone(),
result_tx,
query_limit,
start,
start + duration,
n_connections,
n_http2_parallel,
wait_ongoing_requests_after_deadline,
)
.await;
} else {
client::work_until_with_qps(
client.clone(),
result_tx,
query_limit,
start,
start + duration,
n_connections,
n_http2_parallel,
wait_ongoing_requests_after_deadline,
)
.await;
}
} else {
client::work_until(
client.clone(),
result_tx,
start + duration,
n_connections,
n_http2_parallel,
wait_ongoing_requests_after_deadline,
)
.await;
}
}
}
data_collector
}
};
let duration = start.elapsed();
let (res, print_config) = data_collect_future.await;
printer::print_result(print_config, start, &res, duration)?;
if let Some(db_url) = opts.db_url {
eprintln!("Storing results to {db_url}");
db::store(&client, &db_url, start, res.success())?;
}
Ok(())
}
fn system_resolv_conf() -> anyhow::Result<(ResolverConfig, ResolverOpts)> {
// check if we are running in termux https://github.com/termux/termux-app
#[cfg(unix)]
if env::var("TERMUX_VERSION").is_ok() {
let prefix = env::var("PREFIX")?;
let path = format!("{prefix}/etc/resolv.conf");
let conf_data = std::fs::read(&path).context(format!("DNS: failed to load {path}"))?;
return hickory_resolver::system_conf::parse_resolv_conf(conf_data)
.context(format!("DNS: failed to parse {path}"));
}
hickory_resolver::system_conf::read_system_conf()
.context("DNS: failed to load /etc/resolv.conf")
}
enum WorkMode {
Debug,
FixedNumber {
n_requests: usize,
n_connections: usize,
n_http2_parallel: usize,
query_limit: Option<client::QueryLimit>,
// ignored when query_limit is None
latency_correction: bool,
},
Until {
duration: std::time::Duration,
n_connections: usize,
n_http2_parallel: usize,
query_limit: Option<client::QueryLimit>,
// ignored when query_limit is None
latency_correction: bool,
wait_ongoing_requests_after_deadline: bool,
},
}
impl Opts {
fn work_mode(&self) -> WorkMode {
if self.debug {
WorkMode::Debug
} else if let Some(duration) = self.duration {
WorkMode::Until {
duration: duration.into(),
n_connections: self.n_connections,
n_http2_parallel: self.n_http2_parallel,
query_limit: match self.query_per_second {
Some(0) | None => self.burst_duration.map(|burst_duration| {
client::QueryLimit::Burst(
burst_duration.into(),
self.burst_requests.unwrap_or(1),
)
}),
Some(qps) => Some(client::QueryLimit::Qps(qps)),
},
latency_correction: self.latency_correction,
wait_ongoing_requests_after_deadline: self.wait_ongoing_requests_after_deadline,
}
} else {
WorkMode::FixedNumber {
n_requests: self.n_requests,
n_connections: self.n_connections,
n_http2_parallel: self.n_http2_parallel,
query_limit: match self.query_per_second {
Some(0) | None => self.burst_duration.map(|burst_duration| {
client::QueryLimit::Burst(
burst_duration.into(),
self.burst_requests.unwrap_or(1),
)
}),
Some(qps) => Some(client::QueryLimit::Qps(qps)),
},
latency_correction: self.latency_correction,
}
}
}
}
fn parse_header(s: &str) -> Result<(HeaderName, HeaderValue), anyhow::Error> {
let header = s.splitn(2, ':').collect::<Vec<_>>();
anyhow::ensure!(header.len() == 2, anyhow::anyhow!("Parse header"));
let name = HeaderName::from_str(header[0])?;
let value = HeaderValue::from_str(header[1].trim_start_matches(' '))?;
Ok::<(HeaderName, HeaderValue), anyhow::Error>((name, value))
}
================================================
FILE: src/main.rs
================================================
use clap::Parser;
use spiko::{run, Opts};
fn main() {
let num_workers_threads = std::env::var("TOKIO_WORKER_THREADS")
.ok()
.and_then(|s| s.parse().ok())
// Prefer to use physical cores rather than logical one because it's more performant empirically.
.unwrap_or(num_cpus::get_physical());
let rt = tokio::runtime::Builder::new_multi_thread()
.worker_threads(num_workers_threads)
.enable_all()
.build()
.unwrap();
if let Err(e) = rt.block_on(run(Opts::parse())) {
eprintln!("Error: {}", e);
std::process::exit(libc::EXIT_FAILURE);
}
}
================================================
FILE: src/monitor.rs
================================================
use byte_unit::Byte;
use crossterm::event::{Event, KeyCode, KeyEvent, KeyModifiers};
use hyper::http;
use ratatui::{DefaultTerminal, crossterm};
use ratatui::{
layout::{Constraint, Direction, Layout},
style::{Color, Style},
text::{Line, Span},
widgets::{BarChart, Block, Borders, Gauge, Paragraph},
};
use std::collections::BTreeMap;
use crate::{
client::{ClientError, RequestResult},
printer::PrintConfig,
result_data::{MinMaxMean, ResultData},
timescale::{TimeLabel, TimeScale},
};
/// When the monitor ends
pub enum EndLine {
/// After a duration
Duration(std::time::Duration),
/// After n query done
NumQuery(usize),
}
struct ColorScheme {
light_blue: Option<Color>,
green: Option<Color>,
yellow: Option<Color>,
}
impl ColorScheme {
fn new() -> ColorScheme {
ColorScheme {
light_blue: None,
green: None,
yellow: None,
}
}
fn set_colors(&mut self) {
self.light_blue = Some(Color::Cyan);
self.green = Some(Color::Green);
self.yellow = Some(Color::Yellow);
}
}
pub struct Monitor {
pub print_config: PrintConfig,
pub end_line: EndLine,
/// All workers sends each result to this channel
pub report_receiver: flume::Receiver<Result<RequestResult, ClientError>>,
// When started
pub start: std::time::Instant,
// Frame per second of TUI
pub fps: usize,
pub disable_color: bool,
}
struct IntoRawMode;
impl IntoRawMode {
pub fn new() -> Result<(Self, DefaultTerminal), std::io::Error> {
let terminal = ratatui::try_init()?;
Ok((Self, terminal))
}
}
impl Drop for IntoRawMode {
fn drop(&mut self) {
ratatui::restore();
}
}
impl Monitor {
pub async fn monitor(self) -> Result<(ResultData, PrintConfig), std::io::Error> {
let (raw_mode, mut terminal) = IntoRawMode::new()?;
// Return this when ends to application print summary
// We must not read all data from this due to computational cost.
let mut all: ResultData = Default::default();
// stats for HTTP status
let mut status_dist: BTreeMap<http::StatusCode, usize> = Default::default();
#[cfg(unix)]
// Limit for number open files. eg. ulimit -n
let nofile_limit = rlimit::getrlimit(rlimit::Resource::NOFILE);
// None means auto timescale which depends on how long it takes
let mut timescale_auto = None;
let mut colors = ColorScheme::new();
if !self.disable_color {
colors.set_colors();
}
loop {
let frame_start = std::time::Instant::now();
let is_disconnected = self.report_receiver.is_disconnected();
for report in self.report_receiver.drain() {
if let Ok(report) = report.as_ref() {
*status_dist.entry(report.status).or_default() += 1;
}
all.push(report);
}
if is_disconnected {
break;
}
let now = std::time::Instant::now();
let progress = match &self.end_line {
EndLine::Duration(d) => {
((now - self.start).as_secs_f64() / d.as_secs_f64()).clamp(0.0, 1.0)
}
EndLine::NumQuery(n) => (all.len() as f64 / *n as f64).clamp(0.0, 1.0),
};
let count = 32;
let timescale = if let Some(timescale) = timescale_auto {
timescale
} else {
TimeScale::from_elapsed(self.start.elapsed())
};
let bin = timescale.as_secs_f64();
let mut bar_num_req = vec![0u64; count];
let short_bin = (now - self.start).as_secs_f64() % bin;
for r in all.success().iter().rev() {
let past = (now - r.end).as_secs_f64();
let i = if past <= short_bin {
0
} else {
1 + ((past - short_bin) / bin) as usize
};
if i >= bar_num_req.len() {
break;
}
bar_num_req[i] += 1;
}
let cols = bar_num_req
.iter()
.map(|x| x.to_string().chars().count())
.max()
.unwrap_or(0);
let bar_num_req: Vec<(String, u64)> = bar_num_req
.into_iter()
.enumerate()
.map(|(i, n)| {
(
{
let mut s = TimeLabel { x: i, timescale }.to_string();
if cols > s.len() {
for _ in 0..cols - s.len() {
s.push(' ');
}
}
s
},
n,
)
})
.collect();
let bar_num_req_str: Vec<(&str, u64)> =
bar_num_req.iter().map(|(a, b)| (a.as_str(), *b)).collect();
#[cfg(unix)]
let nofile = std::fs::read_dir("/dev/fd").map(|dir| dir.count());
terminal.draw(|f| {
let row4 = Layout::default()
.direction(Direction::Vertical)
.constraints(
[
Constraint::Length(3),
Constraint::Length(8),
Constraint::Length(all.error_distribution().len() as u16 + 2),
Constraint::Fill(1),
]
.as_ref(),
)
.split(f.area());
let mid = Layout::default()
.direction(Direction::Horizontal)
.constraints([Constraint::Percentage(50), Constraint::Percentage(50)].as_ref())
.split(row4[1]);
let bottom = Layout::default()
.direction(Direction::Horizontal)
.constraints([Constraint::Percentage(50), Constraint::Percentage(50)].as_ref())
.split(row4[3]);
let gauge_label = match &self.end_line {
EndLine::Duration(d) => format!(
"{} / {}",
humantime::Duration::from(std::time::Duration::from_secs(
(now - self.start).as_secs_f64() as u64
)),
humantime::Duration::from(*d)
),
EndLine::NumQuery(n) => format!("{} / {}", all.len(), n),
};
let gauge = Gauge::default()
.block(Block::default().title("Progress").borders(Borders::ALL))
.gauge_style(Style::default().fg(colors.light_blue.unwrap_or(Color::White)))
.label(Span::raw(gauge_label))
.ratio(progress);
f.render_widget(gauge, row4[0]);
let last_1_timescale = {
let success = all.success();
let index = match success.binary_search_by(|probe| {
(now - probe.end)
.as_secs_f64()
.partial_cmp(×cale.as_secs_f64())
// Should be fine
.unwrap()
.reverse()
}) {
Ok(i) => i,
Err(i) => i,
};
&success[index..]
};
let last_1_minmaxmean: MinMaxMean = last_1_timescale
.iter()
.map(|r| r.duration().as_secs_f64())
.collect();
let stats_text = vec![
Line::from(format!("Requests : {}", last_1_timescale.len())),
Line::from(vec![Span::styled(
format!("Slowest: {:.4} secs", last_1_minmaxmean.max(),),
Style::default().fg(colors.yellow.unwrap_or(Color::Reset)),
)]),
Line::from(vec![Span::styled(
format!("Fastest: {:.4} secs", last_1_minmaxmean.min(),),
Style::default().fg(colors.green.unwrap_or(Color::Reset)),
)]),
Line::from(vec![Span::styled(
format!("Average: {:.4} secs", last_1_minmaxmean.mean(),),
Style::default().fg(colors.light_blue.unwrap_or(Color::Reset)),
)]),
Line::from(format!(
"Data: {:.2}",
Byte::from_u64(
last_1_timescale
.iter()
.map(|r| r.len_bytes as u64)
.sum::<u64>()
)
.get_appropriate_unit(byte_unit::UnitType::Binary)
)),
#[cfg(unix)]
// Note: Windows can open 255 * 255 * 255 files. So not showing on windows is OK.
Line::from(format!(
"Number of open files: {} / {}",
nofile
.map(|c| c.to_string())
.unwrap_or_else(|_| "Error".to_string()),
nofile_limit
.as_ref()
.map(|(s, _h)| s.to_string())
.unwrap_or_else(|_| "Unknown".to_string())
)),
];
let stats_title = format!("stats for last {timescale}");
let stats = Paragraph::new(stats_text).block(
Block::default()
.title(Span::raw(stats_title))
.borders(Borders::ALL),
);
f.render_widget(stats, mid[0]);
let mut status_v: Vec<(http::StatusCode, usize)> =
status_dist.clone().into_iter().collect();
status_v.sort_by_key(|t| std::cmp::Reverse(t.1));
let stats2_text = status_v
.into_iter()
.map(|(status, count)| {
Line::from(format!("[{}] {} responses", status.as_str(), count))
})
.collect::<Vec<_>>();
let stats2 = Paragraph::new(stats2_text).block(
Block::default()
.title("Status code distribution")
.borders(Borders::ALL),
);
f.render_widget(stats2, mid[1]);
let mut error_v: Vec<(String, usize)> =
all.error_distribution().clone().into_iter().collect();
error_v.sort_by_key(|t| std::cmp::Reverse(t.1));
let errors_text = error_v
.into_iter()
.map(|(e, count)| Line::from(format!("[{count}] {e}")))
.collect::<Vec<_>>();
let errors = Paragraph::new(errors_text).block(
Block::default()
.title("Error distribution")
.borders(Borders::ALL),
);
f.render_widget(errors, row4[2]);
let title = format!(
"Requests / past {}{}. press -/+/a to change",
timescale,
if timescale_auto.is_none() {
" (auto)"
} else {
""
}
);
let barchart = BarChart::default()
.block(
Block::default()
.title(Span::raw(title))
.style(
Style::default()
.fg(colors.green.unwrap_or(Color::Reset))
.bg(Color::Reset),
)
.borders(Borders::ALL),
)
.data(bar_num_req_str.as_slice())
.bar_width(
bar_num_req
.iter()
.map(|(s, _)| s.chars().count())
.max()
.map(|w| w + 2)
.unwrap_or(1) as u16,
);
f.render_widget(barchart, bottom[0]);
let resp_histo_width = 7;
let resp_histo_data: Vec<(String, u64)> = {
let bins = if bottom[1].width < 2 {
0
} else {
(bottom[1].width as usize - 2) / (resp_histo_width + 1)
}
.max(2);
let values = last_1_timescale
.iter()
.map(|r| r.duration().as_secs_f64())
.collect::<Vec<_>>();
let histo = crate::histogram::histogram(&values, bins);
histo
.into_iter()
.map(|(label, v)| (format!("{label:.4}"), v as u64))
.collect()
};
let resp_histo_data_str: Vec<(&str, u64)> = resp_histo_data
.iter()
.map(|(l, v)| (l.as_str(), *v))
.collect();
let resp_histo = BarChart::default()
.block(
Block::default()
.title("Response time histogram")
.style(
Style::default()
.fg(colors.yellow.unwrap_or(Color::Reset))
.bg(Color::Reset),
)
.borders(Borders::ALL),
)
.data(resp_histo_data_str.as_slice())
.bar_width(resp_histo_width as u16);
f.render_widget(resp_histo, bottom[1]);
})?;
while crossterm::event::poll(std::time::Duration::from_secs(0))? {
match crossterm::event::read()? {
Event::Key(KeyEvent {
code: KeyCode::Char('+'),
..
}) => timescale_auto = Some(timescale.dec()),
Event::Key(KeyEvent {
code: KeyCode::Char('-'),
..
}) => timescale_auto = Some(timescale.inc()),
Event::Key(KeyEvent {
code: KeyCode::Char('a'),
..
}) => {
if timescale_auto.is_some() {
timescale_auto = None;
} else {
timescale_auto = Some(timescale)
}
}
// User pressed q or ctrl-c
Event::Key(KeyEvent {
code: KeyCode::Char('q'),
..
})
| Event::Key(KeyEvent {
code: KeyCode::Char('c'),
modifiers: KeyModifiers::CONTROL,
..
}) => {
drop(terminal);
drop(raw_mode);
let _ = crate::printer::print_result(
self.print_config,
self.start,
&all,
now - self.start,
);
std::process::exit(libc::EXIT_SUCCESS);
}
_ => (),
}
}
let per_frame = std::time::Duration::from_secs(1) / self.fps as u32;
let elapsed = frame_start.elapsed();
if per_frame > elapsed {
tokio::time::sleep(per_frame - elapsed).await;
}
}
Ok((all, self.print_config))
}
}
================================================
FILE: src/pcg64si.rs
================================================
// https://github.com/imneme/pcg-c
use rand::{RngCore, SeedableRng};
use rand_core::impls;
#[derive(Debug, Copy, Clone)]
#[repr(transparent)]
pub struct Pcg64Si {
state: u64,
}
impl RngCore for Pcg64Si {
fn next_u32(&mut self) -> u32 {
self.next_u64() as u32
}
fn next_u64(&mut self) -> u64 {
let old_state = self.state;
self.state = self
.state
.wrapping_mul(6364136223846793005)
.wrapping_add(1442695040888963407);
let word =
((old_state >> ((old_state >> 59) + 5)) ^ old_state).wrapping_mul(12605985483714917081);
(word >> 43) ^ word
}
fn fill_bytes(&mut self, dest: &mut [u8]) {
impls::fill_bytes_via_next(self, dest)
}
}
impl SeedableRng for Pcg64Si {
type Seed = [u8; 8];
fn from_seed(seed: Self::Seed) -> Pcg64Si {
Pcg64Si {
state: u64::from_le_bytes(seed),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::collections::HashSet;
// For a given seed the RNG is deterministic
// thus we can perform some basic tests consistently
#[test]
fn test_rng_next() {
let mut rng = Pcg64Si::from_seed([1, 2, 3, 4, 5, 6, 7, 8]);
let mut values_set: HashSet<u32> = HashSet::new();
// Generate 1000 values modulus 100 (so each value is between 0 and 99)
for _ in 0..1000 {
values_set.insert(rng.next_u32() % 100);
}
// Expect to generate every number between 0 and 99 (the generated values are somewhat evenly distributed)
assert_eq!(values_set.len(), 100);
}
#[test]
fn test_rng_from_seed() {
// Different seeds should result in a different RNG state
let rng1 = Pcg64Si::from_seed([1, 2, 3, 4, 5, 6, 7, 8]);
let rng2 = Pcg64Si::from_seed([1, 2, 3, 4, 5, 6, 7, 7]);
assert_ne!(rng1.state, rng2.state);
}
#[test]
fn test_rng_fill_bytes() {
// This uses the next_u64/u32 functions underneath, so don't need to test the pseudo randomness again
let mut array: [u8; 8] = [0, 0, 0, 0, 0, 0, 0, 0];
let mut rng = Pcg64Si::from_seed([1, 2, 3, 4, 5, 6, 7, 8]);
rng.fill_bytes(&mut array);
assert_ne!(array, [0, 0, 0, 0, 0, 0, 0, 0]);
}
}
================================================
FILE: src/printer.rs
================================================
use crate::result_data::ResultData;
use average::{Max, Min, Variance};
use byte_unit::Byte;
use crossterm::style::{StyledContent, Stylize};
use hyper::http::{self, StatusCode};
use ratatui::crossterm;
use std::{
collections::BTreeMap,
io::Write,
time::{Duration, Instant},
};
#[derive(Clone, Copy)]
struct StyleScheme {
style_enabled: bool,
}
impl StyleScheme {
fn no_style(self, text: &str) -> StyledContent<&str> {
StyledContent::new(crossterm::style::ContentStyle::new(), text)
}
fn heading(self, text: &str) -> StyledContent<&str> {
if self.style_enabled {
text.bold().underlined()
} else {
self.no_style(text)
}
}
fn success_rate(self, text: &str, success_rate: f64) -> StyledContent<&str> {
if self.style_enabled {
if success_rate >= 100.0 {
text.green().bold()
} else if success_rate >= 99.0 {
text.yellow().bold()
} else {
text.red().bold()
}
} else {
self.no_style(text)
}
}
fn fastest(self, text: &str) -> StyledContent<&str> {
if self.style_enabled {
text.green()
} else {
self.no_style(text)
}
}
fn slowest(self, text: &str) -> StyledContent<&str> {
if self.style_enabled {
text.yellow()
} else {
self.no_style(text)
}
}
fn average(self, text: &str) -> StyledContent<&str> {
if self.style_enabled {
text.cyan()
} else {
self.no_style(text)
}
}
fn latency_distribution(self, text: &str, label: f64) -> StyledContent<&str> {
// See #609 for justification of these thresholds
const LATENCY_YELLOW_THRESHOLD: f64 = 0.1;
const LATENCY_RED_THRESHOLD: f64 = 0.4;
if self.style_enabled {
if label <= LATENCY_YELLOW_THRESHOLD {
text.green()
} else if label <= LATENCY_RED_THRESHOLD {
text.yellow()
} else {
text.red()
}
} else {
self.no_style(text)
}
}
fn status_distribution(self, text: &str, status: StatusCode) -> StyledContent<&str> {
if self.style_enabled {
if status.is_success() {
text.green()
} else if status.is_client_error() {
text.yellow()
} else if status.is_server_error() {
text.red()
} else {
text.white()
}
} else {
self.no_style(text)
}
}
}
#[derive(Clone, Copy, Debug)]
pub enum PrintMode {
Text,
Json,
}
pub struct PrintConfig {
pub output: Box<dyn Write + Send + 'static>,
pub mode: PrintMode,
pub disable_style: bool,
pub stats_success_breakdown: bool,
}
pub fn print_result(
mut config: PrintConfig,
start: Instant,
res: &ResultData,
total_duration: Duration,
) -> anyhow::Result<()> {
match config.mode {
PrintMode::Text => print_summary(
&mut config.output,
res,
total_duration,
config.disable_style,
config.stats_success_breakdown,
)?,
PrintMode::Json => print_json(
&mut config.output,
start,
res,
total_duration,
config.stats_success_breakdown,
)?,
}
Ok(())
}
/// Print all summary as JSON
fn print_json<W: Write>(
w: &mut W,
start: Instant,
res: &ResultData,
total_duration: Duration,
stats_success_breakdown: bool,
) -> serde_json::Result<()> {
use serde::Serialize;
#[derive(Serialize)]
struct Summary {
#[serde(rename = "successRate")]
success_rate: f64,
total: f64,
slowest: f64,
fastest: f64,
average: f64,
#[serde(rename = "requestsPerSec")]
requests_per_sec: f64,
#[serde(rename = "totalData")]
total_data: u64,
#[serde(rename = "sizePerRequest")]
size_per_request: Option<u64>,
#[serde(rename = "sizePerSec")]
size_per_sec: f64,
}
#[derive(Serialize)]
struct Triple {
average: f64,
fastest: f64,
slowest: f64,
}
#[derive(Serialize)]
struct Details {
#[serde(rename = "DNSDialup")]
dns_dialup: Triple,
#[serde(rename = "DNSLookup")]
dns_lookup: Triple,
}
#[derive(Serialize)]
struct Rps {
mean: f64,
stddev: f64,
max: f64,
min: f64,
percentiles: BTreeMap<String, f64>,
}
#[derive(Serialize)]
struct Result {
summary: Summary,
#[serde(rename = "responseTimeHistogram")]
response_time_histogram: BTreeMap<String, usize>,
#[serde(rename = "latencyPercentiles")]
latency_percentiles: BTreeMap<String, f64>,
#[serde(
rename = "responseTimeHistogramSuccessful",
skip_serializing_if = "Option::is_none"
)]
response_time_histogram_successful: Option<BTreeMap<String, usize>>,
#[serde(
rename = "latencyPercentilesSuccessful",
skip_serializing_if = "Option::is_none"
)]
latency_percentiles_successful: Option<BTreeMap<String, f64>>,
#[serde(
rename = "responseTimeHistogramNotSuccessful",
skip_serializing_if = "Option::is_none"
)]
response_time_histogram_not_successful: Option<BTreeMap<String, usize>>,
#[serde(
rename = "latencyPercentilesNotSuccessful",
skip_serializing_if = "Option::is_none"
)]
latency_percentiles_not_successful: Option<BTreeMap<String, f64>>,
#[serde(rename = "rps")]
rps: Rps,
details: Details,
#[serde(rename = "statusCodeDistribution")]
status_code_distribution: BTreeMap<String, usize>,
#[serde(rename = "errorDistribution")]
error_distribution: BTreeMap<String, usize>,
}
let latency_stat = res.latency_stat();
let summary = Summary {
success_rate: res.success_rate(),
total: total_duration.as_secs_f64(),
slowest: latency_stat.max(),
fastest: latency_stat.min(),
average: latency_stat.mean(),
requests_per_sec: res.len() as f64 / total_duration.as_secs_f64(),
total_data: res.total_data() as u64,
size_per_request: res.size_per_request(),
size_per_sec: res.total_data() as f64 / total_duration.as_secs_f64(),
};
let durations_statistics = res.duration_all_statistics();
let response_time_histogram = durations_statistics
.histogram
.into_iter()
.map(|(k, v)| (k.to_string(), v))
.collect();
let latency_percentiles = durations_statistics
.percentiles
.into_iter()
.map(|(p, v)| (format!("p{p}"), v))
.collect();
let mut response_time_histogram_successful: Option<BTreeMap<String, usize>> = None;
let mut latency_percentiles_successful: Option<BTreeMap<String, f64>> = None;
let mut response_time_histogram_not_successful: Option<BTreeMap<String, usize>> = None;
let mut latency_percentiles_not_successful: Option<BTreeMap<String, f64>> = None;
if stats_success_breakdown {
let durations_successful_statistics = res.duration_successful_statistics();
response_time_histogram_successful = Some(
durations_successful_statistics
.histogram
.into_iter()
.map(|(k, v)| (k.to_string(), v))
.collect(),
);
latency_percentiles_successful = Some(
durations_successful_statistics
.percentiles
.into_iter()
.map(|(p, v)| (format!("p{p}"), v))
.collect(),
);
let durations_not_successful_statistics = res.duration_not_successful_statistics();
response_time_histogram_not_successful = Some(
durations_not_successful_statistics
.histogram
.into_iter()
.map(|(k, v)| (k.to_string(), v))
.collect(),
);
latency_percentiles_not_successful = Some(
durations_not_successful_statistics
.percentiles
.into_iter()
.map(|(p, v)| (format!("p{p}"), v))
.collect(),
);
}
let mut ends = res
.end_times_from_start(start)
.map(|d| d.as_secs_f64())
.collect::<Vec<_>>();
ends.push(0.0);
float_ord::sort(&mut ends);
let mut rps: Vec<f64> = Vec::new();
// 10ms
const INTERVAL: f64 = 0.01;
let mut r = 0;
loop {
let prev_r = r;
// increment at least 1
if r + 1 < ends.len() {
r += 1;
}
while r + 1 < ends.len() && ends[prev_r] + INTERVAL > ends[r + 1] {
r += 1;
}
if r == prev_r {
break;
}
let n = r - prev_r;
let t = ends[r] - ends[prev_r];
rps.push(n as f64 / t);
}
let rps_percentiles = percentiles(&mut rps);
let variance = rps.iter().collect::<Variance>();
let rps = Rps {
mean: variance.mean(),
stddev: variance.sample_variance().sqrt(),
max: rps.iter().collect::<Max>().max(),
min: rps.iter().collect::<Min>().min(),
percentiles: rps_percentiles,
};
let status_code_distribution = res.status_code_distribution();
let dns_dialup_stat = res.dns_dialup_stat();
let dns_lookup_stat = res.dns_lookup_st
gitextract_ygc3eeo3/
├── .dockerignore
├── .editorconfig
├── .github/
│ ├── ISSUE_TEMPLATE/
│ │ └── bug_report.md
│ ├── dependabot.yml
│ └── workflows/
│ ├── docker.yml
│ ├── publish.yml
│ └── release.yml
├── .gitignore
├── .vscode/
│ └── settings.json
├── CHANGELOG.md
├── Cargo.toml
├── Cross.toml
├── Dockerfile
├── LICENSE
├── Makefile
├── README.md
├── demo.tape
├── pgo/
│ └── server/
│ ├── Cargo.toml
│ └── src/
│ └── main.rs
├── pgo.js
├── schema.json
├── scripts/
│ └── release-version.sh
├── src/
│ ├── aws_auth.rs
│ ├── client.rs
│ ├── db.rs
│ ├── histogram.rs
│ ├── lib.rs
│ ├── main.rs
│ ├── monitor.rs
│ ├── pcg64si.rs
│ ├── printer.rs
│ ├── result_data.rs
│ ├── timescale.rs
│ ├── tls_config.rs
│ └── url_generator.rs
└── tests/
└── tests.rs
SYMBOL INDEX (212 symbols across 15 files)
FILE: pgo/server/src/main.rs
function main (line 7) | async fn main() {
function root (line 20) | async fn root() -> &'static str {
FILE: src/aws_auth.rs
type AwsSignatureConfig (line 11) | pub struct AwsSignatureConfig {
method sign_request (line 32) | pub fn sign_request(
method new (line 106) | pub fn new(
FILE: src/client.rs
type SendRequestHttp1 (line 29) | type SendRequestHttp1 = hyper::client::conn::http1::SendRequest<Full<Byt...
type SendRequestHttp2 (line 30) | type SendRequestHttp2 = hyper::client::conn::http2::SendRequest<Full<Byt...
type ConnectionTime (line 33) | pub struct ConnectionTime {
type RequestResult (line 40) | pub struct RequestResult {
method duration (line 59) | pub fn duration(&self) -> std::time::Duration {
type Dns (line 64) | pub struct Dns {
method lookup (line 72) | async fn lookup<R: Rng>(
type ClientError (line 121) | pub enum ClientError {
type Client (line 174) | pub struct Client {
method is_http2 (line 351) | fn is_http2(&self) -> bool {
method is_proxy_http2 (line 356) | fn is_proxy_http2(&self) -> bool {
method is_work_http2 (line 360) | pub fn is_work_http2(&self) -> bool {
method pre_lookup (line 378) | pub async fn pre_lookup(&self) -> Result<(), ClientError> {
method generate_url (line 398) | pub fn generate_url(&self, rng: &mut Pcg64Si) -> Result<(Cow<Url>, Pcg...
method client (line 403) | async fn client<R: Rng>(
method tls_client (line 466) | async fn tls_client(
method connect_tls (line 481) | async fn connect_tls<S>(
method connect_tls (line 499) | async fn connect_tls<S>(
method client_http1 (line 518) | async fn client_http1<R: Rng>(
method request (line 566) | fn request(&self, url: &Url) -> Result<http::Request<Full<Bytes>>, Cli...
method work_http1 (line 612) | async fn work_http1(
method connect_http2 (line 708) | async fn connect_http2<R: Rng>(
method work_http2 (line 764) | async fn work_http2(
method redirect (line 817) | async fn redirect<R: Rng + Send>(
method default (line 199) | fn default() -> Self {
type ClientStateHttp1 (line 229) | struct ClientStateHttp1 {
method default (line 235) | fn default() -> Self {
type ClientStateHttp2 (line 243) | struct ClientStateHttp2 {
type QueryLimit (line 248) | pub enum QueryLimit {
type Stream (line 255) | enum Stream {
method handshake_http1 (line 269) | async fn handshake_http1(self, with_upgrade: bool) -> Result<SendReque...
method handshake_http2 (line 315) | async fn handshake_http2(self) -> Result<SendRequestHttp2, ClientError> {
function is_cancel_error (line 883) | fn is_cancel_error(res: &Result<RequestResult, ClientError>) -> bool {
function is_too_many_open_files (line 888) | fn is_too_many_open_files(res: &Result<RequestResult, ClientError>) -> b...
function is_hyper_error (line 899) | fn is_hyper_error(res: &Result<RequestResult, ClientError>) -> bool {
function setup_http2 (line 912) | async fn setup_http2(client: &Client) -> Result<(ConnectionTime, SendReq...
function work_http2_once (line 921) | async fn work_http2_once(
function set_connection_time (line 939) | fn set_connection_time<E>(res: &mut Result<RequestResult, E>, connection...
function set_start_latency_correction (line 945) | fn set_start_latency_correction<E>(
function work_debug (line 954) | pub async fn work_debug<W: Write>(w: &mut W, client: Arc<Client>) -> Res...
function work (line 983) | pub async fn work(
function work_with_qps (line 1095) | pub async fn work_with_qps(
function work_with_qps_latency_correction (line 1243) | pub async fn work_with_qps_latency_correction(
function work_until (line 1395) | pub async fn work_until(
function work_until_with_qps (line 1539) | pub async fn work_until_with_qps(
function work_until_with_qps_latency_correction (line 1725) | pub async fn work_until_with_qps_latency_correction(
function work (line 1925) | pub async fn work(
function work_until (line 2129) | pub async fn work_until(
FILE: src/db.rs
function create_db (line 5) | fn create_db(conn: &Connection) -> Result<usize, rusqlite::Error> {
function store (line 20) | pub fn store(
function test_store (line 60) | fn test_store() {
FILE: src/histogram.rs
function histogram (line 1) | pub fn histogram(values: &[f64], bins: usize) -> Vec<(f64, usize)> {
function test_histogram (line 25) | fn test_histogram() {
FILE: src/lib.rs
type Opts (line 52) | pub struct Opts {
method work_mode (line 889) | fn work_mode(&self) -> WorkMode {
type ConnectToEntry (line 287) | pub struct ConnectToEntry {
type Err (line 295) | type Err = String;
method from_str (line 297) | fn from_str(s: &str) -> Result<Self, Self::Err> {
type VsockAddr (line 327) | struct VsockAddr(tokio_vsock::VsockAddr);
type Err (line 331) | type Err = String;
method from_str (line 333) | fn from_str(s: &str) -> Result<Self, Self::Err> {
function run (line 346) | pub async fn run(mut opts: Opts) -> anyhow::Result<()> {
function system_resolv_conf (line 852) | fn system_resolv_conf() -> anyhow::Result<(ResolverConfig, ResolverOpts)> {
type WorkMode (line 867) | enum WorkMode {
function parse_header (line 929) | fn parse_header(s: &str) -> Result<(HeaderName, HeaderValue), anyhow::Er...
FILE: src/main.rs
function main (line 4) | fn main() {
FILE: src/monitor.rs
type EndLine (line 21) | pub enum EndLine {
type ColorScheme (line 28) | struct ColorScheme {
method new (line 35) | fn new() -> ColorScheme {
method set_colors (line 43) | fn set_colors(&mut self) {
type Monitor (line 50) | pub struct Monitor {
method monitor (line 78) | pub async fn monitor(self) -> Result<(ResultData, PrintConfig), std::i...
type IntoRawMode (line 62) | struct IntoRawMode;
method new (line 65) | pub fn new() -> Result<(Self, DefaultTerminal), std::io::Error> {
method drop (line 72) | fn drop(&mut self) {
FILE: src/pcg64si.rs
type Pcg64Si (line 7) | pub struct Pcg64Si {
method next_u32 (line 12) | fn next_u32(&mut self) -> u32 {
method next_u64 (line 16) | fn next_u64(&mut self) -> u64 {
method fill_bytes (line 28) | fn fill_bytes(&mut self, dest: &mut [u8]) {
type Seed (line 34) | type Seed = [u8; 8];
method from_seed (line 36) | fn from_seed(seed: Self::Seed) -> Pcg64Si {
function test_rng_next (line 51) | fn test_rng_next() {
function test_rng_from_seed (line 63) | fn test_rng_from_seed() {
function test_rng_fill_bytes (line 71) | fn test_rng_fill_bytes() {
FILE: src/printer.rs
type StyleScheme (line 14) | struct StyleScheme {
method no_style (line 18) | fn no_style(self, text: &str) -> StyledContent<&str> {
method heading (line 21) | fn heading(self, text: &str) -> StyledContent<&str> {
method success_rate (line 28) | fn success_rate(self, text: &str, success_rate: f64) -> StyledContent<...
method fastest (line 41) | fn fastest(self, text: &str) -> StyledContent<&str> {
method slowest (line 48) | fn slowest(self, text: &str) -> StyledContent<&str> {
method average (line 55) | fn average(self, text: &str) -> StyledContent<&str> {
method latency_distribution (line 63) | fn latency_distribution(self, text: &str, label: f64) -> StyledContent...
method status_distribution (line 81) | fn status_distribution(self, text: &str, status: StatusCode) -> Styled...
type PrintMode (line 99) | pub enum PrintMode {
type PrintConfig (line 104) | pub struct PrintConfig {
function print_result (line 111) | pub fn print_result(
function print_json (line 137) | fn print_json<W: Write>(
function print_summary (line 377) | fn print_summary<W: Write>(
function print_histogram (line 552) | fn print_histogram<W: Write>(
function bar (line 589) | fn bar<W: Write>(w: &mut W, ratio: f64, style: StyleScheme, label: f64) ...
function percentile_iter (line 598) | fn percentile_iter(values: &mut [f64]) -> impl Iterator<Item = (f64, f64...
function print_distribution (line 610) | fn print_distribution<W: Write>(
function percentiles (line 626) | fn percentiles(values: &mut [f64]) -> BTreeMap<String, f64> {
function test_percentile_iter (line 639) | fn test_percentile_iter() {
FILE: src/result_data.rs
type ResultData (line 18) | pub struct ResultData {
method push (line 55) | pub fn push(&mut self, result: Result<RequestResult, ClientError>) {
method len (line 65) | pub fn len(&self) -> usize {
method merge (line 69) | pub fn merge(&mut self, other: ResultData) {
method success (line 79) | pub fn success(&self) -> &[RequestResult] {
method success_rate (line 85) | pub fn success_rate(&self) -> f64 {
method latency_stat (line 99) | pub fn latency_stat(&self) -> MinMaxMean {
method error_distribution (line 106) | pub fn error_distribution(&self) -> &BTreeMap<String, usize> {
method end_times_from_start (line 110) | pub fn end_times_from_start(&self, start: Instant) -> impl Iterator<It...
method status_code_distribution (line 114) | pub fn status_code_distribution(&self) -> BTreeMap<StatusCode, usize> {
method dns_dialup_stat (line 123) | pub fn dns_dialup_stat(&self) -> MinMaxMean {
method dns_lookup_stat (line 133) | pub fn dns_lookup_stat(&self) -> MinMaxMean {
method total_data (line 143) | pub fn total_data(&self) -> usize {
method size_per_request (line 147) | pub fn size_per_request(&self) -> Option<u64> {
method duration_all_statistics (line 155) | pub fn duration_all_statistics(&self) -> Statistics {
method duration_successful_statistics (line 164) | pub fn duration_successful_statistics(&self) -> Statistics {
method duration_not_successful_statistics (line 175) | pub fn duration_not_successful_statistics(&self) -> Statistics {
type Statistics (line 25) | pub struct Statistics {
method new (line 32) | fn new(data: &mut [f64]) -> Self {
function percentile_iter (line 42) | fn percentile_iter(values: &mut [f64]) -> impl Iterator<Item = (f64, f64...
function build_mock_request_result (line 196) | fn build_mock_request_result(
function build_mock_request_results (line 224) | fn build_mock_request_results() -> ResultData {
function test_calculate_success_rate (line 252) | fn test_calculate_success_rate() {
function test_calculate_slowest_request (line 258) | fn test_calculate_slowest_request() {
function test_calculate_average_request (line 264) | fn test_calculate_average_request() {
function test_calculate_total_data (line 270) | fn test_calculate_total_data() {
function test_calculate_size_per_request (line 276) | fn test_calculate_size_per_request() {
function test_calculate_connection_times_dns_dialup_average (line 282) | fn test_calculate_connection_times_dns_dialup_average() {
function test_calculate_connection_times_dns_dialup_fastest (line 288) | fn test_calculate_connection_times_dns_dialup_fastest() {
function test_calculate_connection_times_dns_dialup_slowest (line 294) | fn test_calculate_connection_times_dns_dialup_slowest() {
function test_calculate_connection_times_dns_lookup_average (line 300) | fn test_calculate_connection_times_dns_lookup_average() {
function test_calculate_connection_times_dns_lookup_fastest (line 306) | fn test_calculate_connection_times_dns_lookup_fastest() {
function test_calculate_connection_times_dns_lookup_slowest (line 312) | fn test_calculate_connection_times_dns_lookup_slowest() {
FILE: src/timescale.rs
type TimeScale (line 4) | pub enum TimeScale {
method fmt (line 19) | fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
method as_secs_f64 (line 58) | pub fn as_secs_f64(&self) -> f64 {
method from_elapsed (line 68) | pub fn from_elapsed(duration: Duration) -> Self {
method inc (line 84) | pub fn inc(&self) -> Self {
method dec (line 94) | pub fn dec(&self) -> Self {
type TimeLabel (line 13) | pub struct TimeLabel {
method fmt (line 31) | fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
function assert_timescale_correct_for_seconds_range (line 109) | fn assert_timescale_correct_for_seconds_range(
function test_timescale_ranges (line 124) | fn test_timescale_ranges() {
function test_timescale_inc (line 158) | fn test_timescale_inc() {
function test_timescale_dec (line 171) | fn test_timescale_dec() {
FILE: src/tls_config.rs
type RuslsConfigs (line 2) | pub struct RuslsConfigs {
method new (line 9) | pub fn new(
method config (line 57) | pub fn config(&self, is_http2: bool) -> &std::sync::Arc<rustls::Client...
type NativeTlsConnectors (line 67) | pub struct NativeTlsConnectors {
method new (line 74) | pub fn new(
method connector (line 116) | pub fn connector(&self, is_http2: bool) -> &tokio_native_tls::TlsConne...
type AcceptAnyServerCert (line 128) | pub struct AcceptAnyServerCert;
method verify_server_cert (line 132) | fn verify_server_cert(
method verify_tls12_signature (line 143) | fn verify_tls12_signature(
method verify_tls13_signature (line 152) | fn verify_tls13_signature(
method supported_verify_schemes (line 161) | fn supported_verify_schemes(&self) -> Vec<rustls::SignatureScheme> {
FILE: src/url_generator.rs
type UrlGenerator (line 9) | pub enum UrlGenerator {
method new_static (line 28) | pub fn new_static(url: Url) -> Self {
method new_multi_static (line 32) | pub fn new_multi_static(urls: Vec<Url>) -> Self {
method new_dynamic (line 37) | pub fn new_dynamic(regex: Regex) -> Self {
method generate (line 41) | pub fn generate<R: Rng>(&self, rng: &mut R) -> Result<Cow<Url>, UrlGen...
type UrlGeneratorError (line 16) | pub enum UrlGeneratorError {
function test_url_generator_static (line 73) | fn test_url_generator_static() {
function test_url_generator_multistatic (line 81) | fn test_url_generator_multistatic() {
function test_url_generator_dynamic (line 99) | fn test_url_generator_dynamic() {
function test_url_generator_dynamic_consistency (line 115) | fn test_url_generator_dynamic_consistency() {
function test_url_generator_multi_consistency (line 131) | fn test_url_generator_multi_consistency() {
FILE: tests/tests.rs
function bind_port (line 26) | async fn bind_port() -> (tokio::net::TcpListener, u16) {
function bind_port_ipv6 (line 33) | async fn bind_port_ipv6() -> (tokio::net::TcpListener, u16) {
function get_req (line 40) | async fn get_req(path: &str, args: &[&str]) -> Request<hyper::body::Inco...
function redirect (line 101) | async fn redirect(n: usize, is_relative: bool, limit: usize) -> bool {
function get_host_with_connect_to (line 137) | async fn get_host_with_connect_to(host: &'static str) -> String {
function get_host_with_connect_to_ipv6_target (line 168) | async fn get_host_with_connect_to_ipv6_target(host: &'static str) -> Str...
function get_host_with_connect_to_ipv6_requested (line 198) | async fn get_host_with_connect_to_ipv6_requested() -> String {
function get_host_with_connect_to_redirect (line 228) | async fn get_host_with_connect_to_redirect(host: &'static str) -> String {
function burst_10_req_delay_2s_rate_4 (line 263) | async fn burst_10_req_delay_2s_rate_4(iteration: u8, args: &[&str]) -> u...
function distribution_on_two_matching_connect_to (line 306) | async fn distribution_on_two_matching_connect_to(host: &'static str) -> ...
function test_enable_compression_default (line 362) | async fn test_enable_compression_default() {
function test_setting_custom_header (line 391) | async fn test_setting_custom_header() {
function test_setting_accept_header (line 404) | async fn test_setting_accept_header() {
function test_setting_body (line 429) | async fn test_setting_body() {
function test_setting_content_type_header (line 444) | async fn test_setting_content_type_header() {
function test_setting_method (line 469) | async fn test_setting_method() {
function test_query (line 547) | async fn test_query() {
function test_query_rand_regex (line 566) | async fn test_query_rand_regex() {
function test_redirect (line 595) | async fn test_redirect() {
function test_connect_to (line 607) | async fn test_connect_to() {
function test_connect_to_randomness (line 615) | async fn test_connect_to_randomness() {
function test_connect_to_ipv6_target (line 622) | async fn test_connect_to_ipv6_target() {
function test_connect_to_ipv6_requested (line 630) | async fn test_connect_to_ipv6_requested() {
function test_connect_to_redirect (line 635) | async fn test_connect_to_redirect() {
function test_ipv6 (line 643) | async fn test_ipv6() {
function test_query_limit (line 672) | async fn test_query_limit() {
function test_http2 (line 678) | async fn test_http2() {
function test_unix_socket (line 692) | async fn test_unix_socket() {
function make_root_cert (line 737) | fn make_root_cert() -> rcgen::CertifiedKey {
function bind_proxy (line 757) | async fn bind_proxy<S>(service: S, http2: bool) -> (u16, impl Future<Out...
function test_proxy_with_setting (line 813) | async fn test_proxy_with_setting(https: bool, http2: bool, proxy_http2: ...
function test_proxy (line 853) | async fn test_proxy() {
function test_google (line 864) | fn test_google() {
function test_json_schema (line 875) | async fn test_json_schema() {
function setup_mtls_server (line 936) | fn setup_mtls_server(
function test_mtls (line 1001) | async fn test_mtls() {
Condensed preview — 36 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (272K chars).
[
{
"path": ".dockerignore",
"chars": 48,
"preview": ".git/\n.github/\n.vscode/\n\ndocs/\nscripts/\ntarget/\n"
},
{
"path": ".editorconfig",
"chars": 493,
"preview": "root = true\n\n[*]\ncharset = utf-8\nend_of_line = lf\ninsert_final_newline = true\ntrim_trailing_whitespace = true\nindent_sty"
},
{
"path": ".github/ISSUE_TEMPLATE/bug_report.md",
"chars": 579,
"preview": "---\nname: \"Bug report\"\nabout: Create a bug report for spiko.\ntitle: \"\"\nlabels: bug\nassignees: \"trinhminhtriet\"\n---\n\n<!--"
},
{
"path": ".github/dependabot.yml",
"chars": 207,
"preview": "version: 2\nupdates:\n - package-ecosystem: \"cargo\"\n directory: \"/\"\n schedule:\n interval: \"monthly\"\n - packag"
},
{
"path": ".github/workflows/docker.yml",
"chars": 1801,
"preview": "name: Create and publish a Docker image\n\non:\n push:\n tags:\n - \"v*\"\n\n pull_request:\n types: [closed]\n\nenv:\n "
},
{
"path": ".github/workflows/publish.yml",
"chars": 561,
"preview": "name: Publish to crates.io\n\non:\n push:\n tags:\n - \"v*\"\n\nenv:\n CARGO_TERM_COLOR: always\n\njobs:\n publish:\n na"
},
{
"path": ".github/workflows/release.yml",
"chars": 2476,
"preview": "name: Release\n\non:\n push:\n tags:\n - \"v*\"\n\nenv:\n CARGO_TERM_COLOR: always\n\njobs:\n release:\n name: \"Release\""
},
{
"path": ".gitignore",
"chars": 19,
"preview": ".DS_Store\n\ntarget/\n"
},
{
"path": ".vscode/settings.json",
"chars": 35,
"preview": "{\n \"editor.formatOnSave\": true\n}"
},
{
"path": "CHANGELOG.md",
"chars": 1089,
"preview": "# Changelog\n\n### [v0.1.5](https://github.com/trinhminhtriet/spiko/compare/v0.1.4...v0.1.5) (2024-12-02)\n\n- chore(deps): "
},
{
"path": "Cargo.toml",
"chars": 3237,
"preview": "[package]\nname = \"spiko\"\nversion = \"0.1.16\"\nauthors = [\"Triet Trinh <contact@trinhminhtriet.com>\"]\nedition = \"2021\"\nlice"
},
{
"path": "Cross.toml",
"chars": 192,
"preview": "# For Asahi linux\n[target.aarch64-unknown-linux-gnu.env]\npassthrough = [\"JEMALLOC_SYS_WITH_LG_PAGE=16\"]\n\n[target.aarch64"
},
{
"path": "Dockerfile",
"chars": 542,
"preview": "FROM rust:1.84.0-bookworm AS builder\n\n# Install dependencies including LLVM 14 first\nRUN apt-get update && apt-get insta"
},
{
"path": "LICENSE",
"chars": 1100,
"preview": "MIT License\n\nCopyright (c) 2020 hatoo\nCopyright (c) 2024 trinhminhtriet.com\n\nPermission is hereby granted, free of charg"
},
{
"path": "Makefile",
"chars": 786,
"preview": "NAME := spiko\nAUTHOR := trinhminhtriet\nDATE := $(shell date +%FT%T%Z)\nGIT := $(shell [ -d .git ] && git rev-p"
},
{
"path": "README.md",
"chars": 6415,
"preview": "# 🚀 spiko\n\n```text\n\n _ _\n ___ _ __ (_)| | __ ___\n/ __|| '_ \\ | || |/ / / _ \\\n\\__ \\| |_) || || < | (_) "
},
{
"path": "demo.tape",
"chars": 343,
"preview": "Output media/demo.gif\n\nSet FontSize 16\nSet Width 1440\nSet Height 768\nSet TypingSpeed 400ms\n\nType@100ms \"echo 'https://tr"
},
{
"path": "pgo/server/Cargo.toml",
"chars": 237,
"preview": "[package]\nname = \"server\"\nversion = \"0.1.0\"\nedition = \"2021\"\n\n# See more keys and their definitions at https://doc.rust-"
},
{
"path": "pgo/server/src/main.rs",
"chars": 570,
"preview": "use std::net::SocketAddr;\nuse tokio::net::TcpListener;\n\nuse axum::{routing::get, Router};\n\n#[tokio::main]\nasync fn main("
},
{
"path": "pgo.js",
"chars": 485,
"preview": "import { $ } from \"bun\";\n\nlet additional = [];\n\nif (Bun.argv.length >= 3) {\n additional = Bun.argv.slice(2);\n}\n\nlet s"
},
{
"path": "schema.json",
"chars": 8765,
"preview": "{\n \"$schema\": \"http://json-schema.org/draft-04/schema#\",\n \"description\": \"JSON schema for the output of the `spiko -j`"
},
{
"path": "scripts/release-version.sh",
"chars": 733,
"preview": "#!/bin/bash\nset -xe\n\n[ -z \"$(git status --porcelain)\" ] || (echo \"dirty working directory\" && exit 1)\n\ncurrent_version=\""
},
{
"path": "src/aws_auth.rs",
"chars": 3565,
"preview": "use crate::client::ClientError;\nuse anyhow::Result;\n\nuse bytes::Bytes;\nuse hyper::{\n HeaderMap,\n header::{self, He"
},
{
"path": "src/client.rs",
"chars": 93994,
"preview": "use bytes::Bytes;\nuse http_body_util::{BodyExt, Full};\nuse hyper::{Method, http};\nuse hyper_util::rt::{TokioExecutor, To"
},
{
"path": "src/db.rs",
"chars": 2221,
"preview": "use rusqlite::Connection;\n\nuse crate::client::{Client, RequestResult};\n\nfn create_db(conn: &Connection) -> Result<usize,"
},
{
"path": "src/histogram.rs",
"chars": 2327,
"preview": "pub fn histogram(values: &[f64], bins: usize) -> Vec<(f64, usize)> {\n assert!(bins >= 2);\n let mut bucket: Vec<usi"
},
{
"path": "src/lib.rs",
"chars": 33991,
"preview": "use anyhow::Context;\nuse aws_auth::AwsSignatureConfig;\nuse clap::Parser;\nuse crossterm::tty::IsTty;\nuse hickory_resolver"
},
{
"path": "src/main.rs",
"chars": 634,
"preview": "use clap::Parser;\nuse spiko::{run, Opts};\n\nfn main() {\n let num_workers_threads = std::env::var(\"TOKIO_WORKER_THREADS"
},
{
"path": "src/monitor.rs",
"chars": 16673,
"preview": "use byte_unit::Byte;\nuse crossterm::event::{Event, KeyCode, KeyEvent, KeyModifiers};\nuse hyper::http;\nuse ratatui::{Defa"
},
{
"path": "src/pcg64si.rs",
"chars": 2293,
"preview": "// https://github.com/imneme/pcg-c\nuse rand::{RngCore, SeedableRng};\nuse rand_core::impls;\n\n#[derive(Debug, Copy, Clone)"
},
{
"path": "src/printer.rs",
"chars": 19255,
"preview": "use crate::result_data::ResultData;\nuse average::{Max, Min, Variance};\nuse byte_unit::Byte;\nuse crossterm::style::{Style"
},
{
"path": "src/result_data.rs",
"chars": 9392,
"preview": "use std::{\n collections::BTreeMap,\n time::{Duration, Instant},\n};\n\nuse average::{Estimate, Max, Mean, Min, concate"
},
{
"path": "src/timescale.rs",
"chars": 5573,
"preview": "use std::{fmt, time::Duration};\n\n#[derive(Clone, Copy, PartialEq, Eq, Debug)]\npub enum TimeScale {\n Second,\n TenSe"
},
{
"path": "src/tls_config.rs",
"chars": 5420,
"preview": "#[cfg(feature = \"rustls\")]\npub struct RuslsConfigs {\n no_alpn: std::sync::Arc<rustls::ClientConfig>,\n alpn_h2: std"
},
{
"path": "src/url_generator.rs",
"chars": 4480,
"preview": "use std::{borrow::Cow, string::FromUtf8Error};\n\nuse rand::prelude::*;\nuse rand_regex::Regex;\nuse thiserror::Error;\nuse u"
},
{
"path": "tests/tests.rs",
"chars": 29818,
"preview": "use std::{\n convert::Infallible,\n error::Error as StdError,\n fs::File,\n future::Future,\n io::Write,\n n"
}
]
About this extraction
This page contains the full source code of the trinhminhtriet/spiko GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 36 files (254.2 KB), approximately 57.4k tokens, and a symbol index with 212 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.