Showing preview only (243K chars total). Download the full file or copy to clipboard to get everything.
Repository: wg/wrk
Branch: master
Commit: a211dd5a7050
Files: 46
Total size: 230.7 KB
Directory structure:
gitextract_fdoud_vu/
├── .github/
│ └── workflows/
│ └── build.yml
├── .gitignore
├── CHANGES
├── INSTALL
├── LICENSE
├── Makefile
├── NOTICE
├── README.md
├── SCRIPTING
├── scripts/
│ ├── addr.lua
│ ├── auth.lua
│ ├── counter.lua
│ ├── delay.lua
│ ├── pipeline.lua
│ ├── post.lua
│ ├── report.lua
│ ├── setup.lua
│ └── stop.lua
└── src/
├── ae.c
├── ae.h
├── ae_epoll.c
├── ae_evport.c
├── ae_kqueue.c
├── ae_select.c
├── aprintf.c
├── aprintf.h
├── atomicvar.h
├── config.h
├── http_parser.c
├── http_parser.h
├── main.h
├── net.c
├── net.h
├── script.c
├── script.h
├── ssl.c
├── ssl.h
├── stats.c
├── stats.h
├── units.c
├── units.h
├── wrk.c
├── wrk.h
├── wrk.lua
├── zmalloc.c
└── zmalloc.h
================================================
FILE CONTENTS
================================================
================================================
FILE: .github/workflows/build.yml
================================================
name: build
on:
push:
jobs:
build:
runs-on: ${{ matrix.builder }}
strategy:
matrix:
builder: [macos-latest, ubuntu-latest]
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- name: build
run: make
================================================
FILE: .gitignore
================================================
obj/
wrk
================================================
FILE: CHANGES
================================================
master
* Require OpenSSL 1.1.0+
wrk 4.0.2
* Send hostname using TLS SNI.
* Add optional WITH_OPENSSL and WITH_LUAJIT to use system libs.
* Bundle OpenSSL 1.0.2.
* delay() can return milliseconds to delay sending next request.
wrk 4.0.0
* The wrk global variable is the only global defined by default.
* wrk.init() calls the global init(), remove calls to wrk.init().
* Add wrk.lookup(host, port) and wrk.connect(addr) functions.
* Add setup phase that calls the global setup() for each thread.
* Allow assignment to thread.addr to specify the server address.
* Add thread:set(name, value), thread:get(name), and thread:stop().
* Record latency for every request instead of random samples.
* Latency and requests in done() are now callable, not indexable.
* Only record timeouts when a response is actually received.
* Remove calibration phase and record rate at fixed interval.
* Improve correction of coordinated omission.
================================================
FILE: INSTALL
================================================
Overview
wrk should build on most UNIX-like operating systems and
architectures that have GNU make and are supported by LuaJIT and
OpenSSL. Some systems may require additional CFLAGS or LDFLAGS,
see the top of the Makefile for examples
In many cases simply running `make` (often `gmake` on *BSD) will
do the trick.
Dependencies
wrk requires LuaJIT and OpenSSL and is distributed with appropriate
versions that will be unpacked and built as necessary.
If you are building wrk packages for an OS distribution or otherwise
prefer to use system versions of dependencies you may specify their
location when invoking make with one or more of:
WITH_LUAJIT
WITH_OPENSSL
For example to use the system version of both libraries on Linux:
make WITH_LUAJIT=/usr WITH_OPENSSL=/usr
Or to use the Homebrew version of OpenSSL on Mac OS X:
make WITH_OPENSSL=/usr/local/opt/openssl
================================================
FILE: LICENSE
================================================
Modified Apache 2.0 License
Version 2.0.1, February 2015
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
(e) If the Derivative Work includes substantial changes to features
or functionality of the Work, then you must remove the name of
the Work, and any derivation thereof, from all copies that you
distribute, whether in Source or Object form, except as required
in copyright, patent, trademark, and attribution notices.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
================================================
FILE: Makefile
================================================
CFLAGS += -std=c99 -Wall -O2 -D_REENTRANT
LIBS := -lm -lssl -lcrypto -lpthread
TARGET := $(shell uname -s | tr '[A-Z]' '[a-z]' 2>/dev/null || echo unknown)
ifeq ($(TARGET), sunos)
CFLAGS += -D_PTHREADS -D_POSIX_C_SOURCE=200112L
LIBS += -lsocket
else ifeq ($(TARGET), darwin)
export MACOSX_DEPLOYMENT_TARGET = $(shell sw_vers -productVersion)
else ifeq ($(TARGET), linux)
CFLAGS += -D_POSIX_C_SOURCE=200112L -D_BSD_SOURCE -D_DEFAULT_SOURCE
LIBS += -ldl
LDFLAGS += -Wl,-E
else ifeq ($(TARGET), freebsd)
CFLAGS += -D_DECLARE_C99_LDBL_MATH
LDFLAGS += -Wl,-E
endif
SRC := wrk.c net.c ssl.c aprintf.c stats.c script.c units.c \
ae.c zmalloc.c http_parser.c
BIN := wrk
VER ?= $(shell git describe --tags --always --dirty)
ODIR := obj
OBJ := $(patsubst %.c,$(ODIR)/%.o,$(SRC)) $(ODIR)/bytecode.o $(ODIR)/version.o
LIBS := -lluajit-5.1 $(LIBS)
DEPS :=
CFLAGS += -I$(ODIR)/include
LDFLAGS += -L$(ODIR)/lib
ifneq ($(WITH_LUAJIT),)
CFLAGS += -I$(WITH_LUAJIT)/include
LDFLAGS += -L$(WITH_LUAJIT)/lib
else
CFLAGS += -I$(ODIR)/include/luajit-2.1
DEPS += $(ODIR)/lib/libluajit-5.1.a
endif
ifneq ($(WITH_OPENSSL),)
CFLAGS += -I$(WITH_OPENSSL)/include
LDFLAGS += -L$(WITH_OPENSSL)/lib
else
DEPS += $(ODIR)/lib/libssl.a
endif
all: $(BIN)
clean:
$(RM) -rf $(BIN) obj/*
$(BIN): $(OBJ)
@echo LINK $(BIN)
@$(CC) $(LDFLAGS) -o $@ $^ $(LIBS)
$(OBJ): config.h Makefile $(DEPS) | $(ODIR)
$(ODIR):
@mkdir -p $@
$(ODIR)/bytecode.c: src/wrk.lua $(DEPS)
@echo LUAJIT $<
@$(SHELL) -c 'PATH="obj/bin:$(PATH)" luajit -b "$(CURDIR)/$<" "$(CURDIR)/$@"'
$(ODIR)/version.o:
@echo 'const char *VERSION="$(VER)";' | $(CC) -xc -c -o $@ -
$(ODIR)/%.o : %.c
@echo CC $<
@$(CC) $(CFLAGS) -c -o $@ $<
# Dependencies
LUAJIT := $(notdir $(patsubst %.zip,%,$(wildcard deps/LuaJIT*.zip)))
OPENSSL := $(notdir $(patsubst %.tar.gz,%,$(wildcard deps/openssl*.tar.gz)))
OPENSSL_OPTS = no-shared no-psk no-srp no-dtls no-idea --prefix=$(abspath $(ODIR))
$(ODIR)/$(LUAJIT): deps/$(LUAJIT).zip | $(ODIR)
echo $(LUAJIT)
@unzip -nd $(ODIR) $<
$(ODIR)/$(OPENSSL): deps/$(OPENSSL).tar.gz | $(ODIR)
@tar -C $(ODIR) -xf $<
$(ODIR)/lib/libluajit-5.1.a: $(ODIR)/$(LUAJIT)
@echo Building LuaJIT...
@$(MAKE) -C $< PREFIX=$(abspath $(ODIR)) BUILDMODE=static install
@cd $(ODIR)/bin && ln -s luajit-2.1.0-beta3 luajit
$(ODIR)/lib/libssl.a: $(ODIR)/$(OPENSSL)
@echo Building OpenSSL...
@$(SHELL) -c "cd $< && ./config $(OPENSSL_OPTS)"
@$(MAKE) -C $< depend
@$(MAKE) -C $<
@$(MAKE) -C $< install_sw
@touch $@
# ------------
.PHONY: all clean
.PHONY: $(ODIR)/version.o
.SUFFIXES:
.SUFFIXES: .c .o .lua
vpath %.c src
vpath %.h src
vpath %.lua scripts
================================================
FILE: NOTICE
================================================
=========================================================================
== NOTICE file corresponding to section 4(d) of the Apache License, ==
== Version 2.0, in this case for the wrk distribution. ==
=========================================================================
wrk
Copyright 2012 Will Glozer, http://glozer.net
=========================================================================
== Redis Event Library Notice ==
=========================================================================
This product includes software developed by Salvatore Sanfilippo and
other contributors to the redis project.
Copyright (c) 2006-2010, Salvatore Sanfilippo <antirez at gmail dot com>
Copyright (c) 2009-2012, Salvatore Sanfilippo <antirez at gmail dot com>
Copyright (C) 2009 Harish Mallipeddi - harish.mallipeddi@gmail.com
Copyright (c) 2012, Joyent, Inc. All rights reserved.
Copyright (c) 2006-2009, Salvatore Sanfilippo
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the *
distribution.
* Neither the name of Redis nor the names of its contributors may be
used to endorse or promote products derived from this software
without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
=========================================================================
== HTTP Parser Notice ==
=========================================================================
This product includes software developed by Igor Sysoev, Joyent, Inc.,
and other Node contributors.
http_parser.c is based on src/http/ngx_http_parse.c from NGINX copyright
Igor Sysoev.
Additional changes are licensed under the same terms as NGINX and
copyright Joyent, Inc. and other Node contributors. All rights reserved.
Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to permit
persons to whom the Software is furnished to do so, subject to the
following conditions:
The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN
NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
USE OR OTHER DEALINGS IN THE SOFTWARE.
=========================================================================
== LuaJIT Notice ==
=========================================================================
LuaJIT -- a Just-In-Time Compiler for Lua. http://luajit.org/
Copyright (C) 2005-2013 Mike Pall. All rights reserved.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
================================================
FILE: README.md
================================================
# wrk - a HTTP benchmarking tool
wrk is a modern HTTP benchmarking tool capable of generating significant
load when run on a single multi-core CPU. It combines a multithreaded
design with scalable event notification systems such as epoll and kqueue.
An optional LuaJIT script can perform HTTP request generation, response
processing, and custom reporting. Details are available in SCRIPTING and
several examples are located in [scripts/](scripts/).
## Basic Usage
wrk -t12 -c400 -d30s http://127.0.0.1:8080/index.html
This runs a benchmark for 30 seconds, using 12 threads, and keeping
400 HTTP connections open.
Output:
Running 30s test @ http://127.0.0.1:8080/index.html
12 threads and 400 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 635.91us 0.89ms 12.92ms 93.69%
Req/Sec 56.20k 8.07k 62.00k 86.54%
22464657 requests in 30.00s, 17.76GB read
Requests/sec: 748868.53
Transfer/sec: 606.33MB
## Command Line Options
-c, --connections: total number of HTTP connections to keep open with
each thread handling N = connections/threads
-d, --duration: duration of the test, e.g. 2s, 2m, 2h
-t, --threads: total number of threads to use
-s, --script: LuaJIT script, see SCRIPTING
-H, --header: HTTP header to add to request, e.g. "User-Agent: wrk"
--latency: print detailed latency statistics
--timeout: record a timeout if a response is not received within
this amount of time.
## Benchmarking Tips
The machine running wrk must have a sufficient number of ephemeral ports
available and closed sockets should be recycled quickly. To handle the
initial connection burst the server's listen(2) backlog should be greater
than the number of concurrent connections being tested.
A user script that only changes the HTTP method, path, adds headers or
a body, will have no performance impact. Per-request actions, particularly
building a new HTTP request, and use of response() will necessarily reduce
the amount of load that can be generated.
## Acknowledgements
wrk contains code from a number of open source projects including the
'ae' event loop from redis, the nginx/joyent/node.js 'http-parser',
and Mike Pall's LuaJIT. Please consult the NOTICE file for licensing
details.
## Cryptography Notice
This distribution includes cryptographic software. The country in
which you currently reside may have restrictions on the import,
possession, use, and/or re-export to another country, of encryption
software. BEFORE using any encryption software, please check your
country's laws, regulations and policies concerning the import,
possession, or use, and re-export of encryption software, to see if
this is permitted. See <http://www.wassenaar.org/> for more
information.
The U.S. Government Department of Commerce, Bureau of Industry and
Security (BIS), has classified this software as Export Commodity
Control Number (ECCN) 5D002.C.1, which includes information security
software using or performing cryptographic functions with symmetric
algorithms. The form and manner of this distribution makes it
eligible for export under the License Exception ENC Technology
Software Unrestricted (TSU) exception (see the BIS Export
Administration Regulations, Section 740.13) for both object code and
source code.
================================================
FILE: SCRIPTING
================================================
Overview
wrk supports executing a LuaJIT script during three distinct phases: setup,
running, and done. Each wrk thread has an independent scripting environment
and the setup & done phases execute in a separate environment which does
not participate in the running phase.
The public Lua API consists of a global table and a number of global
functions:
wrk = {
scheme = "http",
host = "localhost",
port = nil,
method = "GET",
path = "/",
headers = {},
body = nil,
thread = <userdata>,
}
function wrk.format(method, path, headers, body)
wrk.format returns a HTTP request string containing the passed parameters
merged with values from the wrk table.
function wrk.lookup(host, service)
wrk.lookup returns a table containing all known addresses for the host
and service pair. This corresponds to the POSIX getaddrinfo() function.
function wrk.connect(addr)
wrk.connect returns true if the address can be connected to, otherwise
it returns false. The address must be one returned from wrk.lookup().
The following globals are optional, and if defined must be functions:
global setup -- called during thread setup
global init -- called when the thread is starting
global delay -- called to get the request delay
global request -- called to generate the HTTP request
global response -- called with HTTP response data
global done -- called with results of run
Setup
function setup(thread)
The setup phase begins after the target IP address has been resolved and all
threads have been initialized but not yet started.
setup() is called once for each thread and receives a userdata object
representing the thread.
thread.addr - get or set the thread's server address
thread:get(name) - get the value of a global in the thread's env
thread:set(name, value) - set the value of a global in the thread's env
thread:stop() - stop the thread
Only boolean, nil, number, and string values or tables of the same may be
transfered via get()/set() and thread:stop() can only be called while the
thread is running.
Running
function init(args)
function delay()
function request()
function response(status, headers, body)
The running phase begins with a single call to init(), followed by
a call to request() and response() for each request cycle.
The init() function receives any extra command line arguments for the
script which must be separated from wrk arguments with "--".
delay() returns the number of milliseconds to delay sending the next
request.
request() returns a string containing the HTTP request. Building a new
request each time is expensive, when testing a high performance server
one solution is to pre-generate all requests in init() and do a quick
lookup in request().
response() is called with the HTTP response status, headers, and body.
Parsing the headers and body is expensive, so if the response global is
nil after the call to init() wrk will ignore the headers and body.
Done
function done(summary, latency, requests)
The done() function receives a table containing result data, and two
statistics objects representing the per-request latency and per-thread
request rate. Duration and latency are microsecond values and rate is
measured in requests per second.
latency.min -- minimum value seen
latency.max -- maximum value seen
latency.mean -- average value seen
latency.stdev -- standard deviation
latency:percentile(99.0) -- 99th percentile value
latency(i) -- raw value and count
summary = {
duration = N, -- run duration in microseconds
requests = N, -- total completed requests
bytes = N, -- total bytes received
errors = {
connect = N, -- total socket connection errors
read = N, -- total socket read errors
write = N, -- total socket write errors
status = N, -- total HTTP status codes > 399
timeout = N -- total request timeouts
}
}
================================================
FILE: scripts/addr.lua
================================================
-- example script that demonstrates use of setup() to pass
-- a random server address to each thread
local addrs = nil
function setup(thread)
if not addrs then
addrs = wrk.lookup(wrk.host, wrk.port or "http")
for i = #addrs, 1, -1 do
if not wrk.connect(addrs[i]) then
table.remove(addrs, i)
end
end
end
thread.addr = addrs[math.random(#addrs)]
end
function init(args)
local msg = "thread addr: %s"
print(msg:format(wrk.thread.addr))
end
================================================
FILE: scripts/auth.lua
================================================
-- example script that demonstrates response handling and
-- retrieving an authentication token to set on all future
-- requests
token = nil
path = "/authenticate"
request = function()
return wrk.format("GET", path)
end
response = function(status, headers, body)
if not token and status == 200 then
token = headers["X-Token"]
path = "/resource"
wrk.headers["X-Token"] = token
end
end
================================================
FILE: scripts/counter.lua
================================================
-- example dynamic request script which demonstrates changing
-- the request path and a header for each request
-------------------------------------------------------------
-- NOTE: each wrk thread has an independent Lua scripting
-- context and thus there will be one counter per thread
counter = 0
request = function()
path = "/" .. counter
wrk.headers["X-Counter"] = counter
counter = counter + 1
return wrk.format(nil, path)
end
================================================
FILE: scripts/delay.lua
================================================
-- example script that demonstrates adding a random
-- 10-50ms delay before each request
function delay()
return math.random(10, 50)
end
================================================
FILE: scripts/pipeline.lua
================================================
-- example script demonstrating HTTP pipelining
init = function(args)
local r = {}
r[1] = wrk.format(nil, "/?foo")
r[2] = wrk.format(nil, "/?bar")
r[3] = wrk.format(nil, "/?baz")
req = table.concat(r)
end
request = function()
return req
end
================================================
FILE: scripts/post.lua
================================================
-- example HTTP POST script which demonstrates setting the
-- HTTP method, body, and adding a header
wrk.method = "POST"
wrk.body = "foo=bar&baz=quux"
wrk.headers["Content-Type"] = "application/x-www-form-urlencoded"
================================================
FILE: scripts/report.lua
================================================
-- example reporting script which demonstrates a custom
-- done() function that prints latency percentiles as CSV
done = function(summary, latency, requests)
io.write("------------------------------\n")
for _, p in pairs({ 50, 90, 99, 99.999 }) do
n = latency:percentile(p)
io.write(string.format("%g%%,%d\n", p, n))
end
end
================================================
FILE: scripts/setup.lua
================================================
-- example script that demonstrates use of setup() to pass
-- data to and from the threads
local counter = 1
local threads = {}
function setup(thread)
thread:set("id", counter)
table.insert(threads, thread)
counter = counter + 1
end
function init(args)
requests = 0
responses = 0
local msg = "thread %d created"
print(msg:format(id))
end
function request()
requests = requests + 1
return wrk.request()
end
function response(status, headers, body)
responses = responses + 1
end
function done(summary, latency, requests)
for index, thread in ipairs(threads) do
local id = thread:get("id")
local requests = thread:get("requests")
local responses = thread:get("responses")
local msg = "thread %d made %d requests and got %d responses"
print(msg:format(id, requests, responses))
end
end
================================================
FILE: scripts/stop.lua
================================================
-- example script that demonstrates use of thread:stop()
local counter = 1
function response()
if counter == 100 then
wrk.thread:stop()
end
counter = counter + 1
end
================================================
FILE: src/ae.c
================================================
/* A simple event-driven programming library. Originally I wrote this code
* for the Jim's event-loop (Jim is a Tcl interpreter) but later translated
* it in form of a library for easy reuse.
*
* Copyright (c) 2006-2010, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include <stdio.h>
#include <sys/time.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdlib.h>
#include <poll.h>
#include <string.h>
#include <time.h>
#include <errno.h>
#include "ae.h"
#include "zmalloc.h"
#include "config.h"
/* Include the best multiplexing layer supported by this system.
* The following should be ordered by performances, descending. */
#ifdef HAVE_EVPORT
#include "ae_evport.c"
#else
#ifdef HAVE_EPOLL
#include "ae_epoll.c"
#else
#ifdef HAVE_KQUEUE
#include "ae_kqueue.c"
#else
#include "ae_select.c"
#endif
#endif
#endif
aeEventLoop *aeCreateEventLoop(int setsize) {
aeEventLoop *eventLoop;
int i;
if ((eventLoop = zmalloc(sizeof(*eventLoop))) == NULL) goto err;
eventLoop->events = zmalloc(sizeof(aeFileEvent)*setsize);
eventLoop->fired = zmalloc(sizeof(aeFiredEvent)*setsize);
if (eventLoop->events == NULL || eventLoop->fired == NULL) goto err;
eventLoop->setsize = setsize;
eventLoop->lastTime = time(NULL);
eventLoop->timeEventHead = NULL;
eventLoop->timeEventNextId = 0;
eventLoop->stop = 0;
eventLoop->maxfd = -1;
eventLoop->beforesleep = NULL;
if (aeApiCreate(eventLoop) == -1) goto err;
/* Events with mask == AE_NONE are not set. So let's initialize the
* vector with it. */
for (i = 0; i < setsize; i++)
eventLoop->events[i].mask = AE_NONE;
return eventLoop;
err:
if (eventLoop) {
zfree(eventLoop->events);
zfree(eventLoop->fired);
zfree(eventLoop);
}
return NULL;
}
/* Return the current set size. */
int aeGetSetSize(aeEventLoop *eventLoop) {
return eventLoop->setsize;
}
/* Resize the maximum set size of the event loop.
* If the requested set size is smaller than the current set size, but
* there is already a file descriptor in use that is >= the requested
* set size minus one, AE_ERR is returned and the operation is not
* performed at all.
*
* Otherwise AE_OK is returned and the operation is successful. */
int aeResizeSetSize(aeEventLoop *eventLoop, int setsize) {
int i;
if (setsize == eventLoop->setsize) return AE_OK;
if (eventLoop->maxfd >= setsize) return AE_ERR;
if (aeApiResize(eventLoop,setsize) == -1) return AE_ERR;
eventLoop->events = zrealloc(eventLoop->events,sizeof(aeFileEvent)*setsize);
eventLoop->fired = zrealloc(eventLoop->fired,sizeof(aeFiredEvent)*setsize);
eventLoop->setsize = setsize;
/* Make sure that if we created new slots, they are initialized with
* an AE_NONE mask. */
for (i = eventLoop->maxfd+1; i < setsize; i++)
eventLoop->events[i].mask = AE_NONE;
return AE_OK;
}
void aeDeleteEventLoop(aeEventLoop *eventLoop) {
aeApiFree(eventLoop);
zfree(eventLoop->events);
zfree(eventLoop->fired);
zfree(eventLoop);
}
void aeStop(aeEventLoop *eventLoop) {
eventLoop->stop = 1;
}
int aeCreateFileEvent(aeEventLoop *eventLoop, int fd, int mask,
aeFileProc *proc, void *clientData)
{
if (fd >= eventLoop->setsize) {
errno = ERANGE;
return AE_ERR;
}
aeFileEvent *fe = &eventLoop->events[fd];
if (aeApiAddEvent(eventLoop, fd, mask) == -1)
return AE_ERR;
fe->mask |= mask;
if (mask & AE_READABLE) fe->rfileProc = proc;
if (mask & AE_WRITABLE) fe->wfileProc = proc;
fe->clientData = clientData;
if (fd > eventLoop->maxfd)
eventLoop->maxfd = fd;
return AE_OK;
}
void aeDeleteFileEvent(aeEventLoop *eventLoop, int fd, int mask)
{
if (fd >= eventLoop->setsize) return;
aeFileEvent *fe = &eventLoop->events[fd];
if (fe->mask == AE_NONE) return;
aeApiDelEvent(eventLoop, fd, mask);
fe->mask = fe->mask & (~mask);
if (fd == eventLoop->maxfd && fe->mask == AE_NONE) {
/* Update the max fd */
int j;
for (j = eventLoop->maxfd-1; j >= 0; j--)
if (eventLoop->events[j].mask != AE_NONE) break;
eventLoop->maxfd = j;
}
}
int aeGetFileEvents(aeEventLoop *eventLoop, int fd) {
if (fd >= eventLoop->setsize) return 0;
aeFileEvent *fe = &eventLoop->events[fd];
return fe->mask;
}
static void aeGetTime(long *seconds, long *milliseconds)
{
struct timeval tv;
gettimeofday(&tv, NULL);
*seconds = tv.tv_sec;
*milliseconds = tv.tv_usec/1000;
}
static void aeAddMillisecondsToNow(long long milliseconds, long *sec, long *ms) {
long cur_sec, cur_ms, when_sec, when_ms;
aeGetTime(&cur_sec, &cur_ms);
when_sec = cur_sec + milliseconds/1000;
when_ms = cur_ms + milliseconds%1000;
if (when_ms >= 1000) {
when_sec ++;
when_ms -= 1000;
}
*sec = when_sec;
*ms = when_ms;
}
long long aeCreateTimeEvent(aeEventLoop *eventLoop, long long milliseconds,
aeTimeProc *proc, void *clientData,
aeEventFinalizerProc *finalizerProc)
{
long long id = eventLoop->timeEventNextId++;
aeTimeEvent *te;
te = zmalloc(sizeof(*te));
if (te == NULL) return AE_ERR;
te->id = id;
aeAddMillisecondsToNow(milliseconds,&te->when_sec,&te->when_ms);
te->timeProc = proc;
te->finalizerProc = finalizerProc;
te->clientData = clientData;
te->next = eventLoop->timeEventHead;
eventLoop->timeEventHead = te;
return id;
}
int aeDeleteTimeEvent(aeEventLoop *eventLoop, long long id)
{
aeTimeEvent *te = eventLoop->timeEventHead;
while(te) {
if (te->id == id) {
te->id = AE_DELETED_EVENT_ID;
return AE_OK;
}
te = te->next;
}
return AE_ERR; /* NO event with the specified ID found */
}
/* Search the first timer to fire.
* This operation is useful to know how many time the select can be
* put in sleep without to delay any event.
* If there are no timers NULL is returned.
*
* Note that's O(N) since time events are unsorted.
* Possible optimizations (not needed by Redis so far, but...):
* 1) Insert the event in order, so that the nearest is just the head.
* Much better but still insertion or deletion of timers is O(N).
* 2) Use a skiplist to have this operation as O(1) and insertion as O(log(N)).
*/
static aeTimeEvent *aeSearchNearestTimer(aeEventLoop *eventLoop)
{
aeTimeEvent *te = eventLoop->timeEventHead;
aeTimeEvent *nearest = NULL;
while(te) {
if (!nearest || te->when_sec < nearest->when_sec ||
(te->when_sec == nearest->when_sec &&
te->when_ms < nearest->when_ms))
nearest = te;
te = te->next;
}
return nearest;
}
/* Process time events */
static int processTimeEvents(aeEventLoop *eventLoop) {
int processed = 0;
aeTimeEvent *te, *prev;
long long maxId;
time_t now = time(NULL);
/* If the system clock is moved to the future, and then set back to the
* right value, time events may be delayed in a random way. Often this
* means that scheduled operations will not be performed soon enough.
*
* Here we try to detect system clock skews, and force all the time
* events to be processed ASAP when this happens: the idea is that
* processing events earlier is less dangerous than delaying them
* indefinitely, and practice suggests it is. */
if (now < eventLoop->lastTime) {
te = eventLoop->timeEventHead;
while(te) {
te->when_sec = 0;
te = te->next;
}
}
eventLoop->lastTime = now;
prev = NULL;
te = eventLoop->timeEventHead;
maxId = eventLoop->timeEventNextId-1;
while(te) {
long now_sec, now_ms;
long long id;
/* Remove events scheduled for deletion. */
if (te->id == AE_DELETED_EVENT_ID) {
aeTimeEvent *next = te->next;
if (prev == NULL)
eventLoop->timeEventHead = te->next;
else
prev->next = te->next;
if (te->finalizerProc)
te->finalizerProc(eventLoop, te->clientData);
zfree(te);
te = next;
continue;
}
/* Make sure we don't process time events created by time events in
* this iteration. Note that this check is currently useless: we always
* add new timers on the head, however if we change the implementation
* detail, this check may be useful again: we keep it here for future
* defense. */
if (te->id > maxId) {
te = te->next;
continue;
}
aeGetTime(&now_sec, &now_ms);
if (now_sec > te->when_sec ||
(now_sec == te->when_sec && now_ms >= te->when_ms))
{
int retval;
id = te->id;
retval = te->timeProc(eventLoop, id, te->clientData);
processed++;
if (retval != AE_NOMORE) {
aeAddMillisecondsToNow(retval,&te->when_sec,&te->when_ms);
} else {
te->id = AE_DELETED_EVENT_ID;
}
}
prev = te;
te = te->next;
}
return processed;
}
/* Process every pending time event, then every pending file event
* (that may be registered by time event callbacks just processed).
* Without special flags the function sleeps until some file event
* fires, or when the next time event occurs (if any).
*
* If flags is 0, the function does nothing and returns.
* if flags has AE_ALL_EVENTS set, all the kind of events are processed.
* if flags has AE_FILE_EVENTS set, file events are processed.
* if flags has AE_TIME_EVENTS set, time events are processed.
* if flags has AE_DONT_WAIT set the function returns ASAP until all
* the events that's possible to process without to wait are processed.
*
* The function returns the number of events processed. */
int aeProcessEvents(aeEventLoop *eventLoop, int flags)
{
int processed = 0, numevents;
/* Nothing to do? return ASAP */
if (!(flags & AE_TIME_EVENTS) && !(flags & AE_FILE_EVENTS)) return 0;
/* Note that we want call select() even if there are no
* file events to process as long as we want to process time
* events, in order to sleep until the next time event is ready
* to fire. */
if (eventLoop->maxfd != -1 ||
((flags & AE_TIME_EVENTS) && !(flags & AE_DONT_WAIT))) {
int j;
aeTimeEvent *shortest = NULL;
struct timeval tv, *tvp;
if (flags & AE_TIME_EVENTS && !(flags & AE_DONT_WAIT))
shortest = aeSearchNearestTimer(eventLoop);
if (shortest) {
long now_sec, now_ms;
aeGetTime(&now_sec, &now_ms);
tvp = &tv;
/* How many milliseconds we need to wait for the next
* time event to fire? */
long long ms =
(shortest->when_sec - now_sec)*1000 +
shortest->when_ms - now_ms;
if (ms > 0) {
tvp->tv_sec = ms/1000;
tvp->tv_usec = (ms % 1000)*1000;
} else {
tvp->tv_sec = 0;
tvp->tv_usec = 0;
}
} else {
/* If we have to check for events but need to return
* ASAP because of AE_DONT_WAIT we need to set the timeout
* to zero */
if (flags & AE_DONT_WAIT) {
tv.tv_sec = tv.tv_usec = 0;
tvp = &tv;
} else {
/* Otherwise we can block */
tvp = NULL; /* wait forever */
}
}
numevents = aeApiPoll(eventLoop, tvp);
for (j = 0; j < numevents; j++) {
aeFileEvent *fe = &eventLoop->events[eventLoop->fired[j].fd];
int mask = eventLoop->fired[j].mask;
int fd = eventLoop->fired[j].fd;
int rfired = 0;
/* note the fe->mask & mask & ... code: maybe an already processed
* event removed an element that fired and we still didn't
* processed, so we check if the event is still valid. */
if (fe->mask & mask & AE_READABLE) {
rfired = 1;
fe->rfileProc(eventLoop,fd,fe->clientData,mask);
}
if (fe->mask & mask & AE_WRITABLE) {
if (!rfired || fe->wfileProc != fe->rfileProc)
fe->wfileProc(eventLoop,fd,fe->clientData,mask);
}
processed++;
}
}
/* Check time events */
if (flags & AE_TIME_EVENTS)
processed += processTimeEvents(eventLoop);
return processed; /* return the number of processed file/time events */
}
/* Wait for milliseconds until the given file descriptor becomes
* writable/readable/exception */
int aeWait(int fd, int mask, long long milliseconds) {
struct pollfd pfd;
int retmask = 0, retval;
memset(&pfd, 0, sizeof(pfd));
pfd.fd = fd;
if (mask & AE_READABLE) pfd.events |= POLLIN;
if (mask & AE_WRITABLE) pfd.events |= POLLOUT;
if ((retval = poll(&pfd, 1, milliseconds))== 1) {
if (pfd.revents & POLLIN) retmask |= AE_READABLE;
if (pfd.revents & POLLOUT) retmask |= AE_WRITABLE;
if (pfd.revents & POLLERR) retmask |= AE_WRITABLE;
if (pfd.revents & POLLHUP) retmask |= AE_WRITABLE;
return retmask;
} else {
return retval;
}
}
void aeMain(aeEventLoop *eventLoop) {
eventLoop->stop = 0;
while (!eventLoop->stop) {
if (eventLoop->beforesleep != NULL)
eventLoop->beforesleep(eventLoop);
aeProcessEvents(eventLoop, AE_ALL_EVENTS);
}
}
char *aeGetApiName(void) {
return aeApiName();
}
void aeSetBeforeSleepProc(aeEventLoop *eventLoop, aeBeforeSleepProc *beforesleep) {
eventLoop->beforesleep = beforesleep;
}
================================================
FILE: src/ae.h
================================================
/* A simple event-driven programming library. Originally I wrote this code
* for the Jim's event-loop (Jim is a Tcl interpreter) but later translated
* it in form of a library for easy reuse.
*
* Copyright (c) 2006-2012, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef __AE_H__
#define __AE_H__
#include <time.h>
#define AE_OK 0
#define AE_ERR -1
#define AE_NONE 0
#define AE_READABLE 1
#define AE_WRITABLE 2
#define AE_FILE_EVENTS 1
#define AE_TIME_EVENTS 2
#define AE_ALL_EVENTS (AE_FILE_EVENTS|AE_TIME_EVENTS)
#define AE_DONT_WAIT 4
#define AE_NOMORE -1
#define AE_DELETED_EVENT_ID -1
/* Macros */
#define AE_NOTUSED(V) ((void) V)
struct aeEventLoop;
/* Types and data structures */
typedef void aeFileProc(struct aeEventLoop *eventLoop, int fd, void *clientData, int mask);
typedef int aeTimeProc(struct aeEventLoop *eventLoop, long long id, void *clientData);
typedef void aeEventFinalizerProc(struct aeEventLoop *eventLoop, void *clientData);
typedef void aeBeforeSleepProc(struct aeEventLoop *eventLoop);
/* File event structure */
typedef struct aeFileEvent {
int mask; /* one of AE_(READABLE|WRITABLE) */
aeFileProc *rfileProc;
aeFileProc *wfileProc;
void *clientData;
} aeFileEvent;
/* Time event structure */
typedef struct aeTimeEvent {
long long id; /* time event identifier. */
long when_sec; /* seconds */
long when_ms; /* milliseconds */
aeTimeProc *timeProc;
aeEventFinalizerProc *finalizerProc;
void *clientData;
struct aeTimeEvent *next;
} aeTimeEvent;
/* A fired event */
typedef struct aeFiredEvent {
int fd;
int mask;
} aeFiredEvent;
/* State of an event based program */
typedef struct aeEventLoop {
int maxfd; /* highest file descriptor currently registered */
int setsize; /* max number of file descriptors tracked */
long long timeEventNextId;
time_t lastTime; /* Used to detect system clock skew */
aeFileEvent *events; /* Registered events */
aeFiredEvent *fired; /* Fired events */
aeTimeEvent *timeEventHead;
int stop;
void *apidata; /* This is used for polling API specific data */
aeBeforeSleepProc *beforesleep;
} aeEventLoop;
/* Prototypes */
aeEventLoop *aeCreateEventLoop(int setsize);
void aeDeleteEventLoop(aeEventLoop *eventLoop);
void aeStop(aeEventLoop *eventLoop);
int aeCreateFileEvent(aeEventLoop *eventLoop, int fd, int mask,
aeFileProc *proc, void *clientData);
void aeDeleteFileEvent(aeEventLoop *eventLoop, int fd, int mask);
int aeGetFileEvents(aeEventLoop *eventLoop, int fd);
long long aeCreateTimeEvent(aeEventLoop *eventLoop, long long milliseconds,
aeTimeProc *proc, void *clientData,
aeEventFinalizerProc *finalizerProc);
int aeDeleteTimeEvent(aeEventLoop *eventLoop, long long id);
int aeProcessEvents(aeEventLoop *eventLoop, int flags);
int aeWait(int fd, int mask, long long milliseconds);
void aeMain(aeEventLoop *eventLoop);
char *aeGetApiName(void);
void aeSetBeforeSleepProc(aeEventLoop *eventLoop, aeBeforeSleepProc *beforesleep);
int aeGetSetSize(aeEventLoop *eventLoop);
int aeResizeSetSize(aeEventLoop *eventLoop, int setsize);
#endif
================================================
FILE: src/ae_epoll.c
================================================
/* Linux epoll(2) based ae.c module
*
* Copyright (c) 2009-2012, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include <sys/epoll.h>
typedef struct aeApiState {
int epfd;
struct epoll_event *events;
} aeApiState;
static int aeApiCreate(aeEventLoop *eventLoop) {
aeApiState *state = zmalloc(sizeof(aeApiState));
if (!state) return -1;
state->events = zmalloc(sizeof(struct epoll_event)*eventLoop->setsize);
if (!state->events) {
zfree(state);
return -1;
}
state->epfd = epoll_create(1024); /* 1024 is just a hint for the kernel */
if (state->epfd == -1) {
zfree(state->events);
zfree(state);
return -1;
}
eventLoop->apidata = state;
return 0;
}
static int aeApiResize(aeEventLoop *eventLoop, int setsize) {
aeApiState *state = eventLoop->apidata;
state->events = zrealloc(state->events, sizeof(struct epoll_event)*setsize);
return 0;
}
static void aeApiFree(aeEventLoop *eventLoop) {
aeApiState *state = eventLoop->apidata;
close(state->epfd);
zfree(state->events);
zfree(state);
}
static int aeApiAddEvent(aeEventLoop *eventLoop, int fd, int mask) {
aeApiState *state = eventLoop->apidata;
struct epoll_event ee = {0}; /* avoid valgrind warning */
/* If the fd was already monitored for some event, we need a MOD
* operation. Otherwise we need an ADD operation. */
int op = eventLoop->events[fd].mask == AE_NONE ?
EPOLL_CTL_ADD : EPOLL_CTL_MOD;
ee.events = 0;
mask |= eventLoop->events[fd].mask; /* Merge old events */
if (mask & AE_READABLE) ee.events |= EPOLLIN;
if (mask & AE_WRITABLE) ee.events |= EPOLLOUT;
ee.data.fd = fd;
if (epoll_ctl(state->epfd,op,fd,&ee) == -1) return -1;
return 0;
}
static void aeApiDelEvent(aeEventLoop *eventLoop, int fd, int delmask) {
aeApiState *state = eventLoop->apidata;
struct epoll_event ee = {0}; /* avoid valgrind warning */
int mask = eventLoop->events[fd].mask & (~delmask);
ee.events = 0;
if (mask & AE_READABLE) ee.events |= EPOLLIN;
if (mask & AE_WRITABLE) ee.events |= EPOLLOUT;
ee.data.fd = fd;
if (mask != AE_NONE) {
epoll_ctl(state->epfd,EPOLL_CTL_MOD,fd,&ee);
} else {
/* Note, Kernel < 2.6.9 requires a non null event pointer even for
* EPOLL_CTL_DEL. */
epoll_ctl(state->epfd,EPOLL_CTL_DEL,fd,&ee);
}
}
static int aeApiPoll(aeEventLoop *eventLoop, struct timeval *tvp) {
aeApiState *state = eventLoop->apidata;
int retval, numevents = 0;
retval = epoll_wait(state->epfd,state->events,eventLoop->setsize,
tvp ? (tvp->tv_sec*1000 + tvp->tv_usec/1000) : -1);
if (retval > 0) {
int j;
numevents = retval;
for (j = 0; j < numevents; j++) {
int mask = 0;
struct epoll_event *e = state->events+j;
if (e->events & EPOLLIN) mask |= AE_READABLE;
if (e->events & EPOLLOUT) mask |= AE_WRITABLE;
if (e->events & EPOLLERR) mask |= AE_WRITABLE;
if (e->events & EPOLLHUP) mask |= AE_WRITABLE;
eventLoop->fired[j].fd = e->data.fd;
eventLoop->fired[j].mask = mask;
}
}
return numevents;
}
static char *aeApiName(void) {
return "epoll";
}
================================================
FILE: src/ae_evport.c
================================================
/* ae.c module for illumos event ports.
*
* Copyright (c) 2012, Joyent, Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include <assert.h>
#include <errno.h>
#include <port.h>
#include <poll.h>
#include <sys/types.h>
#include <sys/time.h>
#include <stdio.h>
static int evport_debug = 0;
/*
* This file implements the ae API using event ports, present on Solaris-based
* systems since Solaris 10. Using the event port interface, we associate file
* descriptors with the port. Each association also includes the set of poll(2)
* events that the consumer is interested in (e.g., POLLIN and POLLOUT).
*
* There's one tricky piece to this implementation: when we return events via
* aeApiPoll, the corresponding file descriptors become dissociated from the
* port. This is necessary because poll events are level-triggered, so if the
* fd didn't become dissociated, it would immediately fire another event since
* the underlying state hasn't changed yet. We must re-associate the file
* descriptor, but only after we know that our caller has actually read from it.
* The ae API does not tell us exactly when that happens, but we do know that
* it must happen by the time aeApiPoll is called again. Our solution is to
* keep track of the last fds returned by aeApiPoll and re-associate them next
* time aeApiPoll is invoked.
*
* To summarize, in this module, each fd association is EITHER (a) represented
* only via the in-kernel association OR (b) represented by pending_fds and
* pending_masks. (b) is only true for the last fds we returned from aeApiPoll,
* and only until we enter aeApiPoll again (at which point we restore the
* in-kernel association).
*/
#define MAX_EVENT_BATCHSZ 512
typedef struct aeApiState {
int portfd; /* event port */
int npending; /* # of pending fds */
int pending_fds[MAX_EVENT_BATCHSZ]; /* pending fds */
int pending_masks[MAX_EVENT_BATCHSZ]; /* pending fds' masks */
} aeApiState;
static int aeApiCreate(aeEventLoop *eventLoop) {
int i;
aeApiState *state = zmalloc(sizeof(aeApiState));
if (!state) return -1;
state->portfd = port_create();
if (state->portfd == -1) {
zfree(state);
return -1;
}
state->npending = 0;
for (i = 0; i < MAX_EVENT_BATCHSZ; i++) {
state->pending_fds[i] = -1;
state->pending_masks[i] = AE_NONE;
}
eventLoop->apidata = state;
return 0;
}
static int aeApiResize(aeEventLoop *eventLoop, int setsize) {
/* Nothing to resize here. */
return 0;
}
static void aeApiFree(aeEventLoop *eventLoop) {
aeApiState *state = eventLoop->apidata;
close(state->portfd);
zfree(state);
}
static int aeApiLookupPending(aeApiState *state, int fd) {
int i;
for (i = 0; i < state->npending; i++) {
if (state->pending_fds[i] == fd)
return (i);
}
return (-1);
}
/*
* Helper function to invoke port_associate for the given fd and mask.
*/
static int aeApiAssociate(const char *where, int portfd, int fd, int mask) {
int events = 0;
int rv, err;
if (mask & AE_READABLE)
events |= POLLIN;
if (mask & AE_WRITABLE)
events |= POLLOUT;
if (evport_debug)
fprintf(stderr, "%s: port_associate(%d, 0x%x) = ", where, fd, events);
rv = port_associate(portfd, PORT_SOURCE_FD, fd, events,
(void *)(uintptr_t)mask);
err = errno;
if (evport_debug)
fprintf(stderr, "%d (%s)\n", rv, rv == 0 ? "no error" : strerror(err));
if (rv == -1) {
fprintf(stderr, "%s: port_associate: %s\n", where, strerror(err));
if (err == EAGAIN)
fprintf(stderr, "aeApiAssociate: event port limit exceeded.");
}
return rv;
}
static int aeApiAddEvent(aeEventLoop *eventLoop, int fd, int mask) {
aeApiState *state = eventLoop->apidata;
int fullmask, pfd;
if (evport_debug)
fprintf(stderr, "aeApiAddEvent: fd %d mask 0x%x\n", fd, mask);
/*
* Since port_associate's "events" argument replaces any existing events, we
* must be sure to include whatever events are already associated when
* we call port_associate() again.
*/
fullmask = mask | eventLoop->events[fd].mask;
pfd = aeApiLookupPending(state, fd);
if (pfd != -1) {
/*
* This fd was recently returned from aeApiPoll. It should be safe to
* assume that the consumer has processed that poll event, but we play
* it safer by simply updating pending_mask. The fd will be
* re-associated as usual when aeApiPoll is called again.
*/
if (evport_debug)
fprintf(stderr, "aeApiAddEvent: adding to pending fd %d\n", fd);
state->pending_masks[pfd] |= fullmask;
return 0;
}
return (aeApiAssociate("aeApiAddEvent", state->portfd, fd, fullmask));
}
static void aeApiDelEvent(aeEventLoop *eventLoop, int fd, int mask) {
aeApiState *state = eventLoop->apidata;
int fullmask, pfd;
if (evport_debug)
fprintf(stderr, "del fd %d mask 0x%x\n", fd, mask);
pfd = aeApiLookupPending(state, fd);
if (pfd != -1) {
if (evport_debug)
fprintf(stderr, "deleting event from pending fd %d\n", fd);
/*
* This fd was just returned from aeApiPoll, so it's not currently
* associated with the port. All we need to do is update
* pending_mask appropriately.
*/
state->pending_masks[pfd] &= ~mask;
if (state->pending_masks[pfd] == AE_NONE)
state->pending_fds[pfd] = -1;
return;
}
/*
* The fd is currently associated with the port. Like with the add case
* above, we must look at the full mask for the file descriptor before
* updating that association. We don't have a good way of knowing what the
* events are without looking into the eventLoop state directly. We rely on
* the fact that our caller has already updated the mask in the eventLoop.
*/
fullmask = eventLoop->events[fd].mask;
if (fullmask == AE_NONE) {
/*
* We're removing *all* events, so use port_dissociate to remove the
* association completely. Failure here indicates a bug.
*/
if (evport_debug)
fprintf(stderr, "aeApiDelEvent: port_dissociate(%d)\n", fd);
if (port_dissociate(state->portfd, PORT_SOURCE_FD, fd) != 0) {
perror("aeApiDelEvent: port_dissociate");
abort(); /* will not return */
}
} else if (aeApiAssociate("aeApiDelEvent", state->portfd, fd,
fullmask) != 0) {
/*
* ENOMEM is a potentially transient condition, but the kernel won't
* generally return it unless things are really bad. EAGAIN indicates
* we've reached an resource limit, for which it doesn't make sense to
* retry (counter-intuitively). All other errors indicate a bug. In any
* of these cases, the best we can do is to abort.
*/
abort(); /* will not return */
}
}
static int aeApiPoll(aeEventLoop *eventLoop, struct timeval *tvp) {
aeApiState *state = eventLoop->apidata;
struct timespec timeout, *tsp;
int mask, i;
uint_t nevents;
port_event_t event[MAX_EVENT_BATCHSZ];
/*
* If we've returned fd events before, we must re-associate them with the
* port now, before calling port_get(). See the block comment at the top of
* this file for an explanation of why.
*/
for (i = 0; i < state->npending; i++) {
if (state->pending_fds[i] == -1)
/* This fd has since been deleted. */
continue;
if (aeApiAssociate("aeApiPoll", state->portfd,
state->pending_fds[i], state->pending_masks[i]) != 0) {
/* See aeApiDelEvent for why this case is fatal. */
abort();
}
state->pending_masks[i] = AE_NONE;
state->pending_fds[i] = -1;
}
state->npending = 0;
if (tvp != NULL) {
timeout.tv_sec = tvp->tv_sec;
timeout.tv_nsec = tvp->tv_usec * 1000;
tsp = &timeout;
} else {
tsp = NULL;
}
/*
* port_getn can return with errno == ETIME having returned some events (!).
* So if we get ETIME, we check nevents, too.
*/
nevents = 1;
if (port_getn(state->portfd, event, MAX_EVENT_BATCHSZ, &nevents,
tsp) == -1 && (errno != ETIME || nevents == 0)) {
if (errno == ETIME || errno == EINTR)
return 0;
/* Any other error indicates a bug. */
perror("aeApiPoll: port_get");
abort();
}
state->npending = nevents;
for (i = 0; i < nevents; i++) {
mask = 0;
if (event[i].portev_events & POLLIN)
mask |= AE_READABLE;
if (event[i].portev_events & POLLOUT)
mask |= AE_WRITABLE;
eventLoop->fired[i].fd = event[i].portev_object;
eventLoop->fired[i].mask = mask;
if (evport_debug)
fprintf(stderr, "aeApiPoll: fd %d mask 0x%x\n",
(int)event[i].portev_object, mask);
state->pending_fds[i] = event[i].portev_object;
state->pending_masks[i] = (uintptr_t)event[i].portev_user;
}
return nevents;
}
static char *aeApiName(void) {
return "evport";
}
================================================
FILE: src/ae_kqueue.c
================================================
/* Kqueue(2)-based ae.c module
*
* Copyright (C) 2009 Harish Mallipeddi - harish.mallipeddi@gmail.com
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include <sys/types.h>
#include <sys/event.h>
#include <sys/time.h>
typedef struct aeApiState {
int kqfd;
struct kevent *events;
} aeApiState;
static int aeApiCreate(aeEventLoop *eventLoop) {
aeApiState *state = zmalloc(sizeof(aeApiState));
if (!state) return -1;
state->events = zmalloc(sizeof(struct kevent)*eventLoop->setsize);
if (!state->events) {
zfree(state);
return -1;
}
state->kqfd = kqueue();
if (state->kqfd == -1) {
zfree(state->events);
zfree(state);
return -1;
}
eventLoop->apidata = state;
return 0;
}
static int aeApiResize(aeEventLoop *eventLoop, int setsize) {
aeApiState *state = eventLoop->apidata;
state->events = zrealloc(state->events, sizeof(struct kevent)*setsize);
return 0;
}
static void aeApiFree(aeEventLoop *eventLoop) {
aeApiState *state = eventLoop->apidata;
close(state->kqfd);
zfree(state->events);
zfree(state);
}
static int aeApiAddEvent(aeEventLoop *eventLoop, int fd, int mask) {
aeApiState *state = eventLoop->apidata;
struct kevent ke;
if (mask & AE_READABLE) {
EV_SET(&ke, fd, EVFILT_READ, EV_ADD, 0, 0, NULL);
if (kevent(state->kqfd, &ke, 1, NULL, 0, NULL) == -1) return -1;
}
if (mask & AE_WRITABLE) {
EV_SET(&ke, fd, EVFILT_WRITE, EV_ADD, 0, 0, NULL);
if (kevent(state->kqfd, &ke, 1, NULL, 0, NULL) == -1) return -1;
}
return 0;
}
static void aeApiDelEvent(aeEventLoop *eventLoop, int fd, int mask) {
aeApiState *state = eventLoop->apidata;
struct kevent ke;
if (mask & AE_READABLE) {
EV_SET(&ke, fd, EVFILT_READ, EV_DELETE, 0, 0, NULL);
kevent(state->kqfd, &ke, 1, NULL, 0, NULL);
}
if (mask & AE_WRITABLE) {
EV_SET(&ke, fd, EVFILT_WRITE, EV_DELETE, 0, 0, NULL);
kevent(state->kqfd, &ke, 1, NULL, 0, NULL);
}
}
static int aeApiPoll(aeEventLoop *eventLoop, struct timeval *tvp) {
aeApiState *state = eventLoop->apidata;
int retval, numevents = 0;
if (tvp != NULL) {
struct timespec timeout;
timeout.tv_sec = tvp->tv_sec;
timeout.tv_nsec = tvp->tv_usec * 1000;
retval = kevent(state->kqfd, NULL, 0, state->events, eventLoop->setsize,
&timeout);
} else {
retval = kevent(state->kqfd, NULL, 0, state->events, eventLoop->setsize,
NULL);
}
if (retval > 0) {
int j;
numevents = retval;
for(j = 0; j < numevents; j++) {
int mask = 0;
struct kevent *e = state->events+j;
if (e->filter == EVFILT_READ) mask |= AE_READABLE;
if (e->filter == EVFILT_WRITE) mask |= AE_WRITABLE;
eventLoop->fired[j].fd = e->ident;
eventLoop->fired[j].mask = mask;
}
}
return numevents;
}
static char *aeApiName(void) {
return "kqueue";
}
================================================
FILE: src/ae_select.c
================================================
/* Select()-based ae.c module.
*
* Copyright (c) 2009-2012, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include <sys/select.h>
#include <string.h>
typedef struct aeApiState {
fd_set rfds, wfds;
/* We need to have a copy of the fd sets as it's not safe to reuse
* FD sets after select(). */
fd_set _rfds, _wfds;
} aeApiState;
static int aeApiCreate(aeEventLoop *eventLoop) {
aeApiState *state = zmalloc(sizeof(aeApiState));
if (!state) return -1;
FD_ZERO(&state->rfds);
FD_ZERO(&state->wfds);
eventLoop->apidata = state;
return 0;
}
static int aeApiResize(aeEventLoop *eventLoop, int setsize) {
/* Just ensure we have enough room in the fd_set type. */
if (setsize >= FD_SETSIZE) return -1;
return 0;
}
static void aeApiFree(aeEventLoop *eventLoop) {
zfree(eventLoop->apidata);
}
static int aeApiAddEvent(aeEventLoop *eventLoop, int fd, int mask) {
aeApiState *state = eventLoop->apidata;
if (mask & AE_READABLE) FD_SET(fd,&state->rfds);
if (mask & AE_WRITABLE) FD_SET(fd,&state->wfds);
return 0;
}
static void aeApiDelEvent(aeEventLoop *eventLoop, int fd, int mask) {
aeApiState *state = eventLoop->apidata;
if (mask & AE_READABLE) FD_CLR(fd,&state->rfds);
if (mask & AE_WRITABLE) FD_CLR(fd,&state->wfds);
}
static int aeApiPoll(aeEventLoop *eventLoop, struct timeval *tvp) {
aeApiState *state = eventLoop->apidata;
int retval, j, numevents = 0;
memcpy(&state->_rfds,&state->rfds,sizeof(fd_set));
memcpy(&state->_wfds,&state->wfds,sizeof(fd_set));
retval = select(eventLoop->maxfd+1,
&state->_rfds,&state->_wfds,NULL,tvp);
if (retval > 0) {
for (j = 0; j <= eventLoop->maxfd; j++) {
int mask = 0;
aeFileEvent *fe = &eventLoop->events[j];
if (fe->mask == AE_NONE) continue;
if (fe->mask & AE_READABLE && FD_ISSET(j,&state->_rfds))
mask |= AE_READABLE;
if (fe->mask & AE_WRITABLE && FD_ISSET(j,&state->_wfds))
mask |= AE_WRITABLE;
eventLoop->fired[numevents].fd = j;
eventLoop->fired[numevents].mask = mask;
numevents++;
}
}
return numevents;
}
static char *aeApiName(void) {
return "select";
}
================================================
FILE: src/aprintf.c
================================================
// Copyright (C) 2012 - Will Glozer. All rights reserved.
#include <stdarg.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
char *aprintf(char **s, const char *fmt, ...) {
char *c = NULL;
int n, len;
va_list ap;
va_start(ap, fmt);
n = vsnprintf(NULL, 0, fmt, ap) + 1;
va_end(ap);
len = *s ? strlen(*s) : 0;
if ((*s = realloc(*s, (len + n) * sizeof(char)))) {
c = *s + len;
va_start(ap, fmt);
vsnprintf(c, n, fmt, ap);
va_end(ap);
}
return c;
}
================================================
FILE: src/aprintf.h
================================================
#ifndef APRINTF_H
#define APRINTF_H
char *aprintf(char **, const char *, ...);
#endif /* APRINTF_H */
================================================
FILE: src/atomicvar.h
================================================
/* This file implements atomic counters using __atomic or __sync macros if
* available, otherwise synchronizing different threads using a mutex.
*
* The exported interaface is composed of three macros:
*
* atomicIncr(var,count) -- Increment the atomic counter
* atomicGetIncr(var,oldvalue_var,count) -- Get and increment the atomic counter
* atomicDecr(var,count) -- Decrement the atomic counter
* atomicGet(var,dstvar) -- Fetch the atomic counter value
* atomicSet(var,value) -- Set the atomic counter value
*
* The variable 'var' should also have a declared mutex with the same
* name and the "_mutex" postfix, for instance:
*
* long myvar;
* pthread_mutex_t myvar_mutex;
* atomicSet(myvar,12345);
*
* If atomic primitives are availble (tested in config.h) the mutex
* is not used.
*
* Never use return value from the macros, instead use the AtomicGetIncr()
* if you need to get the current value and increment it atomically, like
* in the followign example:
*
* long oldvalue;
* atomicGetIncr(myvar,oldvalue,1);
* doSomethingWith(oldvalue);
*
* ----------------------------------------------------------------------------
*
* Copyright (c) 2015, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include <pthread.h>
#ifndef __ATOMIC_VAR_H
#define __ATOMIC_VAR_H
/* To test Redis with Helgrind (a Valgrind tool) it is useful to define
* the following macro, so that __sync macros are used: those can be detected
* by Helgrind (even if they are less efficient) so that no false positive
* is reported. */
// #define __ATOMIC_VAR_FORCE_SYNC_MACROS
#if !defined(__ATOMIC_VAR_FORCE_SYNC_MACROS) && defined(__ATOMIC_RELAXED) && !defined(__sun) && (!defined(__clang__) || !defined(__APPLE__) || __apple_build_version__ > 4210057)
/* Implementation using __atomic macros. */
#define atomicIncr(var,count) __atomic_add_fetch(&var,(count),__ATOMIC_RELAXED)
#define atomicGetIncr(var,oldvalue_var,count) do { \
oldvalue_var = __atomic_fetch_add(&var,(count),__ATOMIC_RELAXED); \
} while(0)
#define atomicDecr(var,count) __atomic_sub_fetch(&var,(count),__ATOMIC_RELAXED)
#define atomicGet(var,dstvar) do { \
dstvar = __atomic_load_n(&var,__ATOMIC_RELAXED); \
} while(0)
#define atomicSet(var,value) __atomic_store_n(&var,value,__ATOMIC_RELAXED)
#define REDIS_ATOMIC_API "atomic-builtin"
#elif defined(HAVE_ATOMIC)
/* Implementation using __sync macros. */
#define atomicIncr(var,count) __sync_add_and_fetch(&var,(count))
#define atomicGetIncr(var,oldvalue_var,count) do { \
oldvalue_var = __sync_fetch_and_add(&var,(count)); \
} while(0)
#define atomicDecr(var,count) __sync_sub_and_fetch(&var,(count))
#define atomicGet(var,dstvar) do { \
dstvar = __sync_sub_and_fetch(&var,0); \
} while(0)
#define atomicSet(var,value) do { \
while(!__sync_bool_compare_and_swap(&var,var,value)); \
} while(0)
#define REDIS_ATOMIC_API "sync-builtin"
#else
/* Implementation using pthread mutex. */
#define atomicIncr(var,count) do { \
pthread_mutex_lock(&var ## _mutex); \
var += (count); \
pthread_mutex_unlock(&var ## _mutex); \
} while(0)
#define atomicGetIncr(var,oldvalue_var,count) do { \
pthread_mutex_lock(&var ## _mutex); \
oldvalue_var = var; \
var += (count); \
pthread_mutex_unlock(&var ## _mutex); \
} while(0)
#define atomicDecr(var,count) do { \
pthread_mutex_lock(&var ## _mutex); \
var -= (count); \
pthread_mutex_unlock(&var ## _mutex); \
} while(0)
#define atomicGet(var,dstvar) do { \
pthread_mutex_lock(&var ## _mutex); \
dstvar = var; \
pthread_mutex_unlock(&var ## _mutex); \
} while(0)
#define atomicSet(var,value) do { \
pthread_mutex_lock(&var ## _mutex); \
var = value; \
pthread_mutex_unlock(&var ## _mutex); \
} while(0)
#define REDIS_ATOMIC_API "pthread-mutex"
#endif
#endif /* __ATOMIC_VAR_H */
================================================
FILE: src/config.h
================================================
#ifndef CONFIG_H
#define CONFIG_H
#if defined(__FreeBSD__) || defined(__APPLE__)
#define HAVE_KQUEUE
#elif defined(__linux__)
#define HAVE_EPOLL
#elif defined (__sun)
#define HAVE_EVPORT
#define _XPG6
#define __EXTENSIONS__
#include <stropts.h>
#include <sys/filio.h>
#include <sys/time.h>
#endif
#endif /* CONFIG_H */
================================================
FILE: src/http_parser.c
================================================
/* Copyright Joyent, Inc. and other Node contributors.
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to
* deal in the Software without restriction, including without limitation the
* rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
* sell copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
* IN THE SOFTWARE.
*/
#include "http_parser.h"
#include <assert.h>
#include <stddef.h>
#include <ctype.h>
#include <stdlib.h>
#include <string.h>
#include <limits.h>
#ifndef ULLONG_MAX
# define ULLONG_MAX ((uint64_t) -1) /* 2^64-1 */
#endif
#ifndef MIN
# define MIN(a,b) ((a) < (b) ? (a) : (b))
#endif
#ifndef ARRAY_SIZE
# define ARRAY_SIZE(a) (sizeof(a) / sizeof((a)[0]))
#endif
#ifndef BIT_AT
# define BIT_AT(a, i) \
(!!((unsigned int) (a)[(unsigned int) (i) >> 3] & \
(1 << ((unsigned int) (i) & 7))))
#endif
#ifndef ELEM_AT
# define ELEM_AT(a, i, v) ((unsigned int) (i) < ARRAY_SIZE(a) ? (a)[(i)] : (v))
#endif
#define SET_ERRNO(e) \
do { \
parser->http_errno = (e); \
} while(0)
#define CURRENT_STATE() p_state
#define UPDATE_STATE(V) p_state = (enum state) (V);
#define RETURN(V) \
do { \
parser->state = CURRENT_STATE(); \
return (V); \
} while (0);
#define REEXECUTE() \
goto reexecute; \
#ifdef __GNUC__
# define LIKELY(X) __builtin_expect(!!(X), 1)
# define UNLIKELY(X) __builtin_expect(!!(X), 0)
#else
# define LIKELY(X) (X)
# define UNLIKELY(X) (X)
#endif
/* Run the notify callback FOR, returning ER if it fails */
#define CALLBACK_NOTIFY_(FOR, ER) \
do { \
assert(HTTP_PARSER_ERRNO(parser) == HPE_OK); \
\
if (LIKELY(settings->on_##FOR)) { \
parser->state = CURRENT_STATE(); \
if (UNLIKELY(0 != settings->on_##FOR(parser))) { \
SET_ERRNO(HPE_CB_##FOR); \
} \
UPDATE_STATE(parser->state); \
\
/* We either errored above or got paused; get out */ \
if (UNLIKELY(HTTP_PARSER_ERRNO(parser) != HPE_OK)) { \
return (ER); \
} \
} \
} while (0)
/* Run the notify callback FOR and consume the current byte */
#define CALLBACK_NOTIFY(FOR) CALLBACK_NOTIFY_(FOR, p - data + 1)
/* Run the notify callback FOR and don't consume the current byte */
#define CALLBACK_NOTIFY_NOADVANCE(FOR) CALLBACK_NOTIFY_(FOR, p - data)
/* Run data callback FOR with LEN bytes, returning ER if it fails */
#define CALLBACK_DATA_(FOR, LEN, ER) \
do { \
assert(HTTP_PARSER_ERRNO(parser) == HPE_OK); \
\
if (FOR##_mark) { \
if (LIKELY(settings->on_##FOR)) { \
parser->state = CURRENT_STATE(); \
if (UNLIKELY(0 != \
settings->on_##FOR(parser, FOR##_mark, (LEN)))) { \
SET_ERRNO(HPE_CB_##FOR); \
} \
UPDATE_STATE(parser->state); \
\
/* We either errored above or got paused; get out */ \
if (UNLIKELY(HTTP_PARSER_ERRNO(parser) != HPE_OK)) { \
return (ER); \
} \
} \
FOR##_mark = NULL; \
} \
} while (0)
/* Run the data callback FOR and consume the current byte */
#define CALLBACK_DATA(FOR) \
CALLBACK_DATA_(FOR, p - FOR##_mark, p - data + 1)
/* Run the data callback FOR and don't consume the current byte */
#define CALLBACK_DATA_NOADVANCE(FOR) \
CALLBACK_DATA_(FOR, p - FOR##_mark, p - data)
/* Set the mark FOR; non-destructive if mark is already set */
#define MARK(FOR) \
do { \
if (!FOR##_mark) { \
FOR##_mark = p; \
} \
} while (0)
/* Don't allow the total size of the HTTP headers (including the status
* line) to exceed HTTP_MAX_HEADER_SIZE. This check is here to protect
* embedders against denial-of-service attacks where the attacker feeds
* us a never-ending header that the embedder keeps buffering.
*
* This check is arguably the responsibility of embedders but we're doing
* it on the embedder's behalf because most won't bother and this way we
* make the web a little safer. HTTP_MAX_HEADER_SIZE is still far bigger
* than any reasonable request or response so this should never affect
* day-to-day operation.
*/
#define COUNT_HEADER_SIZE(V) \
do { \
parser->nread += (V); \
if (UNLIKELY(parser->nread > (HTTP_MAX_HEADER_SIZE))) { \
SET_ERRNO(HPE_HEADER_OVERFLOW); \
goto error; \
} \
} while (0)
#define PROXY_CONNECTION "proxy-connection"
#define CONNECTION "connection"
#define CONTENT_LENGTH "content-length"
#define TRANSFER_ENCODING "transfer-encoding"
#define UPGRADE "upgrade"
#define CHUNKED "chunked"
#define KEEP_ALIVE "keep-alive"
#define CLOSE "close"
static const char *method_strings[] =
{
#define XX(num, name, string) #string,
HTTP_METHOD_MAP(XX)
#undef XX
};
/* Tokens as defined by rfc 2616. Also lowercases them.
* token = 1*<any CHAR except CTLs or separators>
* separators = "(" | ")" | "<" | ">" | "@"
* | "," | ";" | ":" | "\" | <">
* | "/" | "[" | "]" | "?" | "="
* | "{" | "}" | SP | HT
*/
static const char tokens[256] = {
/* 0 nul 1 soh 2 stx 3 etx 4 eot 5 enq 6 ack 7 bel */
0, 0, 0, 0, 0, 0, 0, 0,
/* 8 bs 9 ht 10 nl 11 vt 12 np 13 cr 14 so 15 si */
0, 0, 0, 0, 0, 0, 0, 0,
/* 16 dle 17 dc1 18 dc2 19 dc3 20 dc4 21 nak 22 syn 23 etb */
0, 0, 0, 0, 0, 0, 0, 0,
/* 24 can 25 em 26 sub 27 esc 28 fs 29 gs 30 rs 31 us */
0, 0, 0, 0, 0, 0, 0, 0,
/* 32 sp 33 ! 34 " 35 # 36 $ 37 % 38 & 39 ' */
0, '!', 0, '#', '$', '%', '&', '\'',
/* 40 ( 41 ) 42 * 43 + 44 , 45 - 46 . 47 / */
0, 0, '*', '+', 0, '-', '.', 0,
/* 48 0 49 1 50 2 51 3 52 4 53 5 54 6 55 7 */
'0', '1', '2', '3', '4', '5', '6', '7',
/* 56 8 57 9 58 : 59 ; 60 < 61 = 62 > 63 ? */
'8', '9', 0, 0, 0, 0, 0, 0,
/* 64 @ 65 A 66 B 67 C 68 D 69 E 70 F 71 G */
0, 'a', 'b', 'c', 'd', 'e', 'f', 'g',
/* 72 H 73 I 74 J 75 K 76 L 77 M 78 N 79 O */
'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o',
/* 80 P 81 Q 82 R 83 S 84 T 85 U 86 V 87 W */
'p', 'q', 'r', 's', 't', 'u', 'v', 'w',
/* 88 X 89 Y 90 Z 91 [ 92 \ 93 ] 94 ^ 95 _ */
'x', 'y', 'z', 0, 0, 0, '^', '_',
/* 96 ` 97 a 98 b 99 c 100 d 101 e 102 f 103 g */
'`', 'a', 'b', 'c', 'd', 'e', 'f', 'g',
/* 104 h 105 i 106 j 107 k 108 l 109 m 110 n 111 o */
'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o',
/* 112 p 113 q 114 r 115 s 116 t 117 u 118 v 119 w */
'p', 'q', 'r', 's', 't', 'u', 'v', 'w',
/* 120 x 121 y 122 z 123 { 124 | 125 } 126 ~ 127 del */
'x', 'y', 'z', 0, '|', 0, '~', 0 };
static const int8_t unhex[256] =
{-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,-1,-1,-1,-1,-1,-1
,-1,10,11,12,13,14,15,-1,-1,-1,-1,-1,-1,-1,-1,-1
,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
,-1,10,11,12,13,14,15,-1,-1,-1,-1,-1,-1,-1,-1,-1
,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
};
#if HTTP_PARSER_STRICT
# define T(v) 0
#else
# define T(v) v
#endif
static const uint8_t normal_url_char[32] = {
/* 0 nul 1 soh 2 stx 3 etx 4 eot 5 enq 6 ack 7 bel */
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0,
/* 8 bs 9 ht 10 nl 11 vt 12 np 13 cr 14 so 15 si */
0 | T(2) | 0 | 0 | T(16) | 0 | 0 | 0,
/* 16 dle 17 dc1 18 dc2 19 dc3 20 dc4 21 nak 22 syn 23 etb */
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0,
/* 24 can 25 em 26 sub 27 esc 28 fs 29 gs 30 rs 31 us */
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0,
/* 32 sp 33 ! 34 " 35 # 36 $ 37 % 38 & 39 ' */
0 | 2 | 4 | 0 | 16 | 32 | 64 | 128,
/* 40 ( 41 ) 42 * 43 + 44 , 45 - 46 . 47 / */
1 | 2 | 4 | 8 | 16 | 32 | 64 | 128,
/* 48 0 49 1 50 2 51 3 52 4 53 5 54 6 55 7 */
1 | 2 | 4 | 8 | 16 | 32 | 64 | 128,
/* 56 8 57 9 58 : 59 ; 60 < 61 = 62 > 63 ? */
1 | 2 | 4 | 8 | 16 | 32 | 64 | 0,
/* 64 @ 65 A 66 B 67 C 68 D 69 E 70 F 71 G */
1 | 2 | 4 | 8 | 16 | 32 | 64 | 128,
/* 72 H 73 I 74 J 75 K 76 L 77 M 78 N 79 O */
1 | 2 | 4 | 8 | 16 | 32 | 64 | 128,
/* 80 P 81 Q 82 R 83 S 84 T 85 U 86 V 87 W */
1 | 2 | 4 | 8 | 16 | 32 | 64 | 128,
/* 88 X 89 Y 90 Z 91 [ 92 \ 93 ] 94 ^ 95 _ */
1 | 2 | 4 | 8 | 16 | 32 | 64 | 128,
/* 96 ` 97 a 98 b 99 c 100 d 101 e 102 f 103 g */
1 | 2 | 4 | 8 | 16 | 32 | 64 | 128,
/* 104 h 105 i 106 j 107 k 108 l 109 m 110 n 111 o */
1 | 2 | 4 | 8 | 16 | 32 | 64 | 128,
/* 112 p 113 q 114 r 115 s 116 t 117 u 118 v 119 w */
1 | 2 | 4 | 8 | 16 | 32 | 64 | 128,
/* 120 x 121 y 122 z 123 { 124 | 125 } 126 ~ 127 del */
1 | 2 | 4 | 8 | 16 | 32 | 64 | 0, };
#undef T
enum state
{ s_dead = 1 /* important that this is > 0 */
, s_start_req_or_res
, s_res_or_resp_H
, s_start_res
, s_res_H
, s_res_HT
, s_res_HTT
, s_res_HTTP
, s_res_http_major
, s_res_http_dot
, s_res_http_minor
, s_res_http_end
, s_res_first_status_code
, s_res_status_code
, s_res_status_start
, s_res_status
, s_res_line_almost_done
, s_start_req
, s_req_method
, s_req_spaces_before_url
, s_req_schema
, s_req_schema_slash
, s_req_schema_slash_slash
, s_req_server_start
, s_req_server
, s_req_server_with_at
, s_req_path
, s_req_query_string_start
, s_req_query_string
, s_req_fragment_start
, s_req_fragment
, s_req_http_start
, s_req_http_H
, s_req_http_HT
, s_req_http_HTT
, s_req_http_HTTP
, s_req_http_major
, s_req_http_dot
, s_req_http_minor
, s_req_http_end
, s_req_line_almost_done
, s_header_field_start
, s_header_field
, s_header_value_discard_ws
, s_header_value_discard_ws_almost_done
, s_header_value_discard_lws
, s_header_value_start
, s_header_value
, s_header_value_lws
, s_header_almost_done
, s_chunk_size_start
, s_chunk_size
, s_chunk_parameters
, s_chunk_size_almost_done
, s_headers_almost_done
, s_headers_done
/* Important: 's_headers_done' must be the last 'header' state. All
* states beyond this must be 'body' states. It is used for overflow
* checking. See the PARSING_HEADER() macro.
*/
, s_chunk_data
, s_chunk_data_almost_done
, s_chunk_data_done
, s_body_identity
, s_body_identity_eof
, s_message_done
};
#define PARSING_HEADER(state) (state <= s_headers_done)
enum header_states
{ h_general = 0
, h_C
, h_CO
, h_CON
, h_matching_connection
, h_matching_proxy_connection
, h_matching_content_length
, h_matching_transfer_encoding
, h_matching_upgrade
, h_connection
, h_content_length
, h_transfer_encoding
, h_upgrade
, h_matching_transfer_encoding_chunked
, h_matching_connection_token_start
, h_matching_connection_keep_alive
, h_matching_connection_close
, h_matching_connection_upgrade
, h_matching_connection_token
, h_transfer_encoding_chunked
, h_connection_keep_alive
, h_connection_close
, h_connection_upgrade
};
enum http_host_state
{
s_http_host_dead = 1
, s_http_userinfo_start
, s_http_userinfo
, s_http_host_start
, s_http_host_v6_start
, s_http_host
, s_http_host_v6
, s_http_host_v6_end
, s_http_host_v6_zone_start
, s_http_host_v6_zone
, s_http_host_port_start
, s_http_host_port
};
/* Macros for character classes; depends on strict-mode */
#define CR '\r'
#define LF '\n'
#define LOWER(c) (unsigned char)(c | 0x20)
#define IS_ALPHA(c) (LOWER(c) >= 'a' && LOWER(c) <= 'z')
#define IS_NUM(c) ((c) >= '0' && (c) <= '9')
#define IS_ALPHANUM(c) (IS_ALPHA(c) || IS_NUM(c))
#define IS_HEX(c) (IS_NUM(c) || (LOWER(c) >= 'a' && LOWER(c) <= 'f'))
#define IS_MARK(c) ((c) == '-' || (c) == '_' || (c) == '.' || \
(c) == '!' || (c) == '~' || (c) == '*' || (c) == '\'' || (c) == '(' || \
(c) == ')')
#define IS_USERINFO_CHAR(c) (IS_ALPHANUM(c) || IS_MARK(c) || (c) == '%' || \
(c) == ';' || (c) == ':' || (c) == '&' || (c) == '=' || (c) == '+' || \
(c) == '$' || (c) == ',')
#define STRICT_TOKEN(c) (tokens[(unsigned char)c])
#if HTTP_PARSER_STRICT
#define TOKEN(c) (tokens[(unsigned char)c])
#define IS_URL_CHAR(c) (BIT_AT(normal_url_char, (unsigned char)c))
#define IS_HOST_CHAR(c) (IS_ALPHANUM(c) || (c) == '.' || (c) == '-')
#else
#define TOKEN(c) ((c == ' ') ? ' ' : tokens[(unsigned char)c])
#define IS_URL_CHAR(c) \
(BIT_AT(normal_url_char, (unsigned char)c) || ((c) & 0x80))
#define IS_HOST_CHAR(c) \
(IS_ALPHANUM(c) || (c) == '.' || (c) == '-' || (c) == '_')
#endif
/**
* Verify that a char is a valid visible (printable) US-ASCII
* character or %x80-FF
**/
#define IS_HEADER_CHAR(ch) \
(ch == CR || ch == LF || ch == 9 || ((unsigned char)ch > 31 && ch != 127))
#define start_state (parser->type == HTTP_REQUEST ? s_start_req : s_start_res)
#if HTTP_PARSER_STRICT
# define STRICT_CHECK(cond) \
do { \
if (cond) { \
SET_ERRNO(HPE_STRICT); \
goto error; \
} \
} while (0)
# define NEW_MESSAGE() (http_should_keep_alive(parser) ? start_state : s_dead)
#else
# define STRICT_CHECK(cond)
# define NEW_MESSAGE() start_state
#endif
/* Map errno values to strings for human-readable output */
#define HTTP_STRERROR_GEN(n, s) { "HPE_" #n, s },
static struct {
const char *name;
const char *description;
} http_strerror_tab[] = {
HTTP_ERRNO_MAP(HTTP_STRERROR_GEN)
};
#undef HTTP_STRERROR_GEN
int http_message_needs_eof(const http_parser *parser);
/* Our URL parser.
*
* This is designed to be shared by http_parser_execute() for URL validation,
* hence it has a state transition + byte-for-byte interface. In addition, it
* is meant to be embedded in http_parser_parse_url(), which does the dirty
* work of turning state transitions URL components for its API.
*
* This function should only be invoked with non-space characters. It is
* assumed that the caller cares about (and can detect) the transition between
* URL and non-URL states by looking for these.
*/
static enum state
parse_url_char(enum state s, const char ch)
{
if (ch == ' ' || ch == '\r' || ch == '\n') {
return s_dead;
}
#if HTTP_PARSER_STRICT
if (ch == '\t' || ch == '\f') {
return s_dead;
}
#endif
switch (s) {
case s_req_spaces_before_url:
/* Proxied requests are followed by scheme of an absolute URI (alpha).
* All methods except CONNECT are followed by '/' or '*'.
*/
if (ch == '/' || ch == '*') {
return s_req_path;
}
if (IS_ALPHA(ch)) {
return s_req_schema;
}
break;
case s_req_schema:
if (IS_ALPHA(ch)) {
return s;
}
if (ch == ':') {
return s_req_schema_slash;
}
break;
case s_req_schema_slash:
if (ch == '/') {
return s_req_schema_slash_slash;
}
break;
case s_req_schema_slash_slash:
if (ch == '/') {
return s_req_server_start;
}
break;
case s_req_server_with_at:
if (ch == '@') {
return s_dead;
}
/* FALLTHROUGH */
case s_req_server_start:
case s_req_server:
if (ch == '/') {
return s_req_path;
}
if (ch == '?') {
return s_req_query_string_start;
}
if (ch == '@') {
return s_req_server_with_at;
}
if (IS_USERINFO_CHAR(ch) || ch == '[' || ch == ']') {
return s_req_server;
}
break;
case s_req_path:
if (IS_URL_CHAR(ch)) {
return s;
}
switch (ch) {
case '?':
return s_req_query_string_start;
case '#':
return s_req_fragment_start;
}
break;
case s_req_query_string_start:
case s_req_query_string:
if (IS_URL_CHAR(ch)) {
return s_req_query_string;
}
switch (ch) {
case '?':
/* allow extra '?' in query string */
return s_req_query_string;
case '#':
return s_req_fragment_start;
}
break;
case s_req_fragment_start:
if (IS_URL_CHAR(ch)) {
return s_req_fragment;
}
switch (ch) {
case '?':
return s_req_fragment;
case '#':
return s;
}
break;
case s_req_fragment:
if (IS_URL_CHAR(ch)) {
return s;
}
switch (ch) {
case '?':
case '#':
return s;
}
break;
default:
break;
}
/* We should never fall out of the switch above unless there's an error */
return s_dead;
}
size_t http_parser_execute (http_parser *parser,
const http_parser_settings *settings,
const char *data,
size_t len)
{
char c, ch;
int8_t unhex_val;
const char *p = data;
const char *header_field_mark = 0;
const char *header_value_mark = 0;
const char *url_mark = 0;
const char *body_mark = 0;
const char *status_mark = 0;
enum state p_state = (enum state) parser->state;
const unsigned int lenient = parser->lenient_http_headers;
/* We're in an error state. Don't bother doing anything. */
if (HTTP_PARSER_ERRNO(parser) != HPE_OK) {
return 0;
}
if (len == 0) {
switch (CURRENT_STATE()) {
case s_body_identity_eof:
/* Use of CALLBACK_NOTIFY() here would erroneously return 1 byte read if
* we got paused.
*/
CALLBACK_NOTIFY_NOADVANCE(message_complete);
return 0;
case s_dead:
case s_start_req_or_res:
case s_start_res:
case s_start_req:
return 0;
default:
SET_ERRNO(HPE_INVALID_EOF_STATE);
return 1;
}
}
if (CURRENT_STATE() == s_header_field)
header_field_mark = data;
if (CURRENT_STATE() == s_header_value)
header_value_mark = data;
switch (CURRENT_STATE()) {
case s_req_path:
case s_req_schema:
case s_req_schema_slash:
case s_req_schema_slash_slash:
case s_req_server_start:
case s_req_server:
case s_req_server_with_at:
case s_req_query_string_start:
case s_req_query_string:
case s_req_fragment_start:
case s_req_fragment:
url_mark = data;
break;
case s_res_status:
status_mark = data;
break;
default:
break;
}
for (p=data; p != data + len; p++) {
ch = *p;
if (PARSING_HEADER(CURRENT_STATE()))
COUNT_HEADER_SIZE(1);
reexecute:
switch (CURRENT_STATE()) {
case s_dead:
/* this state is used after a 'Connection: close' message
* the parser will error out if it reads another message
*/
if (LIKELY(ch == CR || ch == LF))
break;
SET_ERRNO(HPE_CLOSED_CONNECTION);
goto error;
case s_start_req_or_res:
{
if (ch == CR || ch == LF)
break;
parser->flags = 0;
parser->content_length = ULLONG_MAX;
if (ch == 'H') {
UPDATE_STATE(s_res_or_resp_H);
CALLBACK_NOTIFY(message_begin);
} else {
parser->type = HTTP_REQUEST;
UPDATE_STATE(s_start_req);
REEXECUTE();
}
break;
}
case s_res_or_resp_H:
if (ch == 'T') {
parser->type = HTTP_RESPONSE;
UPDATE_STATE(s_res_HT);
} else {
if (UNLIKELY(ch != 'E')) {
SET_ERRNO(HPE_INVALID_CONSTANT);
goto error;
}
parser->type = HTTP_REQUEST;
parser->method = HTTP_HEAD;
parser->index = 2;
UPDATE_STATE(s_req_method);
}
break;
case s_start_res:
{
parser->flags = 0;
parser->content_length = ULLONG_MAX;
switch (ch) {
case 'H':
UPDATE_STATE(s_res_H);
break;
case CR:
case LF:
break;
default:
SET_ERRNO(HPE_INVALID_CONSTANT);
goto error;
}
CALLBACK_NOTIFY(message_begin);
break;
}
case s_res_H:
STRICT_CHECK(ch != 'T');
UPDATE_STATE(s_res_HT);
break;
case s_res_HT:
STRICT_CHECK(ch != 'T');
UPDATE_STATE(s_res_HTT);
break;
case s_res_HTT:
STRICT_CHECK(ch != 'P');
UPDATE_STATE(s_res_HTTP);
break;
case s_res_HTTP:
STRICT_CHECK(ch != '/');
UPDATE_STATE(s_res_http_major);
break;
case s_res_http_major:
if (UNLIKELY(!IS_NUM(ch))) {
SET_ERRNO(HPE_INVALID_VERSION);
goto error;
}
parser->http_major = ch - '0';
UPDATE_STATE(s_res_http_dot);
break;
case s_res_http_dot:
{
if (UNLIKELY(ch != '.')) {
SET_ERRNO(HPE_INVALID_VERSION);
goto error;
}
UPDATE_STATE(s_res_http_minor);
break;
}
case s_res_http_minor:
if (UNLIKELY(!IS_NUM(ch))) {
SET_ERRNO(HPE_INVALID_VERSION);
goto error;
}
parser->http_minor = ch - '0';
UPDATE_STATE(s_res_http_end);
break;
case s_res_http_end:
{
if (UNLIKELY(ch != ' ')) {
SET_ERRNO(HPE_INVALID_VERSION);
goto error;
}
UPDATE_STATE(s_res_first_status_code);
break;
}
case s_res_first_status_code:
{
if (!IS_NUM(ch)) {
if (ch == ' ') {
break;
}
SET_ERRNO(HPE_INVALID_STATUS);
goto error;
}
parser->status_code = ch - '0';
UPDATE_STATE(s_res_status_code);
break;
}
case s_res_status_code:
{
if (!IS_NUM(ch)) {
switch (ch) {
case ' ':
UPDATE_STATE(s_res_status_start);
break;
case CR:
case LF:
UPDATE_STATE(s_res_status_start);
REEXECUTE();
break;
default:
SET_ERRNO(HPE_INVALID_STATUS);
goto error;
}
break;
}
parser->status_code *= 10;
parser->status_code += ch - '0';
if (UNLIKELY(parser->status_code > 999)) {
SET_ERRNO(HPE_INVALID_STATUS);
goto error;
}
break;
}
case s_res_status_start:
{
MARK(status);
UPDATE_STATE(s_res_status);
parser->index = 0;
if (ch == CR || ch == LF)
REEXECUTE();
break;
}
case s_res_status:
if (ch == CR) {
UPDATE_STATE(s_res_line_almost_done);
CALLBACK_DATA(status);
break;
}
if (ch == LF) {
UPDATE_STATE(s_header_field_start);
CALLBACK_DATA(status);
break;
}
break;
case s_res_line_almost_done:
STRICT_CHECK(ch != LF);
UPDATE_STATE(s_header_field_start);
break;
case s_start_req:
{
if (ch == CR || ch == LF)
break;
parser->flags = 0;
parser->content_length = ULLONG_MAX;
if (UNLIKELY(!IS_ALPHA(ch))) {
SET_ERRNO(HPE_INVALID_METHOD);
goto error;
}
parser->method = (enum http_method) 0;
parser->index = 1;
switch (ch) {
case 'A': parser->method = HTTP_ACL; break;
case 'B': parser->method = HTTP_BIND; break;
case 'C': parser->method = HTTP_CONNECT; /* or COPY, CHECKOUT */ break;
case 'D': parser->method = HTTP_DELETE; break;
case 'G': parser->method = HTTP_GET; break;
case 'H': parser->method = HTTP_HEAD; break;
case 'L': parser->method = HTTP_LOCK; /* or LINK */ break;
case 'M': parser->method = HTTP_MKCOL; /* or MOVE, MKACTIVITY, MERGE, M-SEARCH, MKCALENDAR */ break;
case 'N': parser->method = HTTP_NOTIFY; break;
case 'O': parser->method = HTTP_OPTIONS; break;
case 'P': parser->method = HTTP_POST;
/* or PROPFIND|PROPPATCH|PUT|PATCH|PURGE */
break;
case 'R': parser->method = HTTP_REPORT; /* or REBIND */ break;
case 'S': parser->method = HTTP_SUBSCRIBE; /* or SEARCH */ break;
case 'T': parser->method = HTTP_TRACE; break;
case 'U': parser->method = HTTP_UNLOCK; /* or UNSUBSCRIBE, UNBIND, UNLINK */ break;
default:
SET_ERRNO(HPE_INVALID_METHOD);
goto error;
}
UPDATE_STATE(s_req_method);
CALLBACK_NOTIFY(message_begin);
break;
}
case s_req_method:
{
const char *matcher;
if (UNLIKELY(ch == '\0')) {
SET_ERRNO(HPE_INVALID_METHOD);
goto error;
}
matcher = method_strings[parser->method];
if (ch == ' ' && matcher[parser->index] == '\0') {
UPDATE_STATE(s_req_spaces_before_url);
} else if (ch == matcher[parser->index]) {
; /* nada */
} else if ((ch >= 'A' && ch <= 'Z') || ch == '-') {
switch (parser->method << 16 | parser->index << 8 | ch) {
#define XX(meth, pos, ch, new_meth) \
case (HTTP_##meth << 16 | pos << 8 | ch): \
parser->method = HTTP_##new_meth; break;
XX(POST, 1, 'U', PUT)
XX(POST, 1, 'A', PATCH)
XX(POST, 1, 'R', PROPFIND)
XX(PUT, 2, 'R', PURGE)
XX(CONNECT, 1, 'H', CHECKOUT)
XX(CONNECT, 2, 'P', COPY)
XX(MKCOL, 1, 'O', MOVE)
XX(MKCOL, 1, 'E', MERGE)
XX(MKCOL, 1, '-', MSEARCH)
XX(MKCOL, 2, 'A', MKACTIVITY)
XX(MKCOL, 3, 'A', MKCALENDAR)
XX(SUBSCRIBE, 1, 'E', SEARCH)
XX(REPORT, 2, 'B', REBIND)
XX(PROPFIND, 4, 'P', PROPPATCH)
XX(LOCK, 1, 'I', LINK)
XX(UNLOCK, 2, 'S', UNSUBSCRIBE)
XX(UNLOCK, 2, 'B', UNBIND)
XX(UNLOCK, 3, 'I', UNLINK)
#undef XX
default:
SET_ERRNO(HPE_INVALID_METHOD);
goto error;
}
} else {
SET_ERRNO(HPE_INVALID_METHOD);
goto error;
}
++parser->index;
break;
}
case s_req_spaces_before_url:
{
if (ch == ' ') break;
MARK(url);
if (parser->method == HTTP_CONNECT) {
UPDATE_STATE(s_req_server_start);
}
UPDATE_STATE(parse_url_char(CURRENT_STATE(), ch));
if (UNLIKELY(CURRENT_STATE() == s_dead)) {
SET_ERRNO(HPE_INVALID_URL);
goto error;
}
break;
}
case s_req_schema:
case s_req_schema_slash:
case s_req_schema_slash_slash:
case s_req_server_start:
{
switch (ch) {
/* No whitespace allowed here */
case ' ':
case CR:
case LF:
SET_ERRNO(HPE_INVALID_URL);
goto error;
default:
UPDATE_STATE(parse_url_char(CURRENT_STATE(), ch));
if (UNLIKELY(CURRENT_STATE() == s_dead)) {
SET_ERRNO(HPE_INVALID_URL);
goto error;
}
}
break;
}
case s_req_server:
case s_req_server_with_at:
case s_req_path:
case s_req_query_string_start:
case s_req_query_string:
case s_req_fragment_start:
case s_req_fragment:
{
switch (ch) {
case ' ':
UPDATE_STATE(s_req_http_start);
CALLBACK_DATA(url);
break;
case CR:
case LF:
parser->http_major = 0;
parser->http_minor = 9;
UPDATE_STATE((ch == CR) ?
s_req_line_almost_done :
s_header_field_start);
CALLBACK_DATA(url);
break;
default:
UPDATE_STATE(parse_url_char(CURRENT_STATE(), ch));
if (UNLIKELY(CURRENT_STATE() == s_dead)) {
SET_ERRNO(HPE_INVALID_URL);
goto error;
}
}
break;
}
case s_req_http_start:
switch (ch) {
case 'H':
UPDATE_STATE(s_req_http_H);
break;
case ' ':
break;
default:
SET_ERRNO(HPE_INVALID_CONSTANT);
goto error;
}
break;
case s_req_http_H:
STRICT_CHECK(ch != 'T');
UPDATE_STATE(s_req_http_HT);
break;
case s_req_http_HT:
STRICT_CHECK(ch != 'T');
UPDATE_STATE(s_req_http_HTT);
break;
case s_req_http_HTT:
STRICT_CHECK(ch != 'P');
UPDATE_STATE(s_req_http_HTTP);
break;
case s_req_http_HTTP:
STRICT_CHECK(ch != '/');
UPDATE_STATE(s_req_http_major);
break;
case s_req_http_major:
if (UNLIKELY(!IS_NUM(ch))) {
SET_ERRNO(HPE_INVALID_VERSION);
goto error;
}
parser->http_major = ch - '0';
UPDATE_STATE(s_req_http_dot);
break;
case s_req_http_dot:
{
if (UNLIKELY(ch != '.')) {
SET_ERRNO(HPE_INVALID_VERSION);
goto error;
}
UPDATE_STATE(s_req_http_minor);
break;
}
case s_req_http_minor:
if (UNLIKELY(!IS_NUM(ch))) {
SET_ERRNO(HPE_INVALID_VERSION);
goto error;
}
parser->http_minor = ch - '0';
UPDATE_STATE(s_req_http_end);
break;
case s_req_http_end:
{
if (ch == CR) {
UPDATE_STATE(s_req_line_almost_done);
break;
}
if (ch == LF) {
UPDATE_STATE(s_header_field_start);
break;
}
SET_ERRNO(HPE_INVALID_VERSION);
goto error;
break;
}
/* end of request line */
case s_req_line_almost_done:
{
if (UNLIKELY(ch != LF)) {
SET_ERRNO(HPE_LF_EXPECTED);
goto error;
}
UPDATE_STATE(s_header_field_start);
break;
}
case s_header_field_start:
{
if (ch == CR) {
UPDATE_STATE(s_headers_almost_done);
break;
}
if (ch == LF) {
/* they might be just sending \n instead of \r\n so this would be
* the second \n to denote the end of headers*/
UPDATE_STATE(s_headers_almost_done);
REEXECUTE();
}
c = TOKEN(ch);
if (UNLIKELY(!c)) {
SET_ERRNO(HPE_INVALID_HEADER_TOKEN);
goto error;
}
MARK(header_field);
parser->index = 0;
UPDATE_STATE(s_header_field);
switch (c) {
case 'c':
parser->header_state = h_C;
break;
case 'p':
parser->header_state = h_matching_proxy_connection;
break;
case 't':
parser->header_state = h_matching_transfer_encoding;
break;
case 'u':
parser->header_state = h_matching_upgrade;
break;
default:
parser->header_state = h_general;
break;
}
break;
}
case s_header_field:
{
const char* start = p;
for (; p != data + len; p++) {
ch = *p;
c = TOKEN(ch);
if (!c)
break;
switch (parser->header_state) {
case h_general:
break;
case h_C:
parser->index++;
parser->header_state = (c == 'o' ? h_CO : h_general);
break;
case h_CO:
parser->index++;
parser->header_state = (c == 'n' ? h_CON : h_general);
break;
case h_CON:
parser->index++;
switch (c) {
case 'n':
parser->header_state = h_matching_connection;
break;
case 't':
parser->header_state = h_matching_content_length;
break;
default:
parser->header_state = h_general;
break;
}
break;
/* connection */
case h_matching_connection:
parser->index++;
if (parser->index > sizeof(CONNECTION)-1
|| c != CONNECTION[parser->index]) {
parser->header_state = h_general;
} else if (parser->index == sizeof(CONNECTION)-2) {
parser->header_state = h_connection;
}
break;
/* proxy-connection */
case h_matching_proxy_connection:
parser->index++;
if (parser->index > sizeof(PROXY_CONNECTION)-1
|| c != PROXY_CONNECTION[parser->index]) {
parser->header_state = h_general;
} else if (parser->index == sizeof(PROXY_CONNECTION)-2) {
parser->header_state = h_connection;
}
break;
/* content-length */
case h_matching_content_length:
parser->index++;
if (parser->index > sizeof(CONTENT_LENGTH)-1
|| c != CONTENT_LENGTH[parser->index]) {
parser->header_state = h_general;
} else if (parser->index == sizeof(CONTENT_LENGTH)-2) {
parser->header_state = h_content_length;
}
break;
/* transfer-encoding */
case h_matching_transfer_encoding:
parser->index++;
if (parser->index > sizeof(TRANSFER_ENCODING)-1
|| c != TRANSFER_ENCODING[parser->index]) {
parser->header_state = h_general;
} else if (parser->index == sizeof(TRANSFER_ENCODING)-2) {
parser->header_state = h_transfer_encoding;
}
break;
/* upgrade */
case h_matching_upgrade:
parser->index++;
if (parser->index > sizeof(UPGRADE)-1
|| c != UPGRADE[parser->index]) {
parser->header_state = h_general;
} else if (parser->index == sizeof(UPGRADE)-2) {
parser->header_state = h_upgrade;
}
break;
case h_connection:
case h_content_length:
case h_transfer_encoding:
case h_upgrade:
if (ch != ' ') parser->header_state = h_general;
break;
default:
assert(0 && "Unknown header_state");
break;
}
}
COUNT_HEADER_SIZE(p - start);
if (p == data + len) {
--p;
break;
}
if (ch == ':') {
UPDATE_STATE(s_header_value_discard_ws);
CALLBACK_DATA(header_field);
break;
}
SET_ERRNO(HPE_INVALID_HEADER_TOKEN);
goto error;
}
case s_header_value_discard_ws:
if (ch == ' ' || ch == '\t') break;
if (ch == CR) {
UPDATE_STATE(s_header_value_discard_ws_almost_done);
break;
}
if (ch == LF) {
UPDATE_STATE(s_header_value_discard_lws);
break;
}
/* FALLTHROUGH */
case s_header_value_start:
{
MARK(header_value);
UPDATE_STATE(s_header_value);
parser->index = 0;
c = LOWER(ch);
switch (parser->header_state) {
case h_upgrade:
parser->flags |= F_UPGRADE;
parser->header_state = h_general;
break;
case h_transfer_encoding:
/* looking for 'Transfer-Encoding: chunked' */
if ('c' == c) {
parser->header_state = h_matching_transfer_encoding_chunked;
} else {
parser->header_state = h_general;
}
break;
case h_content_length:
if (UNLIKELY(!IS_NUM(ch))) {
SET_ERRNO(HPE_INVALID_CONTENT_LENGTH);
goto error;
}
if (parser->flags & F_CONTENTLENGTH) {
SET_ERRNO(HPE_UNEXPECTED_CONTENT_LENGTH);
goto error;
}
parser->flags |= F_CONTENTLENGTH;
parser->content_length = ch - '0';
break;
case h_connection:
/* looking for 'Connection: keep-alive' */
if (c == 'k') {
parser->header_state = h_matching_connection_keep_alive;
/* looking for 'Connection: close' */
} else if (c == 'c') {
parser->header_state = h_matching_connection_close;
} else if (c == 'u') {
parser->header_state = h_matching_connection_upgrade;
} else {
parser->header_state = h_matching_connection_token;
}
break;
/* Multi-value `Connection` header */
case h_matching_connection_token_start:
break;
default:
parser->header_state = h_general;
break;
}
break;
}
case s_header_value:
{
const char* start = p;
enum header_states h_state = (enum header_states) parser->header_state;
for (; p != data + len; p++) {
ch = *p;
if (ch == CR) {
UPDATE_STATE(s_header_almost_done);
parser->header_state = h_state;
CALLBACK_DATA(header_value);
break;
}
if (ch == LF) {
UPDATE_STATE(s_header_almost_done);
COUNT_HEADER_SIZE(p - start);
parser->header_state = h_state;
CALLBACK_DATA_NOADVANCE(header_value);
REEXECUTE();
}
if (!lenient && !IS_HEADER_CHAR(ch)) {
SET_ERRNO(HPE_INVALID_HEADER_TOKEN);
goto error;
}
c = LOWER(ch);
switch (h_state) {
case h_general:
{
const char* p_cr;
const char* p_lf;
size_t limit = data + len - p;
limit = MIN(limit, HTTP_MAX_HEADER_SIZE);
p_cr = (const char*) memchr(p, CR, limit);
p_lf = (const char*) memchr(p, LF, limit);
if (p_cr != NULL) {
if (p_lf != NULL && p_cr >= p_lf)
p = p_lf;
else
p = p_cr;
} else if (UNLIKELY(p_lf != NULL)) {
p = p_lf;
} else {
p = data + len;
}
--p;
break;
}
case h_connection:
case h_transfer_encoding:
assert(0 && "Shouldn't get here.");
break;
case h_content_length:
{
uint64_t t;
if (ch == ' ') break;
if (UNLIKELY(!IS_NUM(ch))) {
SET_ERRNO(HPE_INVALID_CONTENT_LENGTH);
parser->header_state = h_state;
goto error;
}
t = parser->content_length;
t *= 10;
t += ch - '0';
/* Overflow? Test against a conservative limit for simplicity. */
if (UNLIKELY((ULLONG_MAX - 10) / 10 < parser->content_length)) {
SET_ERRNO(HPE_INVALID_CONTENT_LENGTH);
parser->header_state = h_state;
goto error;
}
parser->content_length = t;
break;
}
/* Transfer-Encoding: chunked */
case h_matching_transfer_encoding_chunked:
parser->index++;
if (parser->index > sizeof(CHUNKED)-1
|| c != CHUNKED[parser->index]) {
h_state = h_general;
} else if (parser->index == sizeof(CHUNKED)-2) {
h_state = h_transfer_encoding_chunked;
}
break;
case h_matching_connection_token_start:
/* looking for 'Connection: keep-alive' */
if (c == 'k') {
h_state = h_matching_connection_keep_alive;
/* looking for 'Connection: close' */
} else if (c == 'c') {
h_state = h_matching_connection_close;
} else if (c == 'u') {
h_state = h_matching_connection_upgrade;
} else if (STRICT_TOKEN(c)) {
h_state = h_matching_connection_token;
} else if (c == ' ' || c == '\t') {
/* Skip lws */
} else {
h_state = h_general;
}
break;
/* looking for 'Connection: keep-alive' */
case h_matching_connection_keep_alive:
parser->index++;
if (parser->index > sizeof(KEEP_ALIVE)-1
|| c != KEEP_ALIVE[parser->index]) {
h_state = h_matching_connection_token;
} else if (parser->index == sizeof(KEEP_ALIVE)-2) {
h_state = h_connection_keep_alive;
}
break;
/* looking for 'Connection: close' */
case h_matching_connection_close:
parser->index++;
if (parser->index > sizeof(CLOSE)-1 || c != CLOSE[parser->index]) {
h_state = h_matching_connection_token;
} else if (parser->index == sizeof(CLOSE)-2) {
h_state = h_connection_close;
}
break;
/* looking for 'Connection: upgrade' */
case h_matching_connection_upgrade:
parser->index++;
if (parser->index > sizeof(UPGRADE) - 1 ||
c != UPGRADE[parser->index]) {
h_state = h_matching_connection_token;
} else if (parser->index == sizeof(UPGRADE)-2) {
h_state = h_connection_upgrade;
}
break;
case h_matching_connection_token:
if (ch == ',') {
h_state = h_matching_connection_token_start;
parser->index = 0;
}
break;
case h_transfer_encoding_chunked:
if (ch != ' ') h_state = h_general;
break;
case h_connection_keep_alive:
case h_connection_close:
case h_connection_upgrade:
if (ch == ',') {
if (h_state == h_connection_keep_alive) {
parser->flags |= F_CONNECTION_KEEP_ALIVE;
} else if (h_state == h_connection_close) {
parser->flags |= F_CONNECTION_CLOSE;
} else if (h_state == h_connection_upgrade) {
parser->flags |= F_CONNECTION_UPGRADE;
}
h_state = h_matching_connection_token_start;
parser->index = 0;
} else if (ch != ' ') {
h_state = h_matching_connection_token;
}
break;
default:
UPDATE_STATE(s_header_value);
h_state = h_general;
break;
}
}
parser->header_state = h_state;
COUNT_HEADER_SIZE(p - start);
if (p == data + len)
--p;
break;
}
case s_header_almost_done:
{
if (UNLIKELY(ch != LF)) {
SET_ERRNO(HPE_LF_EXPECTED);
goto error;
}
UPDATE_STATE(s_header_value_lws);
break;
}
case s_header_value_lws:
{
if (ch == ' ' || ch == '\t') {
UPDATE_STATE(s_header_value_start);
REEXECUTE();
}
/* finished the header */
switch (parser->header_state) {
case h_connection_keep_alive:
parser->flags |= F_CONNECTION_KEEP_ALIVE;
break;
case h_connection_close:
parser->flags |= F_CONNECTION_CLOSE;
break;
case h_transfer_encoding_chunked:
parser->flags |= F_CHUNKED;
break;
case h_connection_upgrade:
parser->flags |= F_CONNECTION_UPGRADE;
break;
default:
break;
}
UPDATE_STATE(s_header_field_start);
REEXECUTE();
}
case s_header_value_discard_ws_almost_done:
{
STRICT_CHECK(ch != LF);
UPDATE_STATE(s_header_value_discard_lws);
break;
}
case s_header_value_discard_lws:
{
if (ch == ' ' || ch == '\t') {
UPDATE_STATE(s_header_value_discard_ws);
break;
} else {
switch (parser->header_state) {
case h_connection_keep_alive:
parser->flags |= F_CONNECTION_KEEP_ALIVE;
break;
case h_connection_close:
parser->flags |= F_CONNECTION_CLOSE;
break;
case h_connection_upgrade:
parser->flags |= F_CONNECTION_UPGRADE;
break;
case h_transfer_encoding_chunked:
parser->flags |= F_CHUNKED;
break;
default:
break;
}
/* header value was empty */
MARK(header_value);
UPDATE_STATE(s_header_field_start);
CALLBACK_DATA_NOADVANCE(header_value);
REEXECUTE();
}
}
case s_headers_almost_done:
{
STRICT_CHECK(ch != LF);
if (parser->flags & F_TRAILING) {
/* End of a chunked request */
UPDATE_STATE(s_message_done);
CALLBACK_NOTIFY_NOADVANCE(chunk_complete);
REEXECUTE();
}
/* Cannot use chunked encoding and a content-length header together
per the HTTP specification. */
if ((parser->flags & F_CHUNKED) &&
(parser->flags & F_CONTENTLENGTH)) {
SET_ERRNO(HPE_UNEXPECTED_CONTENT_LENGTH);
goto error;
}
UPDATE_STATE(s_headers_done);
/* Set this here so that on_headers_complete() callbacks can see it */
if ((parser->flags & F_UPGRADE) &&
(parser->flags & F_CONNECTION_UPGRADE)) {
/* For responses, "Upgrade: foo" and "Connection: upgrade" are
* mandatory only when it is a 101 Switching Protocols response,
* otherwise it is purely informational, to announce support.
*/
parser->upgrade =
(parser->type == HTTP_REQUEST || parser->status_code == 101);
} else {
parser->upgrade = (parser->method == HTTP_CONNECT);
}
/* Here we call the headers_complete callback. This is somewhat
* different than other callbacks because if the user returns 1, we
* will interpret that as saying that this message has no body. This
* is needed for the annoying case of recieving a response to a HEAD
* request.
*
* We'd like to use CALLBACK_NOTIFY_NOADVANCE() here but we cannot, so
* we have to simulate it by handling a change in errno below.
*/
if (settings->on_headers_complete) {
switch (settings->on_headers_complete(parser)) {
case 0:
break;
case 2:
parser->upgrade = 1;
/* FALLTHROUGH */
case 1:
parser->flags |= F_SKIPBODY;
break;
default:
SET_ERRNO(HPE_CB_headers_complete);
RETURN(p - data); /* Error */
}
}
if (HTTP_PARSER_ERRNO(parser) != HPE_OK) {
RETURN(p - data);
}
REEXECUTE();
}
case s_headers_done:
{
int hasBody;
STRICT_CHECK(ch != LF);
parser->nread = 0;
hasBody = parser->flags & F_CHUNKED ||
(parser->content_length > 0 && parser->content_length != ULLONG_MAX);
if (parser->upgrade && (parser->method == HTTP_CONNECT ||
(parser->flags & F_SKIPBODY) || !hasBody)) {
/* Exit, the rest of the message is in a different protocol. */
UPDATE_STATE(NEW_MESSAGE());
CALLBACK_NOTIFY(message_complete);
RETURN((p - data) + 1);
}
if (parser->flags & F_SKIPBODY) {
UPDATE_STATE(NEW_MESSAGE());
CALLBACK_NOTIFY(message_complete);
} else if (parser->flags & F_CHUNKED) {
/* chunked encoding - ignore Content-Length header */
UPDATE_STATE(s_chunk_size_start);
} else {
if (parser->content_length == 0) {
/* Content-Length header given but zero: Content-Length: 0\r\n */
UPDATE_STATE(NEW_MESSAGE());
CALLBACK_NOTIFY(message_complete);
} else if (parser->content_length != ULLONG_MAX) {
/* Content-Length header given and non-zero */
UPDATE_STATE(s_body_identity);
} else {
if (!http_message_needs_eof(parser)) {
/* Assume content-length 0 - read the next */
UPDATE_STATE(NEW_MESSAGE());
CALLBACK_NOTIFY(message_complete);
} else {
/* Read body until EOF */
UPDATE_STATE(s_body_identity_eof);
}
}
}
break;
}
case s_body_identity:
{
uint64_t to_read = MIN(parser->content_length,
(uint64_t) ((data + len) - p));
assert(parser->content_length != 0
&& parser->content_length != ULLONG_MAX);
/* The difference between advancing content_length and p is because
* the latter will automaticaly advance on the next loop iteration.
* Further, if content_length ends up at 0, we want to see the last
* byte again for our message complete callback.
*/
MARK(body);
parser->content_length -= to_read;
p += to_read - 1;
if (parser->content_length == 0) {
UPDATE_STATE(s_message_done);
/* Mimic CALLBACK_DATA_NOADVANCE() but with one extra byte.
*
* The alternative to doing this is to wait for the next byte to
* trigger the data callback, just as in every other case. The
* problem with this is that this makes it difficult for the test
* harness to distinguish between complete-on-EOF and
* complete-on-length. It's not clear that this distinction is
* important for applications, but let's keep it for now.
*/
CALLBACK_DATA_(body, p - body_mark + 1, p - data);
REEXECUTE();
}
break;
}
/* read until EOF */
case s_body_identity_eof:
MARK(body);
p = data + len - 1;
break;
case s_message_done:
UPDATE_STATE(NEW_MESSAGE());
CALLBACK_NOTIFY(message_complete);
if (parser->upgrade) {
/* Exit, the rest of the message is in a different protocol. */
RETURN((p - data) + 1);
}
break;
case s_chunk_size_start:
{
assert(parser->nread == 1);
assert(parser->flags & F_CHUNKED);
unhex_val = unhex[(unsigned char)ch];
if (UNLIKELY(unhex_val == -1)) {
SET_ERRNO(HPE_INVALID_CHUNK_SIZE);
goto error;
}
parser->content_length = unhex_val;
UPDATE_STATE(s_chunk_size);
break;
}
case s_chunk_size:
{
uint64_t t;
assert(parser->flags & F_CHUNKED);
if (ch == CR) {
UPDATE_STATE(s_chunk_size_almost_done);
break;
}
unhex_val = unhex[(unsigned char)ch];
if (unhex_val == -1) {
if (ch == ';' || ch == ' ') {
UPDATE_STATE(s_chunk_parameters);
break;
}
SET_ERRNO(HPE_INVALID_CHUNK_SIZE);
goto error;
}
t = parser->content_length;
t *= 16;
t += unhex_val;
/* Overflow? Test against a conservative limit for simplicity. */
if (UNLIKELY((ULLONG_MAX - 16) / 16 < parser->content_length)) {
SET_ERRNO(HPE_INVALID_CONTENT_LENGTH);
goto error;
}
parser->content_length = t;
break;
}
case s_chunk_parameters:
{
assert(parser->flags & F_CHUNKED);
/* just ignore this shit. TODO check for overflow */
if (ch == CR) {
UPDATE_STATE(s_chunk_size_almost_done);
break;
}
break;
}
case s_chunk_size_almost_done:
{
assert(parser->flags & F_CHUNKED);
STRICT_CHECK(ch != LF);
parser->nread = 0;
if (parser->content_length == 0) {
parser->flags |= F_TRAILING;
UPDATE_STATE(s_header_field_start);
} else {
UPDATE_STATE(s_chunk_data);
}
CALLBACK_NOTIFY(chunk_header);
break;
}
case s_chunk_data:
{
uint64_t to_read = MIN(parser->content_length,
(uint64_t) ((data + len) - p));
assert(parser->flags & F_CHUNKED);
assert(parser->content_length != 0
&& parser->content_length != ULLONG_MAX);
/* See the explanation in s_body_identity for why the content
* length and data pointers are managed this way.
*/
MARK(body);
parser->content_length -= to_read;
p += to_read - 1;
if (parser->content_length == 0) {
UPDATE_STATE(s_chunk_data_almost_done);
}
break;
}
case s_chunk_data_almost_done:
assert(parser->flags & F_CHUNKED);
assert(parser->content_length == 0);
STRICT_CHECK(ch != CR);
UPDATE_STATE(s_chunk_data_done);
CALLBACK_DATA(body);
break;
case s_chunk_data_done:
assert(parser->flags & F_CHUNKED);
STRICT_CHECK(ch != LF);
parser->nread = 0;
UPDATE_STATE(s_chunk_size_start);
CALLBACK_NOTIFY(chunk_complete);
break;
default:
assert(0 && "unhandled state");
SET_ERRNO(HPE_INVALID_INTERNAL_STATE);
goto error;
}
}
/* Run callbacks for any marks that we have leftover after we ran our of
* bytes. There should be at most one of these set, so it's OK to invoke
* them in series (unset marks will not result in callbacks).
*
* We use the NOADVANCE() variety of callbacks here because 'p' has already
* overflowed 'data' and this allows us to correct for the off-by-one that
* we'd otherwise have (since CALLBACK_DATA() is meant to be run with a 'p'
* value that's in-bounds).
*/
assert(((header_field_mark ? 1 : 0) +
(header_value_mark ? 1 : 0) +
(url_mark ? 1 : 0) +
(body_mark ? 1 : 0) +
(status_mark ? 1 : 0)) <= 1);
CALLBACK_DATA_NOADVANCE(header_field);
CALLBACK_DATA_NOADVANCE(header_value);
CALLBACK_DATA_NOADVANCE(url);
CALLBACK_DATA_NOADVANCE(body);
CALLBACK_DATA_NOADVANCE(status);
RETURN(len);
error:
if (HTTP_PARSER_ERRNO(parser) == HPE_OK) {
SET_ERRNO(HPE_UNKNOWN);
}
RETURN(p - data);
}
/* Does the parser need to see an EOF to find the end of the message? */
int
http_message_needs_eof (const http_parser *parser)
{
if (parser->type == HTTP_REQUEST) {
return 0;
}
/* See RFC 2616 section 4.4 */
if (parser->status_code / 100 == 1 || /* 1xx e.g. Continue */
parser->status_code == 204 || /* No Content */
parser->status_code == 304 || /* Not Modified */
parser->flags & F_SKIPBODY) { /* response to a HEAD request */
return 0;
}
if ((parser->flags & F_CHUNKED) || parser->content_length != ULLONG_MAX) {
return 0;
}
return 1;
}
int
http_should_keep_alive (const http_parser *parser)
{
if (parser->http_major > 0 && parser->http_minor > 0) {
/* HTTP/1.1 */
if (parser->flags & F_CONNECTION_CLOSE) {
return 0;
}
} else {
/* HTTP/1.0 or earlier */
if (!(parser->flags & F_CONNECTION_KEEP_ALIVE)) {
return 0;
}
}
return !http_message_needs_eof(parser);
}
const char *
http_method_str (enum http_method m)
{
return ELEM_AT(method_strings, m, "<unknown>");
}
void
http_parser_init (http_parser *parser, enum http_parser_type t)
{
void *data = parser->data; /* preserve application data */
memset(parser, 0, sizeof(*parser));
parser->data = data;
parser->type = t;
parser->state = (t == HTTP_REQUEST ? s_start_req : (t == HTTP_RESPONSE ? s_start_res : s_start_req_or_res));
parser->http_errno = HPE_OK;
}
void
http_parser_settings_init(http_parser_settings *settings)
{
memset(settings, 0, sizeof(*settings));
}
const char *
http_errno_name(enum http_errno err) {
assert(((size_t) err) < ARRAY_SIZE(http_strerror_tab));
return http_strerror_tab[err].name;
}
const char *
http_errno_description(enum http_errno err) {
assert(((size_t) err) < ARRAY_SIZE(http_strerror_tab));
return http_strerror_tab[err].description;
}
static enum http_host_state
http_parse_host_char(enum http_host_state s, const char ch) {
switch(s) {
case s_http_userinfo:
case s_http_userinfo_start:
if (ch == '@') {
return s_http_host_start;
}
if (IS_USERINFO_CHAR(ch)) {
return s_http_userinfo;
}
break;
case s_http_host_start:
if (ch == '[') {
return s_http_host_v6_start;
}
if (IS_HOST_CHAR(ch)) {
return s_http_host;
}
break;
case s_http_host:
if (IS_HOST_CHAR(ch)) {
return s_http_host;
}
/* FALLTHROUGH */
case s_http_host_v6_end:
if (ch == ':') {
return s_http_host_port_start;
}
break;
case s_http_host_v6:
if (ch == ']') {
return s_http_host_v6_end;
}
/* FALLTHROUGH */
case s_http_host_v6_start:
if (IS_HEX(ch) || ch == ':' || ch == '.') {
return s_http_host_v6;
}
if (s == s_http_host_v6 && ch == '%') {
return s_http_host_v6_zone_start;
}
break;
case s_http_host_v6_zone:
if (ch == ']') {
return s_http_host_v6_end;
}
/* FALLTHROUGH */
case s_http_host_v6_zone_start:
/* RFC 6874 Zone ID consists of 1*( unreserved / pct-encoded) */
if (IS_ALPHANUM(ch) || ch == '%' || ch == '.' || ch == '-' || ch == '_' ||
ch == '~') {
return s_http_host_v6_zone;
}
break;
case s_http_host_port:
case s_http_host_port_start:
if (IS_NUM(ch)) {
return s_http_host_port;
}
break;
default:
break;
}
return s_http_host_dead;
}
static int
http_parse_host(const char * buf, struct http_parser_url *u, int found_at) {
enum http_host_state s;
const char *p;
size_t buflen = u->field_data[UF_HOST].off + u->field_data[UF_HOST].len;
assert(u->field_set & (1 << UF_HOST));
u->field_data[UF_HOST].len = 0;
s = found_at ? s_http_userinfo_start : s_http_host_start;
for (p = buf + u->field_data[UF_HOST].off; p < buf + buflen; p++) {
enum http_host_state new_s = http_parse_host_char(s, *p);
if (new_s == s_http_host_dead) {
return 1;
}
switch(new_s) {
case s_http_host:
if (s != s_http_host) {
u->field_data[UF_HOST].off = p - buf;
}
u->field_data[UF_HOST].len++;
break;
case s_http_host_v6:
if (s != s_http_host_v6) {
u->field_data[UF_HOST].off = p - buf;
}
u->field_data[UF_HOST].len++;
break;
case s_http_host_v6_zone_start:
case s_http_host_v6_zone:
u->field_data[UF_HOST].len++;
break;
case s_http_host_port:
if (s != s_http_host_port) {
u->field_data[UF_PORT].off = p - buf;
u->field_data[UF_PORT].len = 0;
u->field_set |= (1 << UF_PORT);
}
u->field_data[UF_PORT].len++;
break;
case s_http_userinfo:
if (s != s_http_userinfo) {
u->field_data[UF_USERINFO].off = p - buf ;
u->field_data[UF_USERINFO].len = 0;
u->field_set |= (1 << UF_USERINFO);
}
u->field_data[UF_USERINFO].len++;
break;
default:
break;
}
s = new_s;
}
/* Make sure we don't end somewhere unexpected */
switch (s) {
case s_http_host_start:
case s_http_host_v6_start:
case s_http_host_v6:
case s_http_host_v6_zone_start:
case s_http_host_v6_zone:
case s_http_host_port_start:
case s_http_userinfo:
case s_http_userinfo_start:
return 1;
default:
break;
}
return 0;
}
void
http_parser_url_init(struct http_parser_url *u) {
memset(u, 0, sizeof(*u));
}
int
http_parser_parse_url(const char *buf, size_t buflen, int is_connect,
struct http_parser_url *u)
{
enum state s;
const char *p;
enum http_parser_url_fields uf, old_uf;
int found_at = 0;
u->port = u->field_set = 0;
s = is_connect ? s_req_server_start : s_req_spaces_before_url;
old_uf = UF_MAX;
for (p = buf; p < buf + buflen; p++) {
s = parse_url_char(s, *p);
/* Figure out the next field that we're operating on */
switch (s) {
case s_dead:
return 1;
/* Skip delimeters */
case s_req_schema_slash:
case s_req_schema_slash_slash:
case s_req_server_start:
case s_req_query_string_start:
case s_req_fragment_start:
continue;
case s_req_schema:
uf = UF_SCHEMA;
break;
case s_req_server_with_at:
found_at = 1;
/* FALLTHROUGH */
case s_req_server:
uf = UF_HOST;
break;
case s_req_path:
uf = UF_PATH;
break;
case s_req_query_string:
uf = UF_QUERY;
break;
case s_req_fragment:
uf = UF_FRAGMENT;
break;
default:
assert(!"Unexpected state");
return 1;
}
/* Nothing's changed; soldier on */
if (uf == old_uf) {
u->field_data[uf].len++;
continue;
}
u->field_data[uf].off = p - buf;
u->field_data[uf].len = 1;
u->field_set |= (1 << uf);
old_uf = uf;
}
/* host must be present if there is a schema */
/* parsing http:///toto will fail */
if ((u->field_set & (1 << UF_SCHEMA)) &&
(u->field_set & (1 << UF_HOST)) == 0) {
return 1;
}
if (u->field_set & (1 << UF_HOST)) {
if (http_parse_host(buf, u, found_at) != 0) {
return 1;
}
}
/* CONNECT requests can only contain "hostname:port" */
if (is_connect && u->field_set != ((1 << UF_HOST)|(1 << UF_PORT))) {
return 1;
}
if (u->field_set & (1 << UF_PORT)) {
/* Don't bother with endp; we've already validated the string */
unsigned long v = strtoul(buf + u->field_data[UF_PORT].off, NULL, 10);
/* Ports have a max value of 2^16 */
if (v > 0xffff) {
return 1;
}
u->port = (uint16_t) v;
}
return 0;
}
void
http_parser_pause(http_parser *parser, int paused) {
/* Users should only be pausing/unpausing a parser that is not in an error
* state. In non-debug builds, there's not much that we can do about this
* other than ignore it.
*/
if (HTTP_PARSER_ERRNO(parser) == HPE_OK ||
HTTP_PARSER_ERRNO(parser) == HPE_PAUSED) {
SET_ERRNO((paused) ? HPE_PAUSED : HPE_OK);
} else {
assert(0 && "Attempting to pause parser in error state");
}
}
int
http_body_is_final(const struct http_parser *parser) {
return parser->state == s_message_done;
}
unsigned long
http_parser_version(void) {
return HTTP_PARSER_VERSION_MAJOR * 0x10000 |
HTTP_PARSER_VERSION_MINOR * 0x00100 |
HTTP_PARSER_VERSION_PATCH * 0x00001;
}
================================================
FILE: src/http_parser.h
================================================
/* Copyright Joyent, Inc. and other Node contributors. All rights reserved.
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to
* deal in the Software without restriction, including without limitation the
* rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
* sell copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
* IN THE SOFTWARE.
*/
#ifndef http_parser_h
#define http_parser_h
#ifdef __cplusplus
extern "C" {
#endif
/* Also update SONAME in the Makefile whenever you change these. */
#define HTTP_PARSER_VERSION_MAJOR 2
#define HTTP_PARSER_VERSION_MINOR 7
#define HTTP_PARSER_VERSION_PATCH 1
#include <stddef.h>
#if defined(_WIN32) && !defined(__MINGW32__) && \
(!defined(_MSC_VER) || _MSC_VER<1600) && !defined(__WINE__)
#include <BaseTsd.h>
typedef __int8 int8_t;
typedef unsigned __int8 uint8_t;
typedef __int16 int16_t;
typedef unsigned __int16 uint16_t;
typedef __int32 int32_t;
typedef unsigned __int32 uint32_t;
typedef __int64 int64_t;
typedef unsigned __int64 uint64_t;
#else
#include <stdint.h>
#endif
/* Compile with -DHTTP_PARSER_STRICT=0 to make less checks, but run
* faster
*/
#ifndef HTTP_PARSER_STRICT
# define HTTP_PARSER_STRICT 1
#endif
/* Maximium header size allowed. If the macro is not defined
* before including this header then the default is used. To
* change the maximum header size, define the macro in the build
* environment (e.g. -DHTTP_MAX_HEADER_SIZE=<value>). To remove
* the effective limit on the size of the header, define the macro
* to a very large number (e.g. -DHTTP_MAX_HEADER_SIZE=0x7fffffff)
*/
#ifndef HTTP_MAX_HEADER_SIZE
# define HTTP_MAX_HEADER_SIZE (80*1024)
#endif
typedef struct http_parser http_parser;
typedef struct http_parser_settings http_parser_settings;
/* Callbacks should return non-zero to indicate an error. The parser will
* then halt execution.
*
* The one exception is on_headers_complete. In a HTTP_RESPONSE parser
* returning '1' from on_headers_complete will tell the parser that it
* should not expect a body. This is used when receiving a response to a
* HEAD request which may contain 'Content-Length' or 'Transfer-Encoding:
* chunked' headers that indicate the presence of a body.
*
* Returning `2` from on_headers_complete will tell parser that it should not
* expect neither a body nor any futher responses on this connection. This is
* useful for handling responses to a CONNECT request which may not contain
* `Upgrade` or `Connection: upgrade` headers.
*
* http_data_cb does not return data chunks. It will be called arbitrarily
* many times for each string. E.G. you might get 10 callbacks for "on_url"
* each providing just a few characters more data.
*/
typedef int (*http_data_cb) (http_parser*, const char *at, size_t length);
typedef int (*http_cb) (http_parser*);
/* Status Codes */
#define HTTP_STATUS_MAP(XX) \
XX(100, CONTINUE, Continue) \
XX(101, SWITCHING_PROTOCOLS, Switching Protocols) \
XX(102, PROCESSING, Processing) \
XX(200, OK, OK) \
XX(201, CREATED, Created) \
XX(202, ACCEPTED, Accepted) \
XX(203, NON_AUTHORITATIVE_INFORMATION, Non-Authoritative Information) \
XX(204, NO_CONTENT, No Content) \
XX(205, RESET_CONTENT, Reset Content) \
XX(206, PARTIAL_CONTENT, Partial Content) \
XX(207, MULTI_STATUS, Multi-Status) \
XX(208, ALREADY_REPORTED, Already Reported) \
XX(226, IM_USED, IM Used) \
XX(300, MULTIPLE_CHOICES, Multiple Choices) \
XX(301, MOVED_PERMANENTLY, Moved Permanently) \
XX(302, FOUND, Found) \
XX(303, SEE_OTHER, See Other) \
XX(304, NOT_MODIFIED, Not Modified) \
XX(305, USE_PROXY, Use Proxy) \
XX(307, TEMPORARY_REDIRECT, Temporary Redirect) \
XX(308, PERMANENT_REDIRECT, Permanent Redirect) \
XX(400, BAD_REQUEST, Bad Request) \
XX(401, UNAUTHORIZED, Unauthorized) \
XX(402, PAYMENT_REQUIRED, Payment Required) \
XX(403, FORBIDDEN, Forbidden) \
XX(404, NOT_FOUND, Not Found) \
XX(405, METHOD_NOT_ALLOWED, Method Not Allowed) \
XX(406, NOT_ACCEPTABLE, Not Acceptable) \
XX(407, PROXY_AUTHENTICATION_REQUIRED, Proxy Authentication Required) \
XX(408, REQUEST_TIMEOUT, Request Timeout) \
XX(409, CONFLICT, Conflict) \
XX(410, GONE, Gone) \
XX(411, LENGTH_REQUIRED, Length Required) \
XX(412, PRECONDITION_FAILED, Precondition Failed) \
XX(413, PAYLOAD_TOO_LARGE, Payload Too Large) \
XX(414, URI_TOO_LONG, URI Too Long) \
XX(415, UNSUPPORTED_MEDIA_TYPE, Unsupported Media Type) \
XX(416, RANGE_NOT_SATISFIABLE, Range Not Satisfiable) \
XX(417, EXPECTATION_FAILED, Expectation Failed) \
XX(421, MISDIRECTED_REQUEST, Misdirected Request) \
XX(422, UNPROCESSABLE_ENTITY, Unprocessable Entity) \
XX(423, LOCKED, Locked) \
XX(424, FAILED_DEPENDENCY, Failed Dependency) \
XX(426, UPGRADE_REQUIRED, Upgrade Required) \
XX(428, PRECONDITION_REQUIRED, Precondition Required) \
XX(429, TOO_MANY_REQUESTS, Too Many Requests) \
XX(431, REQUEST_HEADER_FIELDS_TOO_LARGE, Request Header Fields Too Large) \
XX(451, UNAVAILABLE_FOR_LEGAL_REASONS, Unavailable For Legal Reasons) \
XX(500, INTERNAL_SERVER_ERROR, Internal Server Error) \
XX(501, NOT_IMPLEMENTED, Not Implemented) \
XX(502, BAD_GATEWAY, Bad Gateway) \
XX(503, SERVICE_UNAVAILABLE, Service Unavailable) \
XX(504, GATEWAY_TIMEOUT, Gateway Timeout) \
XX(505, HTTP_VERSION_NOT_SUPPORTED, HTTP Version Not Supported) \
XX(506, VARIANT_ALSO_NEGOTIATES, Variant Also Negotiates) \
XX(507, INSUFFICIENT_STORAGE, Insufficient Storage) \
XX(508, LOOP_DETECTED, Loop Detected) \
XX(510, NOT_EXTENDED, Not Extended) \
XX(511, NETWORK_AUTHENTICATION_REQUIRED, Network Authentication Required) \
enum http_status
{
#define XX(num, name, string) HTTP_STATUS_##name = num,
HTTP_STATUS_MAP(XX)
#undef XX
};
/* Request Methods */
#define HTTP_METHOD_MAP(XX) \
XX(0, DELETE, DELETE) \
XX(1, GET, GET) \
XX(2, HEAD, HEAD) \
XX(3, POST, POST) \
XX(4, PUT, PUT) \
/* pathological */ \
XX(5, CONNECT, CONNECT) \
XX(6, OPTIONS, OPTIONS) \
XX(7, TRACE, TRACE) \
/* WebDAV */ \
XX(8, COPY, COPY) \
XX(9, LOCK, LOCK) \
XX(10, MKCOL, MKCOL) \
XX(11, MOVE, MOVE) \
XX(12, PROPFIND, PROPFIND) \
XX(13, PROPPATCH, PROPPATCH) \
XX(14, SEARCH, SEARCH) \
XX(15, UNLOCK, UNLOCK) \
XX(16, BIND, BIND) \
XX(17, REBIND, REBIND) \
XX(18, UNBIND, UNBIND) \
XX(19, ACL, ACL) \
/* subversion */ \
XX(20, REPORT, REPORT) \
XX(21, MKACTIVITY, MKACTIVITY) \
XX(22, CHECKOUT, CHECKOUT) \
XX(23, MERGE, MERGE) \
/* upnp */ \
XX(24, MSEARCH, M-SEARCH) \
XX(25, NOTIFY, NOTIFY) \
XX(26, SUBSCRIBE, SUBSCRIBE) \
XX(27, UNSUBSCRIBE, UNSUBSCRIBE) \
/* RFC-5789 */ \
XX(28, PATCH, PATCH) \
XX(29, PURGE, PURGE) \
/* CalDAV */ \
XX(30, MKCALENDAR, MKCALENDAR) \
/* RFC-2068, section 19.6.1.2 */ \
XX(31, LINK, LINK) \
XX(32, UNLINK, UNLINK) \
enum http_method
{
#define XX(num, name, string) HTTP_##name = num,
HTTP_METHOD_MAP(XX)
#undef XX
};
enum http_parser_type { HTTP_REQUEST, HTTP_RESPONSE, HTTP_BOTH };
/* Flag values for http_parser.flags field */
enum flags
{ F_CHUNKED = 1 << 0
, F_CONNECTION_KEEP_ALIVE = 1 << 1
, F_CONNECTION_CLOSE = 1 << 2
, F_CONNECTION_UPGRADE = 1 << 3
, F_TRAILING = 1 << 4
, F_UPGRADE = 1 << 5
, F_SKIPBODY = 1 << 6
, F_CONTENTLENGTH = 1 << 7
};
/* Map for errno-related constants
*
* The provided argument should be a macro that takes 2 arguments.
*/
#define HTTP_ERRNO_MAP(XX) \
/* No error */ \
XX(OK, "success") \
\
/* Callback-related errors */ \
XX(CB_message_begin, "the on_message_begin callback failed") \
XX(CB_url, "the on_url callback failed") \
XX(CB_header_field, "the on_header_field callback failed") \
XX(CB_header_value, "the on_header_value callback failed") \
XX(CB_headers_complete, "the on_headers_complete callback failed") \
XX(CB_body, "the on_body callback failed") \
XX(CB_message_complete, "the on_message_complete callback failed") \
XX(CB_status, "the on_status callback failed") \
XX(CB_chunk_header, "the on_chunk_header callback failed") \
XX(CB_chunk_complete, "the on_chunk_complete callback failed") \
\
/* Parsing-related errors */ \
XX(INVALID_EOF_STATE, "stream ended at an unexpected time") \
XX(HEADER_OVERFLOW, \
"too many header bytes seen; overflow detected") \
XX(CLOSED_CONNECTION, \
"data received after completed connection: close message") \
XX(INVALID_VERSION, "invalid HTTP version") \
XX(INVALID_STATUS, "invalid HTTP status code") \
XX(INVALID_METHOD, "invalid HTTP method") \
XX(INVALID_URL, "invalid URL") \
XX(INVALID_HOST, "invalid host") \
XX(INVALID_PORT, "invalid port") \
XX(INVALID_PATH, "invalid path") \
XX(INVALID_QUERY_STRING, "invalid query string") \
XX(INVALID_FRAGMENT, "invalid fragment") \
XX(LF_EXPECTED, "LF character expected") \
XX(INVALID_HEADER_TOKEN, "invalid character in header") \
XX(INVALID_CONTENT_LENGTH, \
"invalid character in content-length header") \
XX(UNEXPECTED_CONTENT_LENGTH, \
"unexpected content-length header") \
XX(INVALID_CHUNK_SIZE, \
"invalid character in chunk size header") \
XX(INVALID_CONSTANT, "invalid constant string") \
XX(INVALID_INTERNAL_STATE, "encountered unexpected internal state")\
XX(STRICT, "strict mode assertion failed") \
XX(PAUSED, "parser is paused") \
XX(UNKNOWN, "an unknown error occurred")
/* Define HPE_* values for each errno value above */
#define HTTP_ERRNO_GEN(n, s) HPE_##n,
enum http_errno {
HTTP_ERRNO_MAP(HTTP_ERRNO_GEN)
};
#undef HTTP_ERRNO_GEN
/* Get an http_errno value from an http_parser */
#define HTTP_PARSER_ERRNO(p) ((enum http_errno) (p)->http_errno)
struct http_parser {
/** PRIVATE **/
unsigned int type : 2; /* enum http_parser_type */
unsigned int flags : 8; /* F_* values from 'flags' enum; semi-public */
unsigned int state : 7; /* enum state from http_parser.c */
unsigned int header_state : 7; /* enum header_state from http_parser.c */
unsigned int index : 7; /* index into current matcher */
unsigned int lenient_http_headers : 1;
uint32_t nread; /* # bytes read in various scenarios */
uint64_t content_length; /* # bytes in body (0 if no Content-Length header) */
/** READ-ONLY **/
unsigned short http_major;
unsigned short http_minor;
unsigned int status_code : 16; /* responses only */
unsigned int method : 8; /* requests only */
unsigned int http_errno : 7;
/* 1 = Upgrade header was present and the parser has exited because of that.
* 0 = No upgrade header present.
* Should be checked when http_parser_execute() returns in addition to
* error checking.
*/
unsigned int upgrade : 1;
/** PUBLIC **/
void *data; /* A pointer to get hook to the "connection" or "socket" object */
};
struct http_parser_settings {
http_cb on_message_begin;
http_data_cb on_url;
http_data_cb on_status;
http_data_cb on_header_field;
http_data_cb on_header_value;
http_cb on_headers_complete;
http_data_cb on_body;
http_cb on_message_complete;
/* When on_chunk_header is called, the current chunk length is stored
* in parser->content_length.
*/
http_cb on_chunk_header;
http_cb on_chunk_complete;
};
enum http_parser_url_fields
{ UF_SCHEMA = 0
, UF_HOST = 1
, UF_PORT = 2
, UF_PATH = 3
, UF_QUERY = 4
, UF_FRAGMENT = 5
, UF_USERINFO = 6
, UF_MAX = 7
};
/* Result structure for http_parser_parse_url().
*
* Callers should index into field_data[] with UF_* values iff field_set
* has the relevant (1 << UF_*) bit set. As a courtesy to clients (and
* because we probably have padding left over), we convert any port to
* a uint16_t.
*/
struct http_parser_url {
uint16_t field_set; /* Bitmask of (1 << UF_*) values */
uint16_t port; /* Converted UF_PORT string */
struct {
uint16_t off; /* Offset into buffer in which field starts */
uint16_t len; /* Length of run in buffer */
} field_data[UF_MAX];
};
/* Returns the library version. Bits 16-23 contain the major version number,
* bits 8-15 the minor version number and bits 0-7 the patch level.
* Usage example:
*
* unsigned long version = http_parser_version();
* unsigned major = (version >> 16) & 255;
* unsigned minor = (version >> 8) & 255;
* unsigned patch = version & 255;
* printf("http_parser v%u.%u.%u\n", major, minor, patch);
*/
unsigned long http_parser_version(void);
void http_parser_init(http_parser *parser, enum http_parser_type type);
/* Initialize http_parser_settings members to 0
*/
void http_parser_settings_init(http_parser_settings *settings);
/* Executes the parser. Returns number of parsed bytes. Sets
* `parser->http_errno` on error. */
size_t http_parser_execute(http_parser *parser,
const http_parser_settings *settings,
const char *data,
size_t len);
/* If http_should_keep_alive() in the on_headers_complete or
* on_message_complete callback returns 0, then this should be
* the last message on the connection.
* If you are the server, respond with the "Connection: close" header.
* If you are the client, close the connection.
*/
int http_should_keep_alive(const http_parser *parser);
/* Returns a string version of the HTTP method. */
const char *http_method_str(enum http_method m);
/* Return a string name of the given error */
const char *http_errno_name(enum http_errno err);
/* Return a string description of the given error */
const char *http_errno_description(enum http_errno err);
/* Initialize all http_parser_url members to 0 */
void http_parser_url_init(struct http_parser_url *u);
/* Parse a URL; return nonzero on failure */
int http_parser_parse_url(const char *buf, size_t buflen,
int is_connect,
struct http_parser_url *u);
/* Pause or un-pause the parser; a nonzero value pauses */
void http_parser_pause(http_parser *parser, int paused);
/* Checks if this is the final chunk of the body. */
int http_body_is_final(const http_parser *parser);
#ifdef __cplusplus
}
#endif
#endif
================================================
FILE: src/main.h
================================================
#ifndef MAIN_H
#define MAIN_H
#include <ctype.h>
#include <errno.h>
#include <fcntl.h>
#include <getopt.h>
#include <math.h>
#include <netinet/in.h>
#include <netinet/tcp.h>
#include <stdarg.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <signal.h>
#include <time.h>
#include <unistd.h>
#include <sys/time.h>
#include <sys/uio.h>
#include "ssl.h"
#include "aprintf.h"
#include "stats.h"
#include "units.h"
#include "zmalloc.h"
struct config;
static void *thread_main(void *);
static int connect_socket(thread *, connection *);
static int reconnect_socket(thread *, connection *);
static int record_rate(aeEventLoop *, long long, void *);
static void socket_connected(aeEventLoop *, int, void *, int);
static void socket_writeable(aeEventLoop *, int, void *, int);
static void socket_readable(aeEventLoop *, int, void *, int);
static int response_complete(http_parser *);
static int header_field(http_parser *, const char *, size_t);
static int header_value(http_parser *, const char *, size_t);
static int response_body(http_parser *, const char *, size_t);
static uint64_t time_us();
static int parse_args(struct config *, char **, struct http_parser_url *, char **, int, char **);
static char *copy_url_part(char *, struct http_parser_url *, enum http_parser_url_fields);
static void print_stats_header();
static void print_stats(char *, stats *, char *(*)(long double));
static void print_stats_latency(stats *);
#endif /* MAIN_H */
================================================
FILE: src/net.c
================================================
// Copyright (C) 2013 - Will Glozer. All rights reserved.
#include <errno.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include "net.h"
status sock_connect(connection *c, char *host) {
return OK;
}
status sock_close(connection *c) {
return OK;
}
status sock_read(connection *c, size_t *n) {
ssize_t r = read(c->fd, c->buf, sizeof(c->buf));
*n = (size_t) r;
return r >= 0 ? OK : ERROR;
}
status sock_write(connection *c, char *buf, size_t len, size_t *n) {
ssize_t r;
if ((r = write(c->fd, buf, len)) == -1) {
switch (errno) {
case EAGAIN: return RETRY;
default: return ERROR;
}
}
*n = (size_t) r;
return OK;
}
size_t sock_readable(connection *c) {
int n, rc;
rc = ioctl(c->fd, FIONREAD, &n);
return rc == -1 ? 0 : n;
}
================================================
FILE: src/net.h
================================================
#ifndef NET_H
#define NET_H
#include "config.h"
#include <stdint.h>
#include <openssl/ssl.h>
#include "wrk.h"
typedef enum {
OK,
ERROR,
RETRY
} status;
struct sock {
status ( *connect)(connection *, char *);
status ( *close)(connection *);
status ( *read)(connection *, size_t *);
status ( *write)(connection *, char *, size_t, size_t *);
size_t (*readable)(connection *);
};
status sock_connect(connection *, char *);
status sock_close(connection *);
status sock_read(connection *, size_t *);
status sock_write(connection *, char *, size_t, size_t *);
size_t sock_readable(connection *);
#endif /* NET_H */
================================================
FILE: src/script.c
================================================
// Copyright (C) 2013 - Will Glozer. All rights reserved.
#include <stdlib.h>
#include <string.h>
#include "script.h"
#include "http_parser.h"
#include "zmalloc.h"
typedef struct {
char *name;
int type;
void *value;
} table_field;
static int script_addr_tostring(lua_State *);
static int script_addr_gc(lua_State *);
static int script_stats_call(lua_State *);
static int script_stats_len(lua_State *);
static int script_stats_index(lua_State *);
static int script_thread_index(lua_State *);
static int script_thread_newindex(lua_State *);
static int script_wrk_lookup(lua_State *);
static int script_wrk_connect(lua_State *);
static void set_fields(lua_State *, int, const table_field *);
static void set_field(lua_State *, int, char *, int);
static int push_url_part(lua_State *, char *, struct http_parser_url *, enum http_parser_url_fields);
static const struct luaL_Reg addrlib[] = {
{ "__tostring", script_addr_tostring },
{ "__gc" , script_addr_gc },
{ NULL, NULL }
};
static const struct luaL_Reg statslib[] = {
{ "__call", script_stats_call },
{ "__index", script_stats_index },
{ "__len", script_stats_len },
{ NULL, NULL }
};
static const struct luaL_Reg threadlib[] = {
{ "__index", script_thread_index },
{ "__newindex", script_thread_newindex },
{ NULL, NULL }
};
lua_State *script_create(char *file, char *url, char **headers) {
lua_State *L = luaL_newstate();
luaL_openlibs(L);
(void) luaL_dostring(L, "wrk = require \"wrk\"");
luaL_newmetatable(L, "wrk.addr");
luaL_register(L, NULL, addrlib);
luaL_newmetatable(L, "wrk.stats");
luaL_register(L, NULL, statslib);
luaL_newmetatable(L, "wrk.thread");
luaL_register(L, NULL, threadlib);
struct http_parser_url parts = {};
script_parse_url(url, &parts);
char *path = "/";
if (parts.field_set & (1 << UF_PATH)) {
path = &url[parts.field_data[UF_PATH].off];
}
const table_field fields[] = {
{ "lookup", LUA_TFUNCTION, script_wrk_lookup },
{ "connect", LUA_TFUNCTION, script_wrk_connect },
{ "path", LUA_TSTRING, path },
{ NULL, 0, NULL },
};
lua_getglobal(L, "wrk");
set_field(L, 4, "scheme", push_url_part(L, url, &parts, UF_SCHEMA));
set_field(L, 4, "host", push_url_part(L, url, &parts, UF_HOST));
set_field(L, 4, "port", push_url_part(L, url, &parts, UF_PORT));
set_fields(L, 4, fields);
lua_getfield(L, 4, "headers");
for (char **h = headers; *h; h++) {
char *p = strchr(*h, ':');
if (p && p[1] == ' ') {
lua_pushlstring(L, *h, p - *h);
lua_pushstring(L, p + 2);
lua_settable(L, 5);
}
}
lua_pop(L, 5);
if (file && luaL_dofile(L, file)) {
const char *cause = lua_tostring(L, -1);
fprintf(stderr, "%s: %s\n", file, cause);
}
return L;
}
bool script_resolve(lua_State *L, char *host, char *service) {
lua_getglobal(L, "wrk");
lua_getfield(L, -1, "resolve");
lua_pushstring(L, host);
lua_pushstring(L, service);
lua_call(L, 2, 0);
lua_getfield(L, -1, "addrs");
size_t count = lua_objlen(L, -1);
lua_pop(L, 2);
return count > 0;
}
void script_push_thread(lua_State *L, thread *t) {
thread **ptr = (thread **) lua_newuserdata(L, sizeof(thread **));
*ptr = t;
luaL_getmetatable(L, "wrk.thread");
lua_setmetatable(L, -2);
}
void script_init(lua_State *L, thread *t, int argc, char **argv) {
lua_getglobal(t->L, "wrk");
script_push_thread(t->L, t);
lua_setfield(t->L, -2, "thread");
lua_getglobal(L, "wrk");
lua_getfield(L, -1, "setup");
script_push_thread(L, t);
lua_call(L, 1, 0);
lua_pop(L, 1);
lua_getfield(t->L, -1, "init");
lua_newtable(t->L);
for (int i = 0; i < argc; i++) {
lua_pushstring(t->L, argv[i]);
lua_rawseti(t->L, -2, i);
}
lua_call(t->L, 1, 0);
lua_pop(t->L, 1);
}
uint64_t script_delay(lua_State *L) {
lua_getglobal(L, "delay");
lua_call(L, 0, 1);
uint64_t delay = lua_tonumber(L, -1);
lua_pop(L, 1);
return delay;
}
void script_request(lua_State *L, char **buf, size_t *len) {
int pop = 1;
lua_getglobal(L, "request");
if (!lua_isfunction(L, -1)) {
lua_getglobal(L, "wrk");
lua_getfield(L, -1, "request");
pop += 2;
}
lua_call(L, 0, 1);
const char *str = lua_tolstring(L, -1, len);
*buf = realloc(*buf, *len);
memcpy(*buf, str, *len);
lua_pop(L, pop);
}
void script_response(lua_State *L, int status, buffer *headers, buffer *body) {
lua_getglobal(L, "response");
lua_pushinteger(L, status);
lua_newtable(L);
for (char *c = headers->buffer; c < headers->cursor; ) {
c = buffer_pushlstring(L, c);
c = buffer_pushlstring(L, c);
lua_rawset(L, -3);
}
lua_pushlstring(L, body->buffer, body->cursor - body->buffer);
lua_call(L, 3, 0);
buffer_reset(headers);
buffer_reset(body);
}
bool script_is_function(lua_State *L, char *name) {
lua_getglobal(L, name);
bool is_function = lua_isfunction(L, -1);
lua_pop(L, 1);
return is_function;
}
bool script_is_static(lua_State *L) {
return !script_is_function(L, "request");
}
bool script_want_response(lua_State *L) {
return script_is_function(L, "response");
}
bool script_has_delay(lua_State *L) {
return script_is_function(L, "delay");
}
bool script_has_done(lua_State *L) {
return script_is_function(L, "done");
}
void script_header_done(lua_State *L, luaL_Buffer *buffer) {
luaL_pushresult(buffer);
}
void script_summary(lua_State *L, uint64_t duration, uint64_t requests, uint64_t bytes) {
const table_field fields[] = {
{ "duration", LUA_TNUMBER, &duration },
{ "requests", LUA_TNUMBER, &requests },
{ "bytes", LUA_TNUMBER, &bytes },
{ NULL, 0, NULL },
};
lua_newtable(L);
set_fields(L, 1, fields);
}
void script_errors(lua_State *L, errors *errors) {
uint64_t e[] = {
errors->connect,
errors->read,
errors->write,
errors->status,
errors->timeout
};
const table_field fields[] = {
{ "connect", LUA_TNUMBER, &e[0] },
{ "read", LUA_TNUMBER, &e[1] },
{ "write", LUA_TNUMBER, &e[2] },
{ "status", LUA_TNUMBER, &e[3] },
{ "timeout", LUA_TNUMBER, &e[4] },
{ NULL, 0, NULL },
};
lua_newtable(L);
set_fields(L, 2, fields);
lua_setfield(L, 1, "errors");
}
void script_push_stats(lua_State *L, stats *s) {
stats **ptr = (stats **) lua_newuserdata(L, sizeof(stats **));
*ptr = s;
luaL_getmetatable(L, "wrk.stats");
lua_setmetatable(L, -2);
}
void script_done(lua_State *L, stats *latency, stats *requests) {
lua_getglobal(L, "done");
lua_pushvalue(L, 1);
script_push_stats(L, latency);
script_push_stats(L, requests);
lua_call(L, 3, 0);
lua_pop(L, 1);
}
static int verify_request(http_parser *parser) {
size_t *count = parser->data;
(*count)++;
return 0;
}
size_t script_verify_request(lua_State *L) {
http_parser_settings settings = {
.on_message_complete = verify_request
};
http_parser parser;
char *request = NULL;
size_t len, count = 0;
script_request(L, &request, &len);
http_parser_init(&parser, HTTP_REQUEST);
parser.data = &count;
size_t parsed = http_parser_execute(&parser, &settings, request, len);
if (parsed != len || count == 0) {
enum http_errno err = HTTP_PARSER_ERRNO(&parser);
const char *desc = http_errno_description(err);
const char *msg = err != HPE_OK ? desc : "incomplete request";
int line = 1, column = 1;
for (char *c = request; c < request + parsed; c++) {
column++;
if (*c == '\n') {
column = 1;
line++;
}
}
fprintf(stderr, "%s at %d:%d\n", msg, line, column);
exit(1);
}
return count;
}
static struct addrinfo *checkaddr(lua_State *L) {
struct addrinfo *addr = luaL_checkudata(L, -1, "wrk.addr");
luaL_argcheck(L, addr != NULL, 1, "`addr' expected");
return addr;
}
void script_addr_copy(struct addrinfo *src, struct addrinfo *dst) {
*dst = *src;
dst->ai_addr = zmalloc(src->ai_addrlen);
memcpy(dst->ai_addr, src->ai_addr, src->ai_addrlen);
}
struct addrinfo *script_addr_clone(lua_State *L, struct addrinfo *addr) {
struct addrinfo *udata = lua_newuserdata(L, sizeof(*udata));
luaL_getmetatable(L, "wrk.addr");
lua_setmetatable(L, -2);
script_addr_copy(addr, udata);
return udata;
}
static int script_addr_tostring(lua_State *L) {
struct addrinfo *addr = checkaddr(L);
char host[NI_MAXHOST];
char service[NI_MAXSERV];
int flags = NI_NUMERICHOST | NI_NUMERICSERV;
int rc = getnameinfo(addr->ai_addr, addr->ai_addrlen, host, NI_MAXHOST, service, NI_MAXSERV, flags);
if (rc != 0) {
const char *msg = gai_strerror(rc);
return luaL_error(L, "addr tostring failed %s", msg);
}
lua_pushfstring(L, "%s:%s", host, service);
return 1;
}
static int script_addr_gc(lua_State *L) {
struct addrinfo *addr = checkaddr(L);
zfree(addr->ai_addr);
return 0;
}
static stats *checkstats(lua_State *L) {
stats **s = luaL_checkudata(L, 1, "wrk.stats");
luaL_argcheck(L, s != NULL, 1, "`stats' expected");
return *s;
}
static int script_stats_percentile(lua_State *L) {
stats *s = checkstats(L);
lua_Number p = luaL_checknumber(L, 2);
lua_pushnumber(L, stats_percentile(s, p));
return 1;
}
static int script_stats_call(lua_State *L) {
stats *s = checkstats(L);
uint64_t index = lua_tonumber(L, 2);
uint64_t count;
lua_pushnumber(L, stats_value_at(s, index - 1, &count));
lua_pushnumber(L, count);
return 2;
}
static int script_stats_index(lua_State *L) {
stats *s = checkstats(L);
const char *method = lua_tostring(L, 2);
if (!strcmp("min", method)) lua_pushnumber(L, s->min);
if (!strcmp("max", method)) lua_pushnumber(L, s->max);
if (!strcmp("mean", method)) lua_pushnumber(L, stats_mean(s));
if (!strcmp("stdev", method)) lua_pushnumber(L, stats_stdev(s, stats_mean(s)));
if (!strcmp("percentile", method)) {
lua_pushcfunction(L, script_stats_percentile);
}
return 1;
}
static int script_stats_len(lua_State *L) {
stats *s = checkstats(L);
lua_pushinteger(L, stats_popcount(s));
return 1;
}
static thread *checkthread(lua_State *L) {
thread **t = luaL_checkudata(L, 1, "wrk.thread");
luaL_argcheck(L, t != NULL, 1, "`thread' expected");
return *t;
}
static int script_thread_get(lua_State *L) {
thread *t = checkthread(L);
const char *key = lua_tostring(L, -1);
lua_getglobal(t->L, key);
script_copy_value(t->L, L, -1);
lua_pop(t->L, 1);
return 1;
}
static int script_thread_set(lua_State *L) {
thread *t = checkthread(L);
const char *name = lua_tostring(L, -2);
script_copy_value(L, t->L, -1);
lua_setglobal(t->L, name);
return 0;
}
static int script_thread_stop(lua_State *L) {
thread *t = checkthread(L);
aeStop(t->loop);
return 0;
}
static int script_thread_index(lua_State *L) {
thread *t = checkthread(L);
const char *key = lua_tostring(L, 2);
if (!strcmp("get", key)) lua_pushcfunction(L, script_thread_get);
if (!strcmp("set", key)) lua_pushcfunction(L, script_thread_set);
if (!strcmp("stop", key)) lua_pushcfunction(L, script_thread_stop);
if (!strcmp("addr", key)) script_addr_clone(L, t->addr);
return 1;
}
static int script_thread_newindex(lua_State *L) {
thread *t = checkthread(L);
const char *key = lua_tostring(L, -2);
if (!strcmp("addr", key)) {
struct addrinfo *addr = checkaddr(L);
if (t->addr) zfree(t->addr->ai_addr);
t->addr = zrealloc(t->addr, sizeof(*addr));
script_addr_copy(addr, t->addr);
} else {
luaL_error(L, "cannot set '%s' on thread", luaL_typename(L, -1));
}
return 0;
}
static int script_wrk_lookup(lua_State *L) {
struct addrinfo *addrs;
struct addrinfo hints = {
.ai_family = AF_UNSPEC,
.ai_socktype = SOCK_STREAM
};
int rc, index = 1;
const char *host = lua_tostring(L, -2);
const char *service = lua_tostring(L, -1);
if ((rc = getaddrinfo(host, service, &hints, &addrs)) != 0) {
const char *msg = gai_strerror(rc);
fprintf(stderr, "unable to resolve %s:%s %s\n", host, service, msg);
exit(1);
}
lua_newtable(L);
for (struct addrinfo *addr = addrs; addr != NULL; addr = addr->ai_next) {
script_addr_clone(L, addr);
lua_rawseti(L, -2, index++);
}
freeaddrinfo(addrs);
return 1;
}
static int script_wrk_connect(lua_State *L) {
struct addrinfo *addr = checkaddr(L);
int fd, connected = 0;
if ((fd = socket(addr->ai_family, addr->ai_socktype, addr->ai_protocol)) != -1) {
connected = connect(fd, addr->ai_addr, addr->ai_addrlen) == 0;
close(fd);
}
lua_pushboolean(L, connected);
return 1;
}
void script_copy_value(lua_State *src, lua_State *dst, int index) {
switch (lua_type(src, index)) {
case LUA_TBOOLEAN:
lua_pushboolean(dst, lua_toboolean(src, index));
break;
case LUA_TNIL:
lua_pushnil(dst);
break;
case LUA_TNUMBER:
lua_pushnumber(dst, lua_tonumber(src, index));
break;
case LUA_TSTRING:
lua_pushstring(dst, lua_tostring(src, index));
break;
case LUA_TTABLE:
lua_newtable(dst);
lua_pushnil(src);
while (lua_next(src, index - 1)) {
script_copy_value(src, dst, -2);
script_copy_value(src, dst, -1);
lua_settable(dst, -3);
lua_pop(src, 1);
}
lua_pop(src, 1);
break;
default:
luaL_error(src, "cannot transfer '%s' to thread", luaL_typename(src, index));
}
}
int script_parse_url(char *url, struct http_parser_url *parts) {
if (!http_parser_parse_url(url, strlen(url), 0, parts)) {
if (!(parts->field_set & (1 << UF_SCHEMA))) return 0;
if (!(parts->field_set & (1 << UF_HOST))) return 0;
return 1;
}
return 0;
}
static int push_url_part(lua_State *L, char *url, struct http_parser_url *parts, enum http_parser_url_fields field) {
int type = parts->field_set & (1 << field) ? LUA_TSTRING : LUA_TNIL;
uint16_t off, len;
switch (type) {
case LUA_TSTRING:
off = parts->field_data[field].off;
len = parts->field_data[field].len;
lua_pushlstring(L, &url[off], len);
break;
case LUA_TNIL:
lua_pushnil(L);
}
return type;
}
static void set_field(lua_State *L, int index, char *field, int type) {
(void) type;
lua_setfield(L, index, field);
}
static void set_fields(lua_State *L, int index, const table_field *fields) {
for (int i = 0; fields[i].name; i++) {
table_field f = fields[i];
switch (f.value == NULL ? LUA_TNIL : f.type) {
case LUA_TFUNCTION:
lua_pushcfunction(L, (lua_CFunction) f.value);
break;
case LUA_TNUMBER:
lua_pushinteger(L, *((lua_Integer *) f.value));
break;
case LUA_TSTRING:
lua_pushstring(L, (const char *) f.value);
break;
case LUA_TNIL:
lua_pushnil(L);
break;
}
lua_setfield(L, index, f.name);
}
}
void buffer_append(buffer *b, const char *data, size_t len) {
size_t used = b->cursor - b->buffer;
while (used + len + 1 >= b->length) {
b->length += 1024;
b->buffer = realloc(b->buffer, b->length);
b->cursor = b->buffer + used;
}
memcpy(b->cursor, data, len);
b->cursor += len;
}
void buffer_reset(buffer *b) {
b->cursor = b->buffer;
}
char *buffer_pushlstring(lua_State *L, char *start) {
char *end = strchr(start, 0);
lua_pushlstring(L, start, end - start);
return end + 1;
}
================================================
FILE: src/script.h
================================================
#ifndef SCRIPT_H
#define SCRIPT_H
#include <stdbool.h>
#include <lua.h>
#include <lualib.h>
#include <lauxlib.h>
#include <unistd.h>
#include "stats.h"
#include "wrk.h"
lua_State *script_create(char *, char *, char **);
bool script_resolve(lua_State *, char *, char *);
void script_setup(lua_State *, thread *);
void script_done(lua_State *, stats *, stats *);
void script_init(lua_State *, thread *, int, char **);
uint64_t script_delay(lua_State *);
void script_request(lua_State *, char **, size_t *);
void script_response(lua_State *, int, buffer *, buffer *);
size_t script_verify_request(lua_State *L);
bool script_is_static(lua_State *);
bool script_want_response(lua_State *L);
bool script_has_delay(lua_State *L);
bool script_has_done(lua_State *L);
void script_summary(lua_State *, uint64_t, uint64_t, uint64_t);
void script_errors(lua_State *, errors *);
void script_copy_value(lua_State *, lua_State *, int);
int script_parse_url(char *, struct http_parser_url *);
void buffer_append(buffer *, const char *, size_t);
void buffer_reset(buffer *);
char *buffer_pushlstring(lua_State *, char *);
#endif /* SCRIPT_H */
================================================
FILE: src/ssl.c
================================================
// Copyright (C) 2013 - Will Glozer. All rights reserved.
#include <pthread.h>
#include <openssl/evp.h>
#include <openssl/err.h>
#include <openssl/ssl.h>
#include "ssl.h"
SSL_CTX *ssl_init() {
SSL_CTX *ctx = NULL;
SSL_load_error_strings();
SSL_library_init();
OpenSSL_add_all_algorithms();
if ((ctx = SSL_CTX_new(SSLv23_client_method()))) {
SSL_CTX_set_verify(ctx, SSL_VERIFY_NONE, NULL);
SSL_CTX_set_verify_depth(ctx, 0);
SSL_CTX_set_mode(ctx, SSL_MODE_AUTO_RETRY);
SSL_CTX_set_session_cache_mode(ctx, SSL_SESS_CACHE_CLIENT);
}
return ctx;
}
status ssl_connect(connection *c, char *host) {
int r;
SSL_set_fd(c->ssl, c->fd);
SSL_set_tlsext_host_name(c->ssl, host);
if ((r = SSL_connect(c->ssl)) != 1) {
switch (SSL_get_error(c->ssl, r)) {
case SSL_ERROR_WANT_READ: return RETRY;
case SSL_ERROR_WANT_WRITE: return RETRY;
default: return ERROR;
}
}
return OK;
}
status ssl_close(connection *c) {
SSL_shutdown(c->ssl);
SSL_clear(c->ssl);
return OK;
}
status ssl_read(connection *c, size_t *n) {
int r;
if ((r = SSL_read(c->ssl, c->buf, sizeof(c->buf))) <= 0) {
switch (SSL_get_error(c->ssl, r)) {
case SSL_ERROR_WANT_READ: return RETRY;
case SSL_ERROR_WANT_WRITE: return RETRY;
default: return ERROR;
}
}
*n = (size_t) r;
return OK;
}
status ssl_write(connection *c, char *buf, size_t len, size_t *n) {
int r;
if ((r = SSL_write(c->ssl, buf, len)) <= 0) {
switch (SSL_get_error(c->ssl, r)) {
case SSL_ERROR_WANT_READ: return RETRY;
case SSL_ERROR_WANT_WRITE: return RETRY;
default: return ERROR;
}
}
*n = (size_t) r;
return OK;
}
size_t ssl_readable(connection *c) {
return SSL_pending(c->ssl);
}
================================================
FILE: src/ssl.h
================================================
#ifndef SSL_H
#define SSL_H
#include "net.h"
SSL_CTX *ssl_init();
status ssl_connect(connection *, char *);
status ssl_close(connection *);
status ssl_read(connection *, size_t *);
status ssl_write(connection *, char *, size_t, size_t *);
size_t ssl_readable(connection *);
#endif /* SSL_H */
================================================
FILE: src/stats.c
================================================
// Copyright (C) 2012 - Will Glozer. All rights reserved.
#include <inttypes.h>
#include <stdlib.h>
#include <math.h>
#include "stats.h"
#include "zmalloc.h"
stats *stats_alloc(uint64_t max) {
uint64_t limit = max + 1;
stats *s = zcalloc(sizeof(stats) + sizeof(uint64_t) * limit);
s->limit = limit;
s->min = UINT64_MAX;
return s;
}
void stats_free(stats *stats) {
zfree(stats);
}
int stats_record(stats *stats, uint64_t n) {
if (n >= stats->limit) return 0;
__sync_fetch_and_add(&stats->data[n], 1);
__sync_fetch_and_add(&stats->count, 1);
uint64_t min = stats->min;
uint64_t max = stats->max;
while (n < min) min = __sync_val_compare_and_swap(&stats->min, min, n);
while (n > max) max = __sync_val_compare_and_swap(&stats->max, max, n);
return 1;
}
void stats_correct(stats *stats, int64_t expected) {
for (uint64_t n = expected * 2; n <= stats->max; n++) {
uint64_t count = stats->data[n];
int64_t m = (int64_t) n - expected;
while (count && m > expected) {
stats->data[m] += count;
stats->count += count;
m -= expected;
}
}
}
long double stats_mean(stats *stats) {
if (stats->count == 0) return 0.0;
uint64_t sum = 0;
for (uint64_t i = stats->min; i <= stats->max; i++) {
sum += stats->data[i] * i;
}
return sum / (long double) stats->count;
}
long double stats_stdev(stats *stats, long double mean) {
long double sum = 0.0;
if (stats->count < 2) return 0.0;
for (uint64_t i = stats->min; i <= stats->max; i++) {
if (sta
gitextract_fdoud_vu/
├── .github/
│ └── workflows/
│ └── build.yml
├── .gitignore
├── CHANGES
├── INSTALL
├── LICENSE
├── Makefile
├── NOTICE
├── README.md
├── SCRIPTING
├── scripts/
│ ├── addr.lua
│ ├── auth.lua
│ ├── counter.lua
│ ├── delay.lua
│ ├── pipeline.lua
│ ├── post.lua
│ ├── report.lua
│ ├── setup.lua
│ └── stop.lua
└── src/
├── ae.c
├── ae.h
├── ae_epoll.c
├── ae_evport.c
├── ae_kqueue.c
├── ae_select.c
├── aprintf.c
├── aprintf.h
├── atomicvar.h
├── config.h
├── http_parser.c
├── http_parser.h
├── main.h
├── net.c
├── net.h
├── script.c
├── script.h
├── ssl.c
├── ssl.h
├── stats.c
├── stats.h
├── units.c
├── units.h
├── wrk.c
├── wrk.h
├── wrk.lua
├── zmalloc.c
└── zmalloc.h
SYMBOL INDEX (229 symbols across 20 files)
FILE: src/ae.c
function aeEventLoop (line 63) | aeEventLoop *aeCreateEventLoop(int setsize) {
function aeGetSetSize (line 95) | int aeGetSetSize(aeEventLoop *eventLoop) {
function aeResizeSetSize (line 106) | int aeResizeSetSize(aeEventLoop *eventLoop, int setsize) {
function aeDeleteEventLoop (line 124) | void aeDeleteEventLoop(aeEventLoop *eventLoop) {
function aeStop (line 131) | void aeStop(aeEventLoop *eventLoop) {
function aeCreateFileEvent (line 135) | int aeCreateFileEvent(aeEventLoop *eventLoop, int fd, int mask,
function aeDeleteFileEvent (line 155) | void aeDeleteFileEvent(aeEventLoop *eventLoop, int fd, int mask)
function aeGetFileEvents (line 173) | int aeGetFileEvents(aeEventLoop *eventLoop, int fd) {
function aeGetTime (line 180) | static void aeGetTime(long *seconds, long *milliseconds)
function aeAddMillisecondsToNow (line 189) | static void aeAddMillisecondsToNow(long long milliseconds, long *sec, lo...
function aeCreateTimeEvent (line 203) | long long aeCreateTimeEvent(aeEventLoop *eventLoop, long long milliseconds,
function aeDeleteTimeEvent (line 222) | int aeDeleteTimeEvent(aeEventLoop *eventLoop, long long id)
function aeTimeEvent (line 246) | static aeTimeEvent *aeSearchNearestTimer(aeEventLoop *eventLoop)
function processTimeEvents (line 262) | static int processTimeEvents(aeEventLoop *eventLoop) {
function aeProcessEvents (line 349) | int aeProcessEvents(aeEventLoop *eventLoop, int flags)
function aeWait (line 430) | int aeWait(int fd, int mask, long long milliseconds) {
function aeMain (line 450) | void aeMain(aeEventLoop *eventLoop) {
function aeSetBeforeSleepProc (line 463) | void aeSetBeforeSleepProc(aeEventLoop *eventLoop, aeBeforeSleepProc *bef...
FILE: src/ae.h
type aeEventLoop (line 56) | struct aeEventLoop
type aeEventLoop (line 59) | struct aeEventLoop
type aeEventLoop (line 60) | struct aeEventLoop
type aeEventLoop (line 61) | struct aeEventLoop
type aeEventLoop (line 62) | struct aeEventLoop
type aeFileEvent (line 65) | typedef struct aeFileEvent {
type aeTimeEvent (line 73) | typedef struct aeTimeEvent {
type aeFiredEvent (line 84) | typedef struct aeFiredEvent {
type aeEventLoop (line 90) | typedef struct aeEventLoop {
FILE: src/ae_epoll.c
type aeApiState (line 34) | typedef struct aeApiState {
function aeApiCreate (line 39) | static int aeApiCreate(aeEventLoop *eventLoop) {
function aeApiResize (line 58) | static int aeApiResize(aeEventLoop *eventLoop, int setsize) {
function aeApiFree (line 65) | static void aeApiFree(aeEventLoop *eventLoop) {
function aeApiAddEvent (line 73) | static int aeApiAddEvent(aeEventLoop *eventLoop, int fd, int mask) {
function aeApiDelEvent (line 90) | static void aeApiDelEvent(aeEventLoop *eventLoop, int fd, int delmask) {
function aeApiPoll (line 108) | static int aeApiPoll(aeEventLoop *eventLoop, struct timeval *tvp) {
FILE: src/ae_evport.c
type aeApiState (line 68) | typedef struct aeApiState {
function aeApiCreate (line 75) | static int aeApiCreate(aeEventLoop *eventLoop) {
function aeApiResize (line 97) | static int aeApiResize(aeEventLoop *eventLoop, int setsize) {
function aeApiFree (line 102) | static void aeApiFree(aeEventLoop *eventLoop) {
function aeApiLookupPending (line 109) | static int aeApiLookupPending(aeApiState *state, int fd) {
function aeApiAssociate (line 123) | static int aeApiAssociate(const char *where, int portfd, int fd, int mas...
function aeApiAddEvent (line 152) | static int aeApiAddEvent(aeEventLoop *eventLoop, int fd, int mask) {
function aeApiDelEvent (line 183) | static void aeApiDelEvent(aeEventLoop *eventLoop, int fd, int mask) {
function aeApiPoll (line 243) | static int aeApiPoll(aeEventLoop *eventLoop, struct timeval *tvp) {
FILE: src/ae_kqueue.c
type aeApiState (line 36) | typedef struct aeApiState {
function aeApiCreate (line 41) | static int aeApiCreate(aeEventLoop *eventLoop) {
function aeApiResize (line 60) | static int aeApiResize(aeEventLoop *eventLoop, int setsize) {
function aeApiFree (line 67) | static void aeApiFree(aeEventLoop *eventLoop) {
function aeApiAddEvent (line 75) | static int aeApiAddEvent(aeEventLoop *eventLoop, int fd, int mask) {
function aeApiDelEvent (line 90) | static void aeApiDelEvent(aeEventLoop *eventLoop, int fd, int mask) {
function aeApiPoll (line 104) | static int aeApiPoll(aeEventLoop *eventLoop, struct timeval *tvp) {
FILE: src/ae_select.c
type aeApiState (line 35) | typedef struct aeApiState {
function aeApiCreate (line 42) | static int aeApiCreate(aeEventLoop *eventLoop) {
function aeApiResize (line 52) | static int aeApiResize(aeEventLoop *eventLoop, int setsize) {
function aeApiFree (line 58) | static void aeApiFree(aeEventLoop *eventLoop) {
function aeApiAddEvent (line 62) | static int aeApiAddEvent(aeEventLoop *eventLoop, int fd, int mask) {
function aeApiDelEvent (line 70) | static void aeApiDelEvent(aeEventLoop *eventLoop, int fd, int mask) {
function aeApiPoll (line 77) | static int aeApiPoll(aeEventLoop *eventLoop, struct timeval *tvp) {
FILE: src/http_parser.c
type state (line 276) | enum state
type header_states (line 360) | enum header_states
type http_host_state (line 390) | enum http_host_state
function parse_url_char (line 483) | static enum state
function http_parser_execute (line 631) | size_t http_parser_execute (http_parser *parser,
function http_message_needs_eof (line 2033) | int
function http_should_keep_alive (line 2056) | int
type http_method (line 2076) | enum http_method
function http_parser_init (line 2082) | void
function http_parser_settings_init (line 2093) | void
type http_errno (line 2100) | enum http_errno
type http_errno (line 2106) | enum http_errno
function http_parse_host_char (line 2111) | static enum http_host_state
function http_parse_host (line 2193) | static int
function http_parser_url_init (line 2275) | void
function http_parser_parse_url (line 2280) | int
function http_parser_pause (line 2384) | void
function http_body_is_final (line 2398) | int
function http_parser_version (line 2403) | unsigned long
FILE: src/http_parser.h
type __int8 (line 36) | typedef __int8 int8_t;
type __int16 (line 38) | typedef __int16 int16_t;
type __int32 (line 40) | typedef __int32 int32_t;
type __int64 (line 42) | typedef __int64 int64_t;
type http_parser (line 66) | typedef struct http_parser http_parser;
type http_parser_settings (line 67) | typedef struct http_parser_settings http_parser_settings;
type http_status (line 154) | enum http_status
type http_method (line 205) | enum http_method
type http_parser_type (line 213) | enum http_parser_type { HTTP_REQUEST, HTTP_RESPONSE, HTTP_BOTH }
type flags (line 217) | enum flags
type http_errno (line 281) | enum http_errno {
type http_parser (line 291) | struct http_parser {
type http_parser_settings (line 322) | struct http_parser_settings {
type http_parser_url_fields (line 339) | enum http_parser_url_fields
type http_parser_url (line 358) | struct http_parser_url {
type http_parser_type (line 381) | enum http_parser_type
type http_method (line 406) | enum http_method
type http_errno (line 409) | enum http_errno
type http_errno (line 412) | enum http_errno
type http_parser_url (line 415) | struct http_parser_url
type http_parser_url (line 420) | struct http_parser_url
FILE: src/main.h
type config (line 28) | struct config
type config (line 47) | struct config
type http_parser_url (line 47) | struct http_parser_url
type http_parser_url (line 48) | struct http_parser_url
type http_parser_url_fields (line 48) | enum http_parser_url_fields
FILE: src/net.c
function status (line 9) | status sock_connect(connection *c, char *host) {
function status (line 13) | status sock_close(connection *c) {
function status (line 17) | status sock_read(connection *c, size_t *n) {
function status (line 23) | status sock_write(connection *c, char *buf, size_t len, size_t *n) {
function sock_readable (line 35) | size_t sock_readable(connection *c) {
FILE: src/net.h
type status (line 9) | typedef enum {
type sock (line 15) | struct sock {
FILE: src/script.c
type table_field (line 9) | typedef struct {
type http_parser_url (line 27) | struct http_parser_url
type http_parser_url_fields (line 27) | enum http_parser_url_fields
type luaL_Reg (line 29) | struct luaL_Reg
type luaL_Reg (line 35) | struct luaL_Reg
type luaL_Reg (line 42) | struct luaL_Reg
function lua_State (line 48) | lua_State *script_create(char *file, char *url, char **headers) {
function script_resolve (line 101) | bool script_resolve(lua_State *L, char *host, char *service) {
function script_push_thread (line 115) | void script_push_thread(lua_State *L, thread *t) {
function script_init (line 122) | void script_init(lua_State *L, thread *t, int argc, char **argv) {
function script_delay (line 144) | uint64_t script_delay(lua_State *L) {
function script_request (line 152) | void script_request(lua_State *L, char **buf, size_t *len) {
function script_response (line 167) | void script_response(lua_State *L, int status, buffer *headers, buffer *...
function script_is_function (line 185) | bool script_is_function(lua_State *L, char *name) {
function script_is_static (line 192) | bool script_is_static(lua_State *L) {
function script_want_response (line 196) | bool script_want_response(lua_State *L) {
function script_has_delay (line 200) | bool script_has_delay(lua_State *L) {
function script_has_done (line 204) | bool script_has_done(lua_State *L) {
function script_header_done (line 208) | void script_header_done(lua_State *L, luaL_Buffer *buffer) {
function script_summary (line 212) | void script_summary(lua_State *L, uint64_t duration, uint64_t requests, ...
function script_errors (line 223) | void script_errors(lua_State *L, errors *errors) {
function script_push_stats (line 244) | void script_push_stats(lua_State *L, stats *s) {
function script_done (line 251) | void script_done(lua_State *L, stats *latency, stats *requests) {
function verify_request (line 262) | static int verify_request(http_parser *parser) {
function script_verify_request (line 268) | size_t script_verify_request(lua_State *L) {
type addrinfo (line 303) | struct addrinfo
type addrinfo (line 304) | struct addrinfo
function script_addr_copy (line 309) | void script_addr_copy(struct addrinfo *src, struct addrinfo *dst) {
type addrinfo (line 315) | struct addrinfo
type addrinfo (line 315) | struct addrinfo
type addrinfo (line 316) | struct addrinfo
function script_addr_tostring (line 323) | static int script_addr_tostring(lua_State *L) {
function script_addr_gc (line 339) | static int script_addr_gc(lua_State *L) {
function stats (line 345) | static stats *checkstats(lua_State *L) {
function script_stats_percentile (line 351) | static int script_stats_percentile(lua_State *L) {
function script_stats_call (line 358) | static int script_stats_call(lua_State *L) {
function script_stats_index (line 367) | static int script_stats_index(lua_State *L) {
function script_stats_len (line 380) | static int script_stats_len(lua_State *L) {
function thread (line 386) | static thread *checkthread(lua_State *L) {
function script_thread_get (line 392) | static int script_thread_get(lua_State *L) {
function script_thread_set (line 401) | static int script_thread_set(lua_State *L) {
function script_thread_stop (line 409) | static int script_thread_stop(lua_State *L) {
function script_thread_index (line 415) | static int script_thread_index(lua_State *L) {
function script_thread_newindex (line 425) | static int script_thread_newindex(lua_State *L) {
function script_wrk_lookup (line 439) | static int script_wrk_lookup(lua_State *L) {
function script_wrk_connect (line 466) | static int script_wrk_connect(lua_State *L) {
function script_copy_value (line 477) | void script_copy_value(lua_State *src, lua_State *dst, int index) {
function script_parse_url (line 507) | int script_parse_url(char *url, struct http_parser_url *parts) {
function push_url_part (line 516) | static int push_url_part(lua_State *L, char *url, struct http_parser_url...
function set_field (line 531) | static void set_field(lua_State *L, int index, char *field, int type) {
function set_fields (line 536) | static void set_fields(lua_State *L, int index, const table_field *field...
function buffer_append (line 557) | void buffer_append(buffer *b, const char *data, size_t len) {
function buffer_reset (line 568) | void buffer_reset(buffer *b) {
FILE: src/script.h
type http_parser_url (line 32) | struct http_parser_url
FILE: src/ssl.c
function SSL_CTX (line 11) | SSL_CTX *ssl_init() {
function status (line 28) | status ssl_connect(connection *c, char *host) {
function status (line 42) | status ssl_close(connection *c) {
function status (line 48) | status ssl_read(connection *c, size_t *n) {
function status (line 61) | status ssl_write(connection *c, char *buf, size_t len, size_t *n) {
function ssl_readable (line 74) | size_t ssl_readable(connection *c) {
FILE: src/stats.c
function stats (line 10) | stats *stats_alloc(uint64_t max) {
function stats_free (line 18) | void stats_free(stats *stats) {
function stats_record (line 22) | int stats_record(stats *stats, uint64_t n) {
function stats_correct (line 33) | void stats_correct(stats *stats, int64_t expected) {
function stats_mean (line 45) | long double stats_mean(stats *stats) {
function stats_stdev (line 55) | long double stats_stdev(stats *stats, long double mean) {
function stats_within_stdev (line 66) | long double stats_within_stdev(stats *stats, long double mean, long doub...
function stats_percentile (line 80) | uint64_t stats_percentile(stats *stats, long double p) {
function stats_popcount (line 90) | uint64_t stats_popcount(stats *stats) {
function stats_value_at (line 98) | uint64_t stats_value_at(stats *stats, uint64_t index, uint64_t *count) {
FILE: src/stats.h
type errors (line 10) | typedef struct {
type stats (line 18) | typedef struct {
FILE: src/units.c
type units (line 11) | typedef struct {
function scan_units (line 58) | static int scan_units(char *s, uint64_t *n, units *m) {
function scan_metric (line 98) | int scan_metric(char *s, uint64_t *n) {
function scan_time (line 102) | int scan_time(char *s, uint64_t *n) {
FILE: src/wrk.c
type config (line 7) | struct config {
type sock (line 26) | struct sock
type http_parser_settings (line 34) | struct http_parser_settings
function handler (line 40) | static void handler(int sig) {
function usage (line 44) | static void usage() {
function main (line 61) | int main(int argc, char **argv) {
function connect_socket (line 236) | static int connect_socket(thread *thread, connection *c) {
function reconnect_socket (line 266) | static int reconnect_socket(thread *thread, connection *c) {
function record_rate (line 273) | static int record_rate(aeEventLoop *loop, long long id, void *data) {
function delay_request (line 291) | static int delay_request(aeEventLoop *loop, long long id, void *data) {
function header_field (line 298) | static int header_field(http_parser *parser, const char *at, size_t len) {
function header_value (line 308) | static int header_value(http_parser *parser, const char *at, size_t len) {
function response_body (line 318) | static int response_body(http_parser *parser, const char *at, size_t len) {
function response_complete (line 324) | static int response_complete(http_parser *parser) {
function socket_connected (line 362) | static void socket_connected(aeEventLoop *loop, int fd, void *data, int ...
function socket_writeable (line 384) | static void socket_writeable(aeEventLoop *loop, int fd, void *data, int ...
function socket_readable (line 426) | static void socket_readable(aeEventLoop *loop, int fd, void *data, int m...
function time_us (line 450) | static uint64_t time_us() {
type http_parser_url (line 456) | struct http_parser_url
type http_parser_url_fields (line 456) | enum http_parser_url_fields
type option (line 469) | struct option
function parse_args (line 482) | static int parse_args(struct config *cfg, char **url, struct http_parser...
function print_stats_header (line 546) | static void print_stats_header() {
function print_units (line 550) | static void print_units(long double n, char *(*fmt)(long double), int wi...
function print_stats (line 563) | static void print_stats(char *name, stats *stats, char *(*fmt)(long doub...
function print_stats_latency (line 575) | static void print_stats_latency(stats *stats) {
FILE: src/wrk.h
type thread (line 27) | typedef struct {
type buffer (line 41) | typedef struct {
type connection (line 47) | typedef struct connection {
FILE: src/zmalloc.c
function zlibc_free (line 38) | void zlibc_free(void *ptr) {
function zmalloc_default_oom (line 88) | static void zmalloc_default_oom(size_t size) {
function zfree_no_tcache (line 122) | void zfree_no_tcache(void *ptr) {
function zmalloc_size (line 176) | size_t zmalloc_size(void *ptr) {
function zfree (line 186) | void zfree(void *ptr) {
function zmalloc_used_memory (line 212) | size_t zmalloc_used_memory(void) {
function zmalloc_set_oom_handler (line 218) | void zmalloc_set_oom_handler(void (*oom_handler)(size_t)) {
function zmalloc_get_rss (line 238) | size_t zmalloc_get_rss(void) {
function zmalloc_get_rss (line 278) | size_t zmalloc_get_rss(void) {
function zmalloc_get_rss (line 290) | size_t zmalloc_get_rss(void) {
function zmalloc_get_fragmentation_ratio (line 301) | float zmalloc_get_fragmentation_ratio(size_t rss) {
function zmalloc_get_smap_bytes_by_field (line 316) | size_t zmalloc_get_smap_bytes_by_field(char *field, long pid) {
function zmalloc_get_smap_bytes_by_field (line 344) | size_t zmalloc_get_smap_bytes_by_field(char *field, long pid) {
function zmalloc_get_private_dirty (line 351) | size_t zmalloc_get_private_dirty(long pid) {
function zmalloc_get_memory_size (line 368) | size_t zmalloc_get_memory_size(void) {
Condensed preview — 46 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (247K chars).
[
{
"path": ".github/workflows/build.yml",
"chars": 273,
"preview": "name: build\n\non:\n push:\n\njobs:\n build:\n runs-on: ${{ matrix.builder }}\n strategy:\n matrix:\n builder:"
},
{
"path": ".gitignore",
"chars": 9,
"preview": "obj/\nwrk\n"
},
{
"path": "CHANGES",
"chars": 945,
"preview": "master\n\n * Require OpenSSL 1.1.0+\n\nwrk 4.0.2\n\n * Send hostname using TLS SNI.\n * Add optional WITH_OPENSSL and WITH_LUAJ"
},
{
"path": "INSTALL",
"chars": 916,
"preview": "Overview\n\n wrk should build on most UNIX-like operating systems and\n architectures that have GNU make and are supporte"
},
{
"path": "LICENSE",
"chars": 10489,
"preview": "\n Modified Apache 2.0 License\n Version 2.0.1, February 2015\n\n TERMS A"
},
{
"path": "Makefile",
"chars": 2674,
"preview": "CFLAGS += -std=c99 -Wall -O2 -D_REENTRANT\nLIBS := -lm -lssl -lcrypto -lpthread\n\nTARGET := $(shell uname -s | tr '[A"
},
{
"path": "NOTICE",
"chars": 5420,
"preview": "=========================================================================\n== NOTICE file corresponding to section 4(d) "
},
{
"path": "README.md",
"chars": 3497,
"preview": "# wrk - a HTTP benchmarking tool\n\n wrk is a modern HTTP benchmarking tool capable of generating significant\n load when"
},
{
"path": "SCRIPTING",
"chars": 4158,
"preview": "Overview\n\n wrk supports executing a LuaJIT script during three distinct phases: setup,\n running, and done. Each wrk th"
},
{
"path": "scripts/addr.lua",
"chars": 504,
"preview": "-- example script that demonstrates use of setup() to pass\n-- a random server address to each thread\n\nlocal addrs = nil\n"
},
{
"path": "scripts/auth.lua",
"chars": 416,
"preview": "-- example script that demonstrates response handling and\n-- retrieving an authentication token to set on all future\n-- "
},
{
"path": "scripts/counter.lua",
"chars": 448,
"preview": "-- example dynamic request script which demonstrates changing\n-- the request path and a header for each request\n--------"
},
{
"path": "scripts/delay.lua",
"chars": 141,
"preview": "-- example script that demonstrates adding a random\n-- 10-50ms delay before each request\n\nfunction delay()\n return mat"
},
{
"path": "scripts/pipeline.lua",
"chars": 262,
"preview": "-- example script demonstrating HTTP pipelining\n\ninit = function(args)\n local r = {}\n r[1] = wrk.format(nil, \"/?foo\""
},
{
"path": "scripts/post.lua",
"chars": 220,
"preview": "-- example HTTP POST script which demonstrates setting the\n-- HTTP method, body, and adding a header\n\nwrk.method = \"POST"
},
{
"path": "scripts/report.lua",
"chars": 347,
"preview": "-- example reporting script which demonstrates a custom\n-- done() function that prints latency percentiles as CSV\n\ndone "
},
{
"path": "scripts/setup.lua",
"chars": 864,
"preview": "-- example script that demonstrates use of setup() to pass\n-- data to and from the threads\n\nlocal counter = 1\nlocal thre"
},
{
"path": "scripts/stop.lua",
"chars": 183,
"preview": "-- example script that demonstrates use of thread:stop()\n\nlocal counter = 1\n\nfunction response()\n if counter == 100 th"
},
{
"path": "src/ae.c",
"chars": 15529,
"preview": "/* A simple event-driven programming library. Originally I wrote this code\n * for the Jim's event-loop (Jim is a Tcl int"
},
{
"path": "src/ae.h",
"chars": 4681,
"preview": "/* A simple event-driven programming library. Originally I wrote this code\n * for the Jim's event-loop (Jim is a Tcl int"
},
{
"path": "src/ae_epoll.c",
"chars": 4846,
"preview": "/* Linux epoll(2) based ae.c module\n *\n * Copyright (c) 2009-2012, Salvatore Sanfilippo <antirez at gmail dot com>\n * Al"
},
{
"path": "src/ae_evport.c",
"chars": 10939,
"preview": "/* ae.c module for illumos event ports.\n *\n * Copyright (c) 2012, Joyent, Inc. All rights reserved.\n *\n * Redistribution"
},
{
"path": "src/ae_kqueue.c",
"chars": 4567,
"preview": "/* Kqueue(2)-based ae.c module\n *\n * Copyright (C) 2009 Harish Mallipeddi - harish.mallipeddi@gmail.com\n * All rights re"
},
{
"path": "src/ae_select.c",
"chars": 3828,
"preview": "/* Select()-based ae.c module.\n *\n * Copyright (c) 2009-2012, Salvatore Sanfilippo <antirez at gmail dot com>\n * All rig"
},
{
"path": "src/aprintf.c",
"chars": 536,
"preview": "// Copyright (C) 2012 - Will Glozer. All rights reserved.\n\n#include <stdarg.h>\n#include <stdlib.h>\n#include <stdio.h>\n#"
},
{
"path": "src/aprintf.h",
"chars": 104,
"preview": "#ifndef APRINTF_H\n#define APRINTF_H\n\nchar *aprintf(char **, const char *, ...);\n\n#endif /* APRINTF_H */\n"
},
{
"path": "src/atomicvar.h",
"chars": 5367,
"preview": "/* This file implements atomic counters using __atomic or __sync macros if\n * available, otherwise synchronizing differe"
},
{
"path": "src/config.h",
"chars": 321,
"preview": "#ifndef CONFIG_H\n#define CONFIG_H\n\n#if defined(__FreeBSD__) || defined(__APPLE__)\n#define HAVE_KQUEUE\n#elif defined(__li"
},
{
"path": "src/http_parser.c",
"chars": 69700,
"preview": "/* Copyright Joyent, Inc. and other Node contributors.\n *\n * Permission is hereby granted, free of charge, to any person"
},
{
"path": "src/http_parser.h",
"chars": 18682,
"preview": "/* Copyright Joyent, Inc. and other Node contributors. All rights reserved.\n *\n * Permission is hereby granted, free of "
},
{
"path": "src/main.h",
"chars": 1492,
"preview": "#ifndef MAIN_H\n#define MAIN_H\n\n#include <ctype.h>\n#include <errno.h>\n#include <fcntl.h>\n#include <getopt.h>\n#include <ma"
},
{
"path": "src/net.c",
"chars": 824,
"preview": "// Copyright (C) 2013 - Will Glozer. All rights reserved.\n\n#include <errno.h>\n#include <unistd.h>\n#include <sys/ioctl.h"
},
{
"path": "src/net.h",
"chars": 652,
"preview": "#ifndef NET_H\n#define NET_H\n\n#include \"config.h\"\n#include <stdint.h>\n#include <openssl/ssl.h>\n#include \"wrk.h\"\n\ntypedef "
},
{
"path": "src/script.c",
"chars": 16607,
"preview": "// Copyright (C) 2013 - Will Glozer. All rights reserved.\n\n#include <stdlib.h>\n#include <string.h>\n#include \"script.h\"\n"
},
{
"path": "src/script.h",
"chars": 1135,
"preview": "#ifndef SCRIPT_H\n#define SCRIPT_H\n\n#include <stdbool.h>\n#include <lua.h>\n#include <lualib.h>\n#include <lauxlib.h>\n#inclu"
},
{
"path": "src/ssl.c",
"chars": 1960,
"preview": "// Copyright (C) 2013 - Will Glozer. All rights reserved.\n\n#include <pthread.h>\n\n#include <openssl/evp.h>\n#include <ope"
},
{
"path": "src/ssl.h",
"chars": 297,
"preview": "#ifndef SSL_H\n#define SSL_H\n\n#include \"net.h\"\n\nSSL_CTX *ssl_init();\n\nstatus ssl_connect(connection *, char *);\nstatus ss"
},
{
"path": "src/stats.c",
"chars": 2931,
"preview": "// Copyright (C) 2012 - Will Glozer. All rights reserved.\n\n#include <inttypes.h>\n#include <stdlib.h>\n#include <math.h>\n"
},
{
"path": "src/stats.h",
"chars": 878,
"preview": "#ifndef STATS_H\n#define STATS_H\n\n#include <stdbool.h>\n#include <stdint.h>\n\n#define MAX(X, Y) ((X) > (Y) ? (X) : (Y))\n#de"
},
{
"path": "src/units.c",
"chars": 2150,
"preview": "// Copyright (C) 2012 - Will Glozer. All rights reserved.\n\n#include <stdlib.h>\n#include <stdio.h>\n#include <strings.h>\n"
},
{
"path": "src/units.h",
"chars": 265,
"preview": "#ifndef UNITS_H\n#define UNITS_H\n\nchar *format_binary(long double);\nchar *format_metric(long double);\nchar *format_time_u"
},
{
"path": "src/wrk.c",
"chars": 17037,
"preview": "// Copyright (C) 2012 - Will Glozer. All rights reserved.\n\n#include \"wrk.h\"\n#include \"script.h\"\n#include \"main.h\"\n\nstat"
},
{
"path": "src/wrk.h",
"chars": 1159,
"preview": "#ifndef WRK_H\n#define WRK_H\n\n#include \"config.h\"\n#include <pthread.h>\n#include <inttypes.h>\n#include <sys/types.h>\n#incl"
},
{
"path": "src/wrk.lua",
"chars": 1571,
"preview": "local wrk = {\n scheme = \"http\",\n host = \"localhost\",\n port = nil,\n method = \"GET\",\n path = \"/\",\n "
},
{
"path": "src/zmalloc.c",
"chars": 12629,
"preview": "/* zmalloc - total amount of allocated memory aware version of malloc()\n *\n * Copyright (c) 2009-2010, Salvatore Sanfili"
},
{
"path": "src/zmalloc.h",
"chars": 3779,
"preview": "/* zmalloc - total amount of allocated memory aware version of malloc()\n *\n * Copyright (c) 2009-2010, Salvatore Sanfili"
}
]
About this extraction
This page contains the full source code of the wg/wrk GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 46 files (230.7 KB), approximately 61.1k tokens, and a symbol index with 229 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.