Showing preview only (314K chars total). Download the full file or copy to clipboard to get everything.
Repository: xiaokai-wang/nginx-stream-upsync-module
Branch: master
Commit: d403a5b644c6
Files: 18
Total size: 302.8 KB
Directory structure:
gitextract_yli_f_d8/
├── README.md
├── config
├── src/
│ ├── ngx_stream_http_parser.c
│ ├── ngx_stream_http_parser.h
│ ├── ngx_stream_json.c
│ ├── ngx_stream_json.h
│ ├── ngx_stream_upsync_module.c
│ └── ngx_stream_upsync_module.h
└── test/
├── README
├── conf-server.sh
├── t/
│ ├── lib/
│ │ ├── Test/
│ │ │ ├── Nginx/
│ │ │ │ ├── IMAP.pm
│ │ │ │ ├── POP3.pm
│ │ │ │ └── SMTP.pm
│ │ │ └── Nginx.pm
│ │ └── Time/
│ │ ├── Parse.pm
│ │ └── Zone.pm
│ └── upsync.t
└── test.sh
================================================
FILE CONTENTS
================================================
================================================
FILE: README.md
================================================
Name
====
nginx-stream-upsync-module - Nginx C module, sync upstreams from consul or others, dynamically modify backend-servers attribute(weight, max_fails,...), needn't reload nginx.
It may not always be convenient to modify configuration files and restart NGINX. For example, if you are experiencing large amounts of traffic and high load, restarting NGINX and reloading the configuration at that point further increases load on the system and can temporarily degrade performance.
The module can be more smoothly expansion and constriction, and will not influence the performance.
Another module, [nginx-upsync-module](https://github.com/weibocom/nginx-upsync-module) supports nginx http module(HTTP protocol), please be noticed.
If you want to use [nginx-upsync-module](https://github.com/weibocom/nginx-upsync-module) and [nginx-stream-upsync-module](https://github.com/xiaokai-wang/nginx-stream-upsync-module) both, please refer to [nginx-upsync](https://github.com/CallMeFoxie/nginx-upsync).
Table of Contents
=================
* [Name](#name)
* [Status](#status)
* [Synopsis](#synopsis)
* [Description](#description)
* [Directives](#directives)
* [upsync](#upsync)
* [upsync_interval](#upsync_interval)
* [upsync_timeout](#upsync_timeout)
* [upsync_type](#upsync_type)
* [strong_dependency](#strong_dependency)
* [upsync_dump_path](#upsync_dump_path)
* [upsync_lb](#upsync_lb)
* [upstream_show](#upstream_show)
* [Consul_interface](#consul_interface)
* [Etcd_interface](#etcd_interface)
* [TODO](#todo)
* [Compatibility](#compatibility)
* [Installation](#installation)
* [Code style](#code-style)
* [Author](#author)
* [Copyright and License](#copyright-and-license)
* [See Also](#see-also)
* [Source Dependency](#source-dependency)
Status
======
This module is still under active development and is considered production ready.
Synopsis
========
nginx-consul:
```nginx-consul
stream {
upstream test {
upsync 127.0.0.1:8500/v1/kv/upstreams/test/ upsync_timeout=6m upsync_interval=500ms upsync_type=consul strong_dependency=off;
upsync_dump_path /usr/local/nginx/conf/servers/servers_test.conf;
include /usr/local/nginx/conf/servers/servers_test.conf;
}
upstream bar {
server 127.0.0.1:8090 weight=1 fail_timeout=10 max_fails=3;
}
server {
listen 12345;
proxy_connect_timeout 1s;
proxy_timeout 3s;
proxy_pass test;
}
server {
listen 2345;
upstream_show
}
server {
listen 127.0.0.1:9091;
proxy_responses 1;
proxy_timeout 20s;
proxy_pass bar;
}
}
```
nginx-etcd:
```nginx-etcd
stream {
upstream test {
upsync 127.0.0.1:2379/v2/keys/upstreams/test upsync_timeout=6m upsync_interval=500ms upsync_type=etcd strong_dependency=off;
upsync_dump_path /usr/local/nginx/conf/servers/servers_test.conf;
include /usr/local/nginx/conf/servers/servers_test.conf;
}
upstream bar {
server 127.0.0.1:8090 weight=1 fail_timeout=10 max_fails=3;
}
server {
listen 12345;
proxy_connect_timeout 1s;
proxy_timeout 3s;
proxy_pass test;
}
server {
listen 2345;
upstream_show
}
server {
listen 127.0.0.1:9091;
proxy_responses 1;
proxy_timeout 20s;
proxy_pass bar;
}
}
```
upsync_lb:
```upsync_lb
stream {
upstream test {
least_conn; //hash $uri consistent;
upsync 127.0.0.1:8500/v1/kv/upstreams/test/ upsync_timeout=6m upsync_interval=500ms upsync_type=consul strong_dependency=off;
upsync_dump_path /usr/local/nginx/conf/servers/servers_test.conf;
upsync_lb least_conn; //hash_ketama;
include /usr/local/nginx/conf/servers/servers_test.conf;
}
upstream bar {
server 127.0.0.1:8090 weight=1 fail_timeout=10 max_fails=3;
}
server {
listen 12345;
proxy_connect_timeout 1s;
proxy_timeout 3s;
proxy_pass test;
}
server {
listen 2345;
upstream_show
}
server {
listen 127.0.0.1:9091;
proxy_responses 1;
proxy_timeout 20s;
proxy_pass bar;
}
}
```
NOTE: upstream: include command is neccesary, first time the dumped file should include all the servers.
[Back to TOC](#table-of-contents)
Description
======
This module provides a method to discover backend servers. Supporting dynamicly adding or deleting backend server through consul/etcd and dynamicly adjusting backend servers weight, module will timely pull new backend server list from consul/etcd to upsync nginx ip router. Nginx needn't reload. Having some advantages than others:
* timely
module send key to consul/etcd with index, consul/etcd will compare it with its index, if index doesn't change connection will hang five minutes, in the period any operation to the key-value, will feed back rightaway.
* performance
Pulling from consul/etcd equal a request to nginx, updating ip router nginx needn't reload, so affecting nginx performance is little.
* stability
Even if one pulling failed, it will pull next upsync_interval, so guaranteing backend server stably provides service. And support dumping the latest config to location, so even if consul/etcd hung up, and nginx can be reload anytime.
[Back to TOC](#table-of-contents)
Directives
======
upsync
-----------
```
syntax: upsync $consul/etcd.api.com:$port/v1/kv/upstreams/$upstream_name/ [upsync_type=consul/etcd] [upsync_interval=second/minutes] [upsync_timeout=second/minutes] [strong_dependency=off/on]
```
default: none, if parameters omitted, default parameters are upsync_interval=5s upsync_timeout=6m strong_dependency=off
context: upstream
description: Pull upstream servers from consul/etcd... .
The parameters' meanings are:
* upsync_interval
pulling servers from consul/etcd interval time.
* upsync_timeout
pulling servers from consul/etcd request timeout.
* upsync_type
pulling servers from conf server type.
* strong_dependency
when nginx start up if strong_dependency is on that means servers will be depended on consul/etcd and will pull servers from consul/etcd.
upsync_dump_path
-----------
`syntax: upsync_dump_path $path`
default: /tmp/servers_$host.conf
context: upstream
description: dump the upstream backends to the $path.
upsync_lb
-----------
`syntax: upsync_lb $load_balance`
default: round_robin/ip_hash/hash modula
context: upstream
description: mainly for least_conn and hash consistent, when using one of them, you must point out using upsync_lb.
upsync_show
-----------
`syntax: upsync_show`
default: none
context: server
description: show all upstreams.
```request
curl http://localhost:2345/upstream_show
show all upstreams
```
[Back to TOC](#table-of-contents)
Consul_interface
======
Data can be taken from key/value store or service catalog. In the first case parameter upsync_type of directive must be *consul*. For example
```nginx-consul
upsync 127.0.0.1:8500/v1/kv/upstreams/test upsync_timeout=6m upsync_interval=500ms upsync_type=consul strong_dependency=off;
```
In the second case it must be *consul_services*.
```nginx-consul
upsync 127.0.0.1:8500/v1/catalog/service/test upsync_timeout=6m upsync_interval=500ms upsync_type=consul_services strong_dependency=off;
```
you can add or delete backend server through consul_ui or http_interface. Below are examples for key/value store.
http_interface example:
* add
```
curl -X PUT http://$consul_ip:$port/v1/kv/upstreams/$upstream_name/$backend_ip:$backend_port
```
default: weight=1 max_fails=2 fail_timeout=10 down=0 backup=0;
```
curl -X PUT -d "{\"weight\":1, \"max_fails\":2, \"fail_timeout\":10}" http://$consul_ip:$port/v1/kv/$dir1/$upstream_name/$backend_ip:$backend_port
or
curl -X PUT -d '{"weight":1, "max_fails":2, "fail_timeout":10}' http://$consul_ip:$port/v1/kv/$dir1/$upstream_name/$backend_ip:$backend_port
```
value support json format.
* delete
```
curl -X DELETE http://$consul_ip:$port/v1/kv/upstreams/$upstream_name/$backend_ip:$backend_port
```
* adjust-weight
```
curl -X PUT -d "{\"weight\":2, \"max_fails\":2, \"fail_timeout\":10}" http://$consul_ip:$port/v1/kv/$dir1/$upstream_name/$backend_ip:$backend_port
or
curl -X PUT -d '{"weight":2, "max_fails":2, "fail_timeout":10}' http://$consul_ip:$port/v1/kv/$dir1/$upstream_name/$backend_ip:$backend_port
```
* mark server-down
```
curl -X PUT -d "{\"weight\":2, \"max_fails\":2, \"fail_timeout\":10, \"down\":1}" http://$consul_ip:$port/v1/kv/$dir1/$upstream_name/$backend_ip:$backend_port
or
curl -X PUT -d '{"weight":2, "max_fails":2, "fail_timeout":10, "down":1}' http://$consul_ip:$port/v1/kv/$dir1/$upstream_name/$backend_ip:$backend_port
```
* check
```
curl http://$consul_ip:$port/v1/kv/upstreams/$upstream_name?recurse
```
[Back to TOC](#table-of-contents)
Etcd_interface
======
you can add or delete backend server through http_interface.
mainly like etcd, http_interface example:
* add
```
curl -X PUT http://$etcd_ip:$port/v2/keys/upstreams/$upstream_name/$backend_ip:$backend_port
```
default: weight=1 max_fails=2 fail_timeout=10 down=0 backup=0;
```
curl -X PUT -d value="{\"weight\":1, \"max_fails\":2, \"fail_timeout\":10}" http://$etcd_ip:$port/v2/keys/$dir1/$upstream_name/$backend_ip:$backend_port
```
value support json format.
* delete
```
curl -X DELETE http://$etcd_ip:$port/v2/keys/upstreams/$upstream_name/$backend_ip:$backend_port
```
* adjust-weight
```
curl -X PUT -d "{\"weight\":2, \"max_fails\":2, \"fail_timeout\":10}" http://$etcd_ip:$port/v2/keys/$dir1/$upstream_name/$backend_ip:$backend_port
```
* mark server-down
```
curl -X PUT -d value="{\"weight\":2, \"max_fails\":2, \"fail_timeout\":10, \"down\":1}" http://$etcd_ip:$port/v2/keys/$dir1/$upstream_name/$backend_ip:$backend_port
```
* check
```
curl http://$etcd_ip:$port/v2/keys/upstreams/$upstream_name
```
[Back to TOC](#table-of-contents)
TODO
====
* support zookeeper and so on
[Back to TOC](#table-of-contents)
Compatibility
=============
The module was developed base on nginx-1.9.10.
Master branch compatible with nginx-1.11.0+.
Nginx-1.10.3- branch compatible with nginx-1.9.10 ~ nginx-1.11.0+.
[Back to TOC](#table-of-contents)
Installation
============
This module can be used independently, can be download[Github](https://github.com/xiaokai-wang/nginx-stream-upsync-module.git).
Grab the nginx source code from [nginx.org](http://nginx.org/), for example, the version 1.8.0 (see nginx compatibility), and then build the source with this module:
```bash
wget 'http://nginx.org/download/nginx-1.8.0.tar.gz'
tar -xzvf nginx-1.8.0.tar.gz
cd nginx-1.8.0/
```
```bash
./configure --add-module=/path/to/nginx-stream_upsync-module
make
make install
```
if you support nginx-upstream-check-module
```bash
./configure --add-module=/path/to/nginx-upstream-check-module --add-module=/path/to/nginx-stream_upsync-module
make
make install
```
[Back to TOC](#table-of-contents)
Code style
======
Code style is mainly based on [style](http://tengine.taobao.org/book/appendix_a.html)
[Back to TOC](#table-of-contents)
Author
======
Xiaokai Wang (王晓开) <xiaokai.wang@live.com>, Weibo Inc.
[Back to TOC](#table-of-contents)
Copyright and License
=====================
This README template copy from agentzh.
This module is licensed under the BSD license.
Copyright (C) 2014 by Xiaokai Wang <xiaokai.wang@live.com></xiaokai.wang@live.com>
All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
[Back to TOC](#table-of-contents)
see also
========
* the nginx_upstream_check_module: https://github.com/alibaba/tengine/blob/master/src/http/ngx_http_upstream_check_module.c
* the nginx_upstream_check_module patch: https://github.com/yaoweibin/nginx_upstream_check_module
* or based on https://github.com/xiaokai-wang/nginx_upstream_check_module
[back to toc](#table-of-contents)
source dependency
========
* Cjson: https://github.com/kbranigan/cJSON
* http-parser: https://github.com/nodejs/http-parser
[back to toc](#table-of-contents)
================================================
FILE: config
================================================
ngx_addon_name=ngx_stream_upsync_module
ngx_feature_libs="-lm"
ngx_module_incs=$ngx_addon_dir/src
_STREAM_UPSYNC_SRCS="\
$ngx_addon_dir/src/ngx_stream_upsync_module.c \
$ngx_addon_dir/src/ngx_stream_json.c \
$ngx_addon_dir/src/ngx_stream_http_parser.c \
"
have=NGX_STREAM_UPSYNC . auto/have
if test -n "$ngx_module_link"; then
ngx_module_type=STREAM
ngx_module_name=$ngx_addon_name
ngx_module_srcs="$_STREAM_UPSYNC_SRCS"
ngx_module_libs=$ngx_feature_libs
. auto/module
else
NGX_ADDON_SRCS="$NGX_ADDON_SRCS $_STREAM_UPSYNC_SRCS"
CORE_LIBS="$CORE_LIBS $ngx_feature_libs"
CORE_INCS="$CORE_INCS $ngx_module_incs"
STREAM_MODULES="$STREAM_MODULES $ngx_addon_name"
fi
================================================
FILE: src/ngx_stream_http_parser.c
================================================
/* Based on src/http/ngx_http_parse.c from NGINX copyright Igor Sysoev
*
* Additional changes are licensed under the same terms as NGINX and
* copyright Joyent, Inc. and other Node contributors. All rights reserved.
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to
* deal in the Software without restriction, including without limitation the
* rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
* sell copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
* IN THE SOFTWARE.
*/
#include <ngx_config.h>
#ifndef NGX_HTTP_UPSYNC
#include "ngx_stream_http_parser.h"
#include <assert.h>
#include <stddef.h>
#include <ctype.h>
#include <stdlib.h>
#include <string.h>
#include <limits.h>
#ifndef ULLONG_MAX
# define ULLONG_MAX ((uint64_t) -1) /* 2^64-1 */
#endif
#ifndef MIN
# define MIN(a,b) ((a) < (b) ? (a) : (b))
#endif
#ifndef ARRAY_SIZE
# define ARRAY_SIZE(a) (sizeof(a) / sizeof((a)[0]))
#endif
#ifndef BIT_AT
# define BIT_AT(a, i) \
(!!((unsigned int) (a)[(unsigned int) (i) >> 3] & \
(1 << ((unsigned int) (i) & 7))))
#endif
#ifndef ELEM_AT
# define ELEM_AT(a, i, v) ((unsigned int) (i) < ARRAY_SIZE(a) ? (a)[(i)] : (v))
#endif
#define SET_ERRNO(e) \
do { \
parser->http_errno = (e); \
} while(0)
#define CURRENT_STATE() p_state
#define UPDATE_STATE(V) p_state = (enum state) (V);
#define RETURN(V) \
do { \
parser->state = CURRENT_STATE(); \
return (V); \
} while (0);
#define REEXECUTE() \
goto reexecute; \
#ifdef __GNUC__
# define LIKELY(X) __builtin_expect(!!(X), 1)
# define UNLIKELY(X) __builtin_expect(!!(X), 0)
#else
# define LIKELY(X) (X)
# define UNLIKELY(X) (X)
#endif
/* Run the notify callback FOR, returning ER if it fails */
#define CALLBACK_NOTIFY_(FOR, ER) \
do { \
assert(HTTP_PARSER_ERRNO(parser) == HPE_OK); \
\
if (LIKELY(settings->on_##FOR)) { \
parser->state = CURRENT_STATE(); \
if (UNLIKELY(0 != settings->on_##FOR(parser))) { \
SET_ERRNO(HPE_CB_##FOR); \
} \
UPDATE_STATE(parser->state); \
\
/* We either errored above or got paused; get out */ \
if (UNLIKELY(HTTP_PARSER_ERRNO(parser) != HPE_OK)) { \
return (ER); \
} \
} \
} while (0)
/* Run the notify callback FOR and consume the current byte */
#define CALLBACK_NOTIFY(FOR) CALLBACK_NOTIFY_(FOR, p - data + 1)
/* Run the notify callback FOR and don't consume the current byte */
#define CALLBACK_NOTIFY_NOADVANCE(FOR) CALLBACK_NOTIFY_(FOR, p - data)
/* Run data callback FOR with LEN bytes, returning ER if it fails */
#define CALLBACK_DATA_(FOR, LEN, ER) \
do { \
assert(HTTP_PARSER_ERRNO(parser) == HPE_OK); \
\
if (FOR##_mark) { \
if (LIKELY(settings->on_##FOR)) { \
parser->state = CURRENT_STATE(); \
if (UNLIKELY(0 != \
settings->on_##FOR(parser, FOR##_mark, (LEN)))) { \
SET_ERRNO(HPE_CB_##FOR); \
} \
UPDATE_STATE(parser->state); \
\
/* We either errored above or got paused; get out */ \
if (UNLIKELY(HTTP_PARSER_ERRNO(parser) != HPE_OK)) { \
return (ER); \
} \
} \
FOR##_mark = NULL; \
} \
} while (0)
/* Run the data callback FOR and consume the current byte */
#define CALLBACK_DATA(FOR) \
CALLBACK_DATA_(FOR, p - FOR##_mark, p - data + 1)
/* Run the data callback FOR and don't consume the current byte */
#define CALLBACK_DATA_NOADVANCE(FOR) \
CALLBACK_DATA_(FOR, p - FOR##_mark, p - data)
/* Set the mark FOR; non-destructive if mark is already set */
#define MARK(FOR) \
do { \
if (!FOR##_mark) { \
FOR##_mark = p; \
} \
} while (0)
/* Don't allow the total size of the HTTP headers (including the status
* line) to exceed HTTP_MAX_HEADER_SIZE. This check is here to protect
* embedders against denial-of-service attacks where the attacker feeds
* us a never-ending header that the embedder keeps buffering.
*
* This check is arguably the responsibility of embedders but we're doing
* it on the embedder's behalf because most won't bother and this way we
* make the web a little safer. HTTP_MAX_HEADER_SIZE is still far bigger
* than any reasonable request or response so this should never affect
* day-to-day operation.
*/
#define COUNT_HEADER_SIZE(V) \
do { \
parser->nread += (V); \
if (UNLIKELY(parser->nread > (HTTP_MAX_HEADER_SIZE))) { \
SET_ERRNO(HPE_HEADER_OVERFLOW); \
goto error; \
} \
} while (0)
#define PROXY_CONNECTION "proxy-connection"
#define CONNECTION "connection"
#define CONTENT_LENGTH "content-length"
#define TRANSFER_ENCODING "transfer-encoding"
#define UPGRADE "upgrade"
#define CHUNKED "chunked"
#define KEEP_ALIVE "keep-alive"
#define CLOSE "close"
static const char *method_strings[] =
{
#define XX(num, name, string) #string,
HTTP_METHOD_MAP(XX)
#undef XX
};
/* Tokens as defined by rfc 2616. Also lowercases them.
* token = 1*<any CHAR except CTLs or separators>
* separators = "(" | ")" | "<" | ">" | "@"
* | "," | ";" | ":" | "\" | <">
* | "/" | "[" | "]" | "?" | "="
* | "{" | "}" | SP | HT
*/
static const char tokens[256] = {
/* 0 nul 1 soh 2 stx 3 etx 4 eot 5 enq 6 ack 7 bel */
0, 0, 0, 0, 0, 0, 0, 0,
/* 8 bs 9 ht 10 nl 11 vt 12 np 13 cr 14 so 15 si */
0, 0, 0, 0, 0, 0, 0, 0,
/* 16 dle 17 dc1 18 dc2 19 dc3 20 dc4 21 nak 22 syn 23 etb */
0, 0, 0, 0, 0, 0, 0, 0,
/* 24 can 25 em 26 sub 27 esc 28 fs 29 gs 30 rs 31 us */
0, 0, 0, 0, 0, 0, 0, 0,
/* 32 sp 33 ! 34 " 35 # 36 $ 37 % 38 & 39 ' */
0, '!', 0, '#', '$', '%', '&', '\'',
/* 40 ( 41 ) 42 * 43 + 44 , 45 - 46 . 47 / */
0, 0, '*', '+', 0, '-', '.', 0,
/* 48 0 49 1 50 2 51 3 52 4 53 5 54 6 55 7 */
'0', '1', '2', '3', '4', '5', '6', '7',
/* 56 8 57 9 58 : 59 ; 60 < 61 = 62 > 63 ? */
'8', '9', 0, 0, 0, 0, 0, 0,
/* 64 @ 65 A 66 B 67 C 68 D 69 E 70 F 71 G */
0, 'a', 'b', 'c', 'd', 'e', 'f', 'g',
/* 72 H 73 I 74 J 75 K 76 L 77 M 78 N 79 O */
'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o',
/* 80 P 81 Q 82 R 83 S 84 T 85 U 86 V 87 W */
'p', 'q', 'r', 's', 't', 'u', 'v', 'w',
/* 88 X 89 Y 90 Z 91 [ 92 \ 93 ] 94 ^ 95 _ */
'x', 'y', 'z', 0, 0, 0, '^', '_',
/* 96 ` 97 a 98 b 99 c 100 d 101 e 102 f 103 g */
'`', 'a', 'b', 'c', 'd', 'e', 'f', 'g',
/* 104 h 105 i 106 j 107 k 108 l 109 m 110 n 111 o */
'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o',
/* 112 p 113 q 114 r 115 s 116 t 117 u 118 v 119 w */
'p', 'q', 'r', 's', 't', 'u', 'v', 'w',
/* 120 x 121 y 122 z 123 { 124 | 125 } 126 ~ 127 del */
'x', 'y', 'z', 0, '|', 0, '~', 0 };
static const int8_t unhex[256] =
{-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,-1,-1,-1,-1,-1,-1
,-1,10,11,12,13,14,15,-1,-1,-1,-1,-1,-1,-1,-1,-1
,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
,-1,10,11,12,13,14,15,-1,-1,-1,-1,-1,-1,-1,-1,-1
,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1
};
#if HTTP_PARSER_STRICT
# define T(v) 0
#else
# define T(v) v
#endif
static const uint8_t normal_url_char[32] = {
/* 0 nul 1 soh 2 stx 3 etx 4 eot 5 enq 6 ack 7 bel */
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0,
/* 8 bs 9 ht 10 nl 11 vt 12 np 13 cr 14 so 15 si */
0 | T(2) | 0 | 0 | T(16) | 0 | 0 | 0,
/* 16 dle 17 dc1 18 dc2 19 dc3 20 dc4 21 nak 22 syn 23 etb */
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0,
/* 24 can 25 em 26 sub 27 esc 28 fs 29 gs 30 rs 31 us */
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0,
/* 32 sp 33 ! 34 " 35 # 36 $ 37 % 38 & 39 ' */
0 | 2 | 4 | 0 | 16 | 32 | 64 | 128,
/* 40 ( 41 ) 42 * 43 + 44 , 45 - 46 . 47 / */
1 | 2 | 4 | 8 | 16 | 32 | 64 | 128,
/* 48 0 49 1 50 2 51 3 52 4 53 5 54 6 55 7 */
1 | 2 | 4 | 8 | 16 | 32 | 64 | 128,
/* 56 8 57 9 58 : 59 ; 60 < 61 = 62 > 63 ? */
1 | 2 | 4 | 8 | 16 | 32 | 64 | 0,
/* 64 @ 65 A 66 B 67 C 68 D 69 E 70 F 71 G */
1 | 2 | 4 | 8 | 16 | 32 | 64 | 128,
/* 72 H 73 I 74 J 75 K 76 L 77 M 78 N 79 O */
1 | 2 | 4 | 8 | 16 | 32 | 64 | 128,
/* 80 P 81 Q 82 R 83 S 84 T 85 U 86 V 87 W */
1 | 2 | 4 | 8 | 16 | 32 | 64 | 128,
/* 88 X 89 Y 90 Z 91 [ 92 \ 93 ] 94 ^ 95 _ */
1 | 2 | 4 | 8 | 16 | 32 | 64 | 128,
/* 96 ` 97 a 98 b 99 c 100 d 101 e 102 f 103 g */
1 | 2 | 4 | 8 | 16 | 32 | 64 | 128,
/* 104 h 105 i 106 j 107 k 108 l 109 m 110 n 111 o */
1 | 2 | 4 | 8 | 16 | 32 | 64 | 128,
/* 112 p 113 q 114 r 115 s 116 t 117 u 118 v 119 w */
1 | 2 | 4 | 8 | 16 | 32 | 64 | 128,
/* 120 x 121 y 122 z 123 { 124 | 125 } 126 ~ 127 del */
1 | 2 | 4 | 8 | 16 | 32 | 64 | 0, };
#undef T
enum state
{ s_dead = 1 /* important that this is > 0 */
, s_start_req_or_res
, s_res_or_resp_H
, s_start_res
, s_res_H
, s_res_HT
, s_res_HTT
, s_res_HTTP
, s_res_first_http_major
, s_res_http_major
, s_res_first_http_minor
, s_res_http_minor
, s_res_first_status_code
, s_res_status_code
, s_res_status_start
, s_res_status
, s_res_line_almost_done
, s_start_req
, s_req_method
, s_req_spaces_before_url
, s_req_schema
, s_req_schema_slash
, s_req_schema_slash_slash
, s_req_server_start
, s_req_server
, s_req_server_with_at
, s_req_path
, s_req_query_string_start
, s_req_query_string
, s_req_fragment_start
, s_req_fragment
, s_req_http_start
, s_req_http_H
, s_req_http_HT
, s_req_http_HTT
, s_req_http_HTTP
, s_req_first_http_major
, s_req_http_major
, s_req_first_http_minor
, s_req_http_minor
, s_req_line_almost_done
, s_header_field_start
, s_header_field
, s_header_value_discard_ws
, s_header_value_discard_ws_almost_done
, s_header_value_discard_lws
, s_header_value_start
, s_header_value
, s_header_value_lws
, s_header_almost_done
, s_chunk_size_start
, s_chunk_size
, s_chunk_parameters
, s_chunk_size_almost_done
, s_headers_almost_done
, s_headers_done
/* Important: 's_headers_done' must be the last 'header' state. All
* states beyond this must be 'body' states. It is used for overflow
* checking. See the PARSING_HEADER() macro.
*/
, s_chunk_data
, s_chunk_data_almost_done
, s_chunk_data_done
, s_body_identity
, s_body_identity_eof
, s_message_done
};
#define PARSING_HEADER(state) (state <= s_headers_done)
enum header_states
{ h_general = 0
, h_C
, h_CO
, h_CON
, h_matching_connection
, h_matching_proxy_connection
, h_matching_content_length
, h_matching_transfer_encoding
, h_matching_upgrade
, h_connection
, h_content_length
, h_transfer_encoding
, h_upgrade
, h_matching_transfer_encoding_chunked
, h_matching_connection_token_start
, h_matching_connection_keep_alive
, h_matching_connection_close
, h_matching_connection_upgrade
, h_matching_connection_token
, h_transfer_encoding_chunked
, h_connection_keep_alive
, h_connection_close
, h_connection_upgrade
};
enum http_host_state
{
s_http_host_dead = 1
, s_http_userinfo_start
, s_http_userinfo
, s_http_host_start
, s_http_host_v6_start
, s_http_host
, s_http_host_v6
, s_http_host_v6_end
, s_http_host_port_start
, s_http_host_port
};
/* Macros for character classes; depends on strict-mode */
#define CR '\r'
#define LF '\n'
#define LOWER(c) (unsigned char)(c | 0x20)
#define IS_ALPHA(c) (LOWER(c) >= 'a' && LOWER(c) <= 'z')
#define IS_NUM(c) ((c) >= '0' && (c) <= '9')
#define IS_ALPHANUM(c) (IS_ALPHA(c) || IS_NUM(c))
#define IS_HEX(c) (IS_NUM(c) || (LOWER(c) >= 'a' && LOWER(c) <= 'f'))
#define IS_MARK(c) ((c) == '-' || (c) == '_' || (c) == '.' || \
(c) == '!' || (c) == '~' || (c) == '*' || (c) == '\'' || (c) == '(' || \
(c) == ')')
#define IS_USERINFO_CHAR(c) (IS_ALPHANUM(c) || IS_MARK(c) || (c) == '%' || \
(c) == ';' || (c) == ':' || (c) == '&' || (c) == '=' || (c) == '+' || \
(c) == '$' || (c) == ',')
#define STRICT_TOKEN(c) (tokens[(unsigned char)c])
#if HTTP_PARSER_STRICT
#define TOKEN(c) (tokens[(unsigned char)c])
#define IS_URL_CHAR(c) (BIT_AT(normal_url_char, (unsigned char)c))
#define IS_HOST_CHAR(c) (IS_ALPHANUM(c) || (c) == '.' || (c) == '-')
#else
#define TOKEN(c) ((c == ' ') ? ' ' : tokens[(unsigned char)c])
#define IS_URL_CHAR(c) \
(BIT_AT(normal_url_char, (unsigned char)c) || ((c) & 0x80))
#define IS_HOST_CHAR(c) \
(IS_ALPHANUM(c) || (c) == '.' || (c) == '-' || (c) == '_')
#endif
#define start_state (parser->type == HTTP_REQUEST ? s_start_req : s_start_res)
#if HTTP_PARSER_STRICT
# define STRICT_CHECK(cond) \
do { \
if (cond) { \
SET_ERRNO(HPE_STRICT); \
goto error; \
} \
} while (0)
# define NEW_MESSAGE() (http_should_keep_alive(parser) ? start_state : s_dead)
#else
# define STRICT_CHECK(cond)
# define NEW_MESSAGE() start_state
#endif
/* Map errno values to strings for human-readable output */
#define HTTP_STRERROR_GEN(n, s) { "HPE_" #n, s },
static struct {
const char *name;
const char *description;
} http_strerror_tab[] = {
HTTP_ERRNO_MAP(HTTP_STRERROR_GEN)
};
#undef HTTP_STRERROR_GEN
int http_message_needs_eof(const http_parser *parser);
/* Our URL parser.
*
* This is designed to be shared by http_parser_execute() for URL validation,
* hence it has a state transition + byte-for-byte interface. In addition, it
* is meant to be embedded in http_parser_parse_url(), which does the dirty
* work of turning state transitions URL components for its API.
*
* This function should only be invoked with non-space characters. It is
* assumed that the caller cares about (and can detect) the transition between
* URL and non-URL states by looking for these.
*/
static enum state
parse_url_char(enum state s, const char ch)
{
if (ch == ' ' || ch == '\r' || ch == '\n') {
return s_dead;
}
#if HTTP_PARSER_STRICT
if (ch == '\t' || ch == '\f') {
return s_dead;
}
#endif
switch (s) {
case s_req_spaces_before_url:
/* Proxied requests are followed by scheme of an absolute URI (alpha).
* All methods except CONNECT are followed by '/' or '*'.
*/
if (ch == '/' || ch == '*') {
return s_req_path;
}
if (IS_ALPHA(ch)) {
return s_req_schema;
}
break;
case s_req_schema:
if (IS_ALPHA(ch)) {
return s;
}
if (ch == ':') {
return s_req_schema_slash;
}
break;
case s_req_schema_slash:
if (ch == '/') {
return s_req_schema_slash_slash;
}
break;
case s_req_schema_slash_slash:
if (ch == '/') {
return s_req_server_start;
}
break;
case s_req_server_with_at:
if (ch == '@') {
return s_dead;
}
/* FALLTHROUGH */
case s_req_server_start:
case s_req_server:
if (ch == '/') {
return s_req_path;
}
if (ch == '?') {
return s_req_query_string_start;
}
if (ch == '@') {
return s_req_server_with_at;
}
if (IS_USERINFO_CHAR(ch) || ch == '[' || ch == ']') {
return s_req_server;
}
break;
case s_req_path:
if (IS_URL_CHAR(ch)) {
return s;
}
switch (ch) {
case '?':
return s_req_query_string_start;
case '#':
return s_req_fragment_start;
}
break;
case s_req_query_string_start:
case s_req_query_string:
if (IS_URL_CHAR(ch)) {
return s_req_query_string;
}
switch (ch) {
case '?':
/* allow extra '?' in query string */
return s_req_query_string;
case '#':
return s_req_fragment_start;
}
break;
case s_req_fragment_start:
if (IS_URL_CHAR(ch)) {
return s_req_fragment;
}
switch (ch) {
case '?':
return s_req_fragment;
case '#':
return s;
}
break;
case s_req_fragment:
if (IS_URL_CHAR(ch)) {
return s;
}
switch (ch) {
case '?':
case '#':
return s;
}
break;
default:
break;
}
/* We should never fall out of the switch above unless there's an error */
return s_dead;
}
size_t http_parser_execute (http_parser *parser,
const http_parser_settings *settings,
const char *data,
size_t len)
{
char c, ch;
int8_t unhex_val;
const char *p = data;
const char *header_field_mark = 0;
const char *header_value_mark = 0;
const char *url_mark = 0;
const char *body_mark = 0;
const char *status_mark = 0;
enum state p_state = (enum state) parser->state;
/* We're in an error state. Don't bother doing anything. */
if (HTTP_PARSER_ERRNO(parser) != HPE_OK) {
return 0;
}
if (len == 0) {
switch (CURRENT_STATE()) {
case s_body_identity_eof:
/* Use of CALLBACK_NOTIFY() here would erroneously return 1 byte read if
* we got paused.
*/
CALLBACK_NOTIFY_NOADVANCE(message_complete);
return 0;
case s_dead:
case s_start_req_or_res:
case s_start_res:
case s_start_req:
return 0;
default:
SET_ERRNO(HPE_INVALID_EOF_STATE);
return 1;
}
}
if (CURRENT_STATE() == s_header_field)
header_field_mark = data;
if (CURRENT_STATE() == s_header_value)
header_value_mark = data;
switch (CURRENT_STATE()) {
case s_req_path:
case s_req_schema:
case s_req_schema_slash:
case s_req_schema_slash_slash:
case s_req_server_start:
case s_req_server:
case s_req_server_with_at:
case s_req_query_string_start:
case s_req_query_string:
case s_req_fragment_start:
case s_req_fragment:
url_mark = data;
break;
case s_res_status:
status_mark = data;
break;
default:
break;
}
for (p=data; p != data + len; p++) {
ch = *p;
if (PARSING_HEADER(CURRENT_STATE()))
COUNT_HEADER_SIZE(1);
reexecute:
switch (CURRENT_STATE()) {
case s_dead:
/* this state is used after a 'Connection: close' message
* the parser will error out if it reads another message
*/
if (LIKELY(ch == CR || ch == LF))
break;
SET_ERRNO(HPE_CLOSED_CONNECTION);
goto error;
case s_start_req_or_res:
{
if (ch == CR || ch == LF)
break;
parser->flags = 0;
parser->content_length = ULLONG_MAX;
if (ch == 'H') {
UPDATE_STATE(s_res_or_resp_H);
CALLBACK_NOTIFY(message_begin);
} else {
parser->type = HTTP_REQUEST;
UPDATE_STATE(s_start_req);
REEXECUTE();
}
break;
}
case s_res_or_resp_H:
if (ch == 'T') {
parser->type = HTTP_RESPONSE;
UPDATE_STATE(s_res_HT);
} else {
if (UNLIKELY(ch != 'E')) {
SET_ERRNO(HPE_INVALID_CONSTANT);
goto error;
}
parser->type = HTTP_REQUEST;
parser->method = HTTP_HEAD;
parser->index = 2;
UPDATE_STATE(s_req_method);
}
break;
case s_start_res:
{
parser->flags = 0;
parser->content_length = ULLONG_MAX;
switch (ch) {
case 'H':
UPDATE_STATE(s_res_H);
break;
case CR:
case LF:
break;
default:
SET_ERRNO(HPE_INVALID_CONSTANT);
goto error;
}
CALLBACK_NOTIFY(message_begin);
break;
}
case s_res_H:
STRICT_CHECK(ch != 'T');
UPDATE_STATE(s_res_HT);
break;
case s_res_HT:
STRICT_CHECK(ch != 'T');
UPDATE_STATE(s_res_HTT);
break;
case s_res_HTT:
STRICT_CHECK(ch != 'P');
UPDATE_STATE(s_res_HTTP);
break;
case s_res_HTTP:
STRICT_CHECK(ch != '/');
UPDATE_STATE(s_res_first_http_major);
break;
case s_res_first_http_major:
if (UNLIKELY(ch < '0' || ch > '9')) {
SET_ERRNO(HPE_INVALID_VERSION);
goto error;
}
parser->http_major = ch - '0';
UPDATE_STATE(s_res_http_major);
break;
/* major HTTP version or dot */
case s_res_http_major:
{
if (ch == '.') {
UPDATE_STATE(s_res_first_http_minor);
break;
}
if (!IS_NUM(ch)) {
SET_ERRNO(HPE_INVALID_VERSION);
goto error;
}
parser->http_major *= 10;
parser->http_major += ch - '0';
if (UNLIKELY(parser->http_major > 999)) {
SET_ERRNO(HPE_INVALID_VERSION);
goto error;
}
break;
}
/* first digit of minor HTTP version */
case s_res_first_http_minor:
if (UNLIKELY(!IS_NUM(ch))) {
SET_ERRNO(HPE_INVALID_VERSION);
goto error;
}
parser->http_minor = ch - '0';
UPDATE_STATE(s_res_http_minor);
break;
/* minor HTTP version or end of request line */
case s_res_http_minor:
{
if (ch == ' ') {
UPDATE_STATE(s_res_first_status_code);
break;
}
if (UNLIKELY(!IS_NUM(ch))) {
SET_ERRNO(HPE_INVALID_VERSION);
goto error;
}
parser->http_minor *= 10;
parser->http_minor += ch - '0';
if (UNLIKELY(parser->http_minor > 999)) {
SET_ERRNO(HPE_INVALID_VERSION);
goto error;
}
break;
}
case s_res_first_status_code:
{
if (!IS_NUM(ch)) {
if (ch == ' ') {
break;
}
SET_ERRNO(HPE_INVALID_STATUS);
goto error;
}
parser->status_code = ch - '0';
UPDATE_STATE(s_res_status_code);
break;
}
case s_res_status_code:
{
if (!IS_NUM(ch)) {
switch (ch) {
case ' ':
UPDATE_STATE(s_res_status_start);
break;
case CR:
UPDATE_STATE(s_res_line_almost_done);
break;
case LF:
UPDATE_STATE(s_header_field_start);
break;
default:
SET_ERRNO(HPE_INVALID_STATUS);
goto error;
}
break;
}
parser->status_code *= 10;
parser->status_code += ch - '0';
if (UNLIKELY(parser->status_code > 999)) {
SET_ERRNO(HPE_INVALID_STATUS);
goto error;
}
break;
}
case s_res_status_start:
{
if (ch == CR) {
UPDATE_STATE(s_res_line_almost_done);
break;
}
if (ch == LF) {
UPDATE_STATE(s_header_field_start);
break;
}
MARK(status);
UPDATE_STATE(s_res_status);
parser->index = 0;
break;
}
case s_res_status:
if (ch == CR) {
UPDATE_STATE(s_res_line_almost_done);
CALLBACK_DATA(status);
break;
}
if (ch == LF) {
UPDATE_STATE(s_header_field_start);
CALLBACK_DATA(status);
break;
}
break;
case s_res_line_almost_done:
STRICT_CHECK(ch != LF);
UPDATE_STATE(s_header_field_start);
break;
case s_start_req:
{
if (ch == CR || ch == LF)
break;
parser->flags = 0;
parser->content_length = ULLONG_MAX;
if (UNLIKELY(!IS_ALPHA(ch))) {
SET_ERRNO(HPE_INVALID_METHOD);
goto error;
}
parser->method = (enum http_method) 0;
parser->index = 1;
switch (ch) {
case 'C': parser->method = HTTP_CONNECT; /* or COPY, CHECKOUT */ break;
case 'D': parser->method = HTTP_DELETE; break;
case 'G': parser->method = HTTP_GET; break;
case 'H': parser->method = HTTP_HEAD; break;
case 'L': parser->method = HTTP_LOCK; break;
case 'M': parser->method = HTTP_MKCOL; /* or MOVE, MKACTIVITY, MERGE, M-SEARCH, MKCALENDAR */ break;
case 'N': parser->method = HTTP_NOTIFY; break;
case 'O': parser->method = HTTP_OPTIONS; break;
case 'P': parser->method = HTTP_POST;
/* or PROPFIND|PROPPATCH|PUT|PATCH|PURGE */
break;
case 'R': parser->method = HTTP_REPORT; break;
case 'S': parser->method = HTTP_SUBSCRIBE; /* or SEARCH */ break;
case 'T': parser->method = HTTP_TRACE; break;
case 'U': parser->method = HTTP_UNLOCK; /* or UNSUBSCRIBE */ break;
default:
SET_ERRNO(HPE_INVALID_METHOD);
goto error;
}
UPDATE_STATE(s_req_method);
CALLBACK_NOTIFY(message_begin);
break;
}
case s_req_method:
{
const char *matcher;
if (UNLIKELY(ch == '\0')) {
SET_ERRNO(HPE_INVALID_METHOD);
goto error;
}
matcher = method_strings[parser->method];
if (ch == ' ' && matcher[parser->index] == '\0') {
UPDATE_STATE(s_req_spaces_before_url);
} else if (ch == matcher[parser->index]) {
; /* nada */
} else if (parser->method == HTTP_CONNECT) {
if (parser->index == 1 && ch == 'H') {
parser->method = HTTP_CHECKOUT;
} else if (parser->index == 2 && ch == 'P') {
parser->method = HTTP_COPY;
} else {
SET_ERRNO(HPE_INVALID_METHOD);
goto error;
}
} else if (parser->method == HTTP_MKCOL) {
if (parser->index == 1 && ch == 'O') {
parser->method = HTTP_MOVE;
} else if (parser->index == 1 && ch == 'E') {
parser->method = HTTP_MERGE;
} else if (parser->index == 1 && ch == '-') {
parser->method = HTTP_MSEARCH;
} else if (parser->index == 2 && ch == 'A') {
parser->method = HTTP_MKACTIVITY;
} else if (parser->index == 3 && ch == 'A') {
parser->method = HTTP_MKCALENDAR;
} else {
SET_ERRNO(HPE_INVALID_METHOD);
goto error;
}
} else if (parser->method == HTTP_SUBSCRIBE) {
if (parser->index == 1 && ch == 'E') {
parser->method = HTTP_SEARCH;
} else {
SET_ERRNO(HPE_INVALID_METHOD);
goto error;
}
} else if (parser->index == 1 && parser->method == HTTP_POST) {
if (ch == 'R') {
parser->method = HTTP_PROPFIND; /* or HTTP_PROPPATCH */
} else if (ch == 'U') {
parser->method = HTTP_PUT; /* or HTTP_PURGE */
} else if (ch == 'A') {
parser->method = HTTP_PATCH;
} else {
SET_ERRNO(HPE_INVALID_METHOD);
goto error;
}
} else if (parser->index == 2) {
if (parser->method == HTTP_PUT) {
if (ch == 'R') {
parser->method = HTTP_PURGE;
} else {
SET_ERRNO(HPE_INVALID_METHOD);
goto error;
}
} else if (parser->method == HTTP_UNLOCK) {
if (ch == 'S') {
parser->method = HTTP_UNSUBSCRIBE;
} else {
SET_ERRNO(HPE_INVALID_METHOD);
goto error;
}
} else {
SET_ERRNO(HPE_INVALID_METHOD);
goto error;
}
} else if (parser->index == 4 && parser->method == HTTP_PROPFIND && ch == 'P') {
parser->method = HTTP_PROPPATCH;
} else {
SET_ERRNO(HPE_INVALID_METHOD);
goto error;
}
++parser->index;
break;
}
case s_req_spaces_before_url:
{
if (ch == ' ') break;
MARK(url);
if (parser->method == HTTP_CONNECT) {
UPDATE_STATE(s_req_server_start);
}
UPDATE_STATE(parse_url_char(CURRENT_STATE(), ch));
if (UNLIKELY(CURRENT_STATE() == s_dead)) {
SET_ERRNO(HPE_INVALID_URL);
goto error;
}
break;
}
case s_req_schema:
case s_req_schema_slash:
case s_req_schema_slash_slash:
case s_req_server_start:
{
switch (ch) {
/* No whitespace allowed here */
case ' ':
case CR:
case LF:
SET_ERRNO(HPE_INVALID_URL);
goto error;
default:
UPDATE_STATE(parse_url_char(CURRENT_STATE(), ch));
if (UNLIKELY(CURRENT_STATE() == s_dead)) {
SET_ERRNO(HPE_INVALID_URL);
goto error;
}
}
break;
}
case s_req_server:
case s_req_server_with_at:
case s_req_path:
case s_req_query_string_start:
case s_req_query_string:
case s_req_fragment_start:
case s_req_fragment:
{
switch (ch) {
case ' ':
UPDATE_STATE(s_req_http_start);
CALLBACK_DATA(url);
break;
case CR:
case LF:
parser->http_major = 0;
parser->http_minor = 9;
UPDATE_STATE((ch == CR) ?
s_req_line_almost_done :
s_header_field_start);
CALLBACK_DATA(url);
break;
default:
UPDATE_STATE(parse_url_char(CURRENT_STATE(), ch));
if (UNLIKELY(CURRENT_STATE() == s_dead)) {
SET_ERRNO(HPE_INVALID_URL);
goto error;
}
}
break;
}
case s_req_http_start:
switch (ch) {
case 'H':
UPDATE_STATE(s_req_http_H);
break;
case ' ':
break;
default:
SET_ERRNO(HPE_INVALID_CONSTANT);
goto error;
}
break;
case s_req_http_H:
STRICT_CHECK(ch != 'T');
UPDATE_STATE(s_req_http_HT);
break;
case s_req_http_HT:
STRICT_CHECK(ch != 'T');
UPDATE_STATE(s_req_http_HTT);
break;
case s_req_http_HTT:
STRICT_CHECK(ch != 'P');
UPDATE_STATE(s_req_http_HTTP);
break;
case s_req_http_HTTP:
STRICT_CHECK(ch != '/');
UPDATE_STATE(s_req_first_http_major);
break;
/* first digit of major HTTP version */
case s_req_first_http_major:
if (UNLIKELY(ch < '1' || ch > '9')) {
SET_ERRNO(HPE_INVALID_VERSION);
goto error;
}
parser->http_major = ch - '0';
UPDATE_STATE(s_req_http_major);
break;
/* major HTTP version or dot */
case s_req_http_major:
{
if (ch == '.') {
UPDATE_STATE(s_req_first_http_minor);
break;
}
if (UNLIKELY(!IS_NUM(ch))) {
SET_ERRNO(HPE_INVALID_VERSION);
goto error;
}
parser->http_major *= 10;
parser->http_major += ch - '0';
if (UNLIKELY(parser->http_major > 999)) {
SET_ERRNO(HPE_INVALID_VERSION);
goto error;
}
break;
}
/* first digit of minor HTTP version */
case s_req_first_http_minor:
if (UNLIKELY(!IS_NUM(ch))) {
SET_ERRNO(HPE_INVALID_VERSION);
goto error;
}
parser->http_minor = ch - '0';
UPDATE_STATE(s_req_http_minor);
break;
/* minor HTTP version or end of request line */
case s_req_http_minor:
{
if (ch == CR) {
UPDATE_STATE(s_req_line_almost_done);
break;
}
if (ch == LF) {
UPDATE_STATE(s_header_field_start);
break;
}
/* XXX allow spaces after digit? */
if (UNLIKELY(!IS_NUM(ch))) {
SET_ERRNO(HPE_INVALID_VERSION);
goto error;
}
parser->http_minor *= 10;
parser->http_minor += ch - '0';
if (UNLIKELY(parser->http_minor > 999)) {
SET_ERRNO(HPE_INVALID_VERSION);
goto error;
}
break;
}
/* end of request line */
case s_req_line_almost_done:
{
if (UNLIKELY(ch != LF)) {
SET_ERRNO(HPE_LF_EXPECTED);
goto error;
}
UPDATE_STATE(s_header_field_start);
break;
}
case s_header_field_start:
{
if (ch == CR) {
UPDATE_STATE(s_headers_almost_done);
break;
}
if (ch == LF) {
/* they might be just sending \n instead of \r\n so this would be
* the second \n to denote the end of headers*/
UPDATE_STATE(s_headers_almost_done);
REEXECUTE();
}
c = TOKEN(ch);
if (UNLIKELY(!c)) {
SET_ERRNO(HPE_INVALID_HEADER_TOKEN);
goto error;
}
MARK(header_field);
parser->index = 0;
UPDATE_STATE(s_header_field);
switch (c) {
case 'c':
parser->header_state = h_C;
break;
case 'p':
parser->header_state = h_matching_proxy_connection;
break;
case 't':
parser->header_state = h_matching_transfer_encoding;
break;
case 'u':
parser->header_state = h_matching_upgrade;
break;
default:
parser->header_state = h_general;
break;
}
break;
}
case s_header_field:
{
const char* start = p;
for (; p != data + len; p++) {
ch = *p;
c = TOKEN(ch);
if (!c)
break;
switch (parser->header_state) {
case h_general:
break;
case h_C:
parser->index++;
parser->header_state = (c == 'o' ? h_CO : h_general);
break;
case h_CO:
parser->index++;
parser->header_state = (c == 'n' ? h_CON : h_general);
break;
case h_CON:
parser->index++;
switch (c) {
case 'n':
parser->header_state = h_matching_connection;
break;
case 't':
parser->header_state = h_matching_content_length;
break;
default:
parser->header_state = h_general;
break;
}
break;
/* connection */
case h_matching_connection:
parser->index++;
if (parser->index > sizeof(CONNECTION)-1
|| c != CONNECTION[parser->index]) {
parser->header_state = h_general;
} else if (parser->index == sizeof(CONNECTION)-2) {
parser->header_state = h_connection;
}
break;
/* proxy-connection */
case h_matching_proxy_connection:
parser->index++;
if (parser->index > sizeof(PROXY_CONNECTION)-1
|| c != PROXY_CONNECTION[parser->index]) {
parser->header_state = h_general;
} else if (parser->index == sizeof(PROXY_CONNECTION)-2) {
parser->header_state = h_connection;
}
break;
/* content-length */
case h_matching_content_length:
parser->index++;
if (parser->index > sizeof(CONTENT_LENGTH)-1
|| c != CONTENT_LENGTH[parser->index]) {
parser->header_state = h_general;
} else if (parser->index == sizeof(CONTENT_LENGTH)-2) {
parser->header_state = h_content_length;
}
break;
/* transfer-encoding */
case h_matching_transfer_encoding:
parser->index++;
if (parser->index > sizeof(TRANSFER_ENCODING)-1
|| c != TRANSFER_ENCODING[parser->index]) {
parser->header_state = h_general;
} else if (parser->index == sizeof(TRANSFER_ENCODING)-2) {
parser->header_state = h_transfer_encoding;
}
break;
/* upgrade */
case h_matching_upgrade:
parser->index++;
if (parser->index > sizeof(UPGRADE)-1
|| c != UPGRADE[parser->index]) {
parser->header_state = h_general;
} else if (parser->index == sizeof(UPGRADE)-2) {
parser->header_state = h_upgrade;
}
break;
case h_connection:
case h_content_length:
case h_transfer_encoding:
case h_upgrade:
if (ch != ' ') parser->header_state = h_general;
break;
default:
assert(0 && "Unknown header_state");
break;
}
}
COUNT_HEADER_SIZE(p - start);
if (p == data + len) {
--p;
break;
}
if (ch == ':') {
UPDATE_STATE(s_header_value_discard_ws);
CALLBACK_DATA(header_field);
break;
}
SET_ERRNO(HPE_INVALID_HEADER_TOKEN);
goto error;
}
case s_header_value_discard_ws:
if (ch == ' ' || ch == '\t') break;
if (ch == CR) {
UPDATE_STATE(s_header_value_discard_ws_almost_done);
break;
}
if (ch == LF) {
UPDATE_STATE(s_header_value_discard_lws);
break;
}
/* FALLTHROUGH */
case s_header_value_start:
{
MARK(header_value);
UPDATE_STATE(s_header_value);
parser->index = 0;
c = LOWER(ch);
switch (parser->header_state) {
case h_upgrade:
parser->flags |= F_UPGRADE;
parser->header_state = h_general;
break;
case h_transfer_encoding:
/* looking for 'Transfer-Encoding: chunked' */
if ('c' == c) {
parser->header_state = h_matching_transfer_encoding_chunked;
} else {
parser->header_state = h_general;
}
break;
case h_content_length:
if (UNLIKELY(!IS_NUM(ch))) {
SET_ERRNO(HPE_INVALID_CONTENT_LENGTH);
goto error;
}
parser->content_length = ch - '0';
break;
case h_connection:
/* looking for 'Connection: keep-alive' */
if (c == 'k') {
parser->header_state = h_matching_connection_keep_alive;
/* looking for 'Connection: close' */
} else if (c == 'c') {
parser->header_state = h_matching_connection_close;
} else if (c == 'u') {
parser->header_state = h_matching_connection_upgrade;
} else {
parser->header_state = h_matching_connection_token;
}
break;
/* Multi-value `Connection` header */
case h_matching_connection_token_start:
break;
default:
parser->header_state = h_general;
break;
}
break;
}
case s_header_value:
{
const char* start = p;
enum header_states h_state = (enum header_states) parser->header_state;
for (; p != data + len; p++) {
ch = *p;
if (ch == CR) {
UPDATE_STATE(s_header_almost_done);
parser->header_state = h_state;
CALLBACK_DATA(header_value);
break;
}
if (ch == LF) {
UPDATE_STATE(s_header_almost_done);
COUNT_HEADER_SIZE(p - start);
parser->header_state = h_state;
CALLBACK_DATA_NOADVANCE(header_value);
REEXECUTE();
}
c = LOWER(ch);
switch (h_state) {
case h_general:
{
const char* p_cr;
const char* p_lf;
size_t limit = data + len - p;
limit = MIN(limit, HTTP_MAX_HEADER_SIZE);
p_cr = (const char*) memchr(p, CR, limit);
p_lf = (const char*) memchr(p, LF, limit);
if (p_cr != NULL) {
if (p_lf != NULL && p_cr >= p_lf)
p = p_lf;
else
p = p_cr;
} else if (UNLIKELY(p_lf != NULL)) {
p = p_lf;
} else {
p = data + len;
}
--p;
break;
}
case h_connection:
case h_transfer_encoding:
assert(0 && "Shouldn't get here.");
break;
case h_content_length:
{
uint64_t t;
if (ch == ' ') break;
if (UNLIKELY(!IS_NUM(ch))) {
SET_ERRNO(HPE_INVALID_CONTENT_LENGTH);
parser->header_state = h_state;
goto error;
}
t = parser->content_length;
t *= 10;
t += ch - '0';
/* Overflow? Test against a conservative limit for simplicity. */
if (UNLIKELY((ULLONG_MAX - 10) / 10 < parser->content_length)) {
SET_ERRNO(HPE_INVALID_CONTENT_LENGTH);
parser->header_state = h_state;
goto error;
}
parser->content_length = t;
break;
}
/* Transfer-Encoding: chunked */
case h_matching_transfer_encoding_chunked:
parser->index++;
if (parser->index > sizeof(CHUNKED)-1
|| c != CHUNKED[parser->index]) {
h_state = h_general;
} else if (parser->index == sizeof(CHUNKED)-2) {
h_state = h_transfer_encoding_chunked;
}
break;
case h_matching_connection_token_start:
/* looking for 'Connection: keep-alive' */
if (c == 'k') {
h_state = h_matching_connection_keep_alive;
/* looking for 'Connection: close' */
} else if (c == 'c') {
h_state = h_matching_connection_close;
} else if (c == 'u') {
h_state = h_matching_connection_upgrade;
} else if (STRICT_TOKEN(c)) {
h_state = h_matching_connection_token;
} else if (c == ' ' || c == '\t') {
/* Skip lws */
} else {
h_state = h_general;
}
break;
/* looking for 'Connection: keep-alive' */
case h_matching_connection_keep_alive:
parser->index++;
if (parser->index > sizeof(KEEP_ALIVE)-1
|| c != KEEP_ALIVE[parser->index]) {
h_state = h_matching_connection_token;
} else if (parser->index == sizeof(KEEP_ALIVE)-2) {
h_state = h_connection_keep_alive;
}
break;
/* looking for 'Connection: close' */
case h_matching_connection_close:
parser->index++;
if (parser->index > sizeof(CLOSE)-1 || c != CLOSE[parser->index]) {
h_state = h_matching_connection_token;
} else if (parser->index == sizeof(CLOSE)-2) {
h_state = h_connection_close;
}
break;
/* looking for 'Connection: upgrade' */
case h_matching_connection_upgrade:
parser->index++;
if (parser->index > sizeof(UPGRADE) - 1 ||
c != UPGRADE[parser->index]) {
h_state = h_matching_connection_token;
} else if (parser->index == sizeof(UPGRADE)-2) {
h_state = h_connection_upgrade;
}
break;
case h_matching_connection_token:
if (ch == ',') {
h_state = h_matching_connection_token_start;
parser->index = 0;
}
break;
case h_transfer_encoding_chunked:
if (ch != ' ') h_state = h_general;
break;
case h_connection_keep_alive:
case h_connection_close:
case h_connection_upgrade:
if (ch == ',') {
if (h_state == h_connection_keep_alive) {
parser->flags |= F_CONNECTION_KEEP_ALIVE;
} else if (h_state == h_connection_close) {
parser->flags |= F_CONNECTION_CLOSE;
} else if (h_state == h_connection_upgrade) {
parser->flags |= F_CONNECTION_UPGRADE;
}
h_state = h_matching_connection_token_start;
parser->index = 0;
} else if (ch != ' ') {
h_state = h_matching_connection_token;
}
break;
default:
UPDATE_STATE(s_header_value);
h_state = h_general;
break;
}
}
parser->header_state = h_state;
COUNT_HEADER_SIZE(p - start);
if (p == data + len)
--p;
break;
}
case s_header_almost_done:
{
STRICT_CHECK(ch != LF);
UPDATE_STATE(s_header_value_lws);
break;
}
case s_header_value_lws:
{
if (ch == ' ' || ch == '\t') {
UPDATE_STATE(s_header_value_start);
REEXECUTE();
}
/* finished the header */
switch (parser->header_state) {
case h_connection_keep_alive:
parser->flags |= F_CONNECTION_KEEP_ALIVE;
break;
case h_connection_close:
parser->flags |= F_CONNECTION_CLOSE;
break;
case h_transfer_encoding_chunked:
parser->flags |= F_CHUNKED;
break;
case h_connection_upgrade:
parser->flags |= F_CONNECTION_UPGRADE;
break;
default:
break;
}
UPDATE_STATE(s_header_field_start);
REEXECUTE();
}
case s_header_value_discard_ws_almost_done:
{
STRICT_CHECK(ch != LF);
UPDATE_STATE(s_header_value_discard_lws);
break;
}
case s_header_value_discard_lws:
{
if (ch == ' ' || ch == '\t') {
UPDATE_STATE(s_header_value_discard_ws);
break;
} else {
switch (parser->header_state) {
case h_connection_keep_alive:
parser->flags |= F_CONNECTION_KEEP_ALIVE;
break;
case h_connection_close:
parser->flags |= F_CONNECTION_CLOSE;
break;
case h_connection_upgrade:
parser->flags |= F_CONNECTION_UPGRADE;
break;
case h_transfer_encoding_chunked:
parser->flags |= F_CHUNKED;
break;
default:
break;
}
/* header value was empty */
MARK(header_value);
UPDATE_STATE(s_header_field_start);
CALLBACK_DATA_NOADVANCE(header_value);
REEXECUTE();
}
}
case s_headers_almost_done:
{
STRICT_CHECK(ch != LF);
if (parser->flags & F_TRAILING) {
/* End of a chunked request */
UPDATE_STATE(NEW_MESSAGE());
CALLBACK_NOTIFY(message_complete);
break;
}
UPDATE_STATE(s_headers_done);
/* Set this here so that on_headers_complete() callbacks can see it */
parser->upgrade =
((parser->flags & (F_UPGRADE | F_CONNECTION_UPGRADE)) ==
(F_UPGRADE | F_CONNECTION_UPGRADE) ||
parser->method == HTTP_CONNECT);
/* Here we call the headers_complete callback. This is somewhat
* different than other callbacks because if the user returns 1, we
* will interpret that as saying that this message has no body. This
* is needed for the annoying case of recieving a response to a HEAD
* request.
*
* We'd like to use CALLBACK_NOTIFY_NOADVANCE() here but we cannot, so
* we have to simulate it by handling a change in errno below.
*/
if (settings->on_headers_complete) {
switch (settings->on_headers_complete(parser)) {
case 0:
break;
case 1:
parser->flags |= F_SKIPBODY;
break;
default:
SET_ERRNO(HPE_CB_headers_complete);
RETURN(p - data); /* Error */
}
}
if (HTTP_PARSER_ERRNO(parser) != HPE_OK) {
RETURN(p - data);
}
REEXECUTE();
}
case s_headers_done:
{
STRICT_CHECK(ch != LF);
parser->nread = 0;
/* Exit, the rest of the connect is in a different protocol. */
if (parser->upgrade) {
UPDATE_STATE(NEW_MESSAGE());
CALLBACK_NOTIFY(message_complete);
RETURN((p - data) + 1);
}
if (parser->flags & F_SKIPBODY) {
UPDATE_STATE(NEW_MESSAGE());
CALLBACK_NOTIFY(message_complete);
} else if (parser->flags & F_CHUNKED) {
/* chunked encoding - ignore Content-Length header */
UPDATE_STATE(s_chunk_size_start);
} else {
if (parser->content_length == 0) {
/* Content-Length header given but zero: Content-Length: 0\r\n */
UPDATE_STATE(NEW_MESSAGE());
CALLBACK_NOTIFY(message_complete);
} else if (parser->content_length != ULLONG_MAX) {
/* Content-Length header given and non-zero */
UPDATE_STATE(s_body_identity);
} else {
if (parser->type == HTTP_REQUEST ||
!http_message_needs_eof(parser)) {
/* Assume content-length 0 - read the next */
UPDATE_STATE(NEW_MESSAGE());
CALLBACK_NOTIFY(message_complete);
} else {
/* Read body until EOF */
UPDATE_STATE(s_body_identity_eof);
}
}
}
break;
}
case s_body_identity:
{
uint64_t to_read = MIN(parser->content_length,
(uint64_t) ((data + len) - p));
assert(parser->content_length != 0
&& parser->content_length != ULLONG_MAX);
/* The difference between advancing content_length and p is because
* the latter will automaticaly advance on the next loop iteration.
* Further, if content_length ends up at 0, we want to see the last
* byte again for our message complete callback.
*/
MARK(body);
parser->content_length -= to_read;
p += to_read - 1;
if (parser->content_length == 0) {
UPDATE_STATE(s_message_done);
/* Mimic CALLBACK_DATA_NOADVANCE() but with one extra byte.
*
* The alternative to doing this is to wait for the next byte to
* trigger the data callback, just as in every other case. The
* problem with this is that this makes it difficult for the test
* harness to distinguish between complete-on-EOF and
* complete-on-length. It's not clear that this distinction is
* important for applications, but let's keep it for now.
*/
CALLBACK_DATA_(body, p - body_mark + 1, p - data);
REEXECUTE();
}
break;
}
/* read until EOF */
case s_body_identity_eof:
MARK(body);
p = data + len - 1;
break;
case s_message_done:
UPDATE_STATE(NEW_MESSAGE());
CALLBACK_NOTIFY(message_complete);
break;
case s_chunk_size_start:
{
assert(parser->nread == 1);
assert(parser->flags & F_CHUNKED);
unhex_val = unhex[(unsigned char)ch];
if (UNLIKELY(unhex_val == -1)) {
SET_ERRNO(HPE_INVALID_CHUNK_SIZE);
goto error;
}
parser->content_length = unhex_val;
UPDATE_STATE(s_chunk_size);
break;
}
case s_chunk_size:
{
uint64_t t;
assert(parser->flags & F_CHUNKED);
if (ch == CR) {
UPDATE_STATE(s_chunk_size_almost_done);
break;
}
unhex_val = unhex[(unsigned char)ch];
if (unhex_val == -1) {
if (ch == ';' || ch == ' ') {
UPDATE_STATE(s_chunk_parameters);
break;
}
SET_ERRNO(HPE_INVALID_CHUNK_SIZE);
goto error;
}
t = parser->content_length;
t *= 16;
t += unhex_val;
/* Overflow? Test against a conservative limit for simplicity. */
if (UNLIKELY((ULLONG_MAX - 16) / 16 < parser->content_length)) {
SET_ERRNO(HPE_INVALID_CONTENT_LENGTH);
goto error;
}
parser->content_length = t;
break;
}
case s_chunk_parameters:
{
assert(parser->flags & F_CHUNKED);
/* just ignore this shit. TODO check for overflow */
if (ch == CR) {
UPDATE_STATE(s_chunk_size_almost_done);
break;
}
break;
}
case s_chunk_size_almost_done:
{
assert(parser->flags & F_CHUNKED);
STRICT_CHECK(ch != LF);
parser->nread = 0;
if (parser->content_length == 0) {
parser->flags |= F_TRAILING;
UPDATE_STATE(s_header_field_start);
} else {
UPDATE_STATE(s_chunk_data);
}
break;
}
case s_chunk_data:
{
uint64_t to_read = MIN(parser->content_length,
(uint64_t) ((data + len) - p));
assert(parser->flags & F_CHUNKED);
assert(parser->content_length != 0
&& parser->content_length != ULLONG_MAX);
/* See the explanation in s_body_identity for why the content
* length and data pointers are managed this way.
*/
MARK(body);
parser->content_length -= to_read;
p += to_read - 1;
if (parser->content_length == 0) {
UPDATE_STATE(s_chunk_data_almost_done);
}
break;
}
case s_chunk_data_almost_done:
assert(parser->flags & F_CHUNKED);
assert(parser->content_length == 0);
STRICT_CHECK(ch != CR);
UPDATE_STATE(s_chunk_data_done);
CALLBACK_DATA(body);
break;
case s_chunk_data_done:
assert(parser->flags & F_CHUNKED);
STRICT_CHECK(ch != LF);
parser->nread = 0;
UPDATE_STATE(s_chunk_size_start);
break;
default:
assert(0 && "unhandled state");
SET_ERRNO(HPE_INVALID_INTERNAL_STATE);
goto error;
}
}
/* Run callbacks for any marks that we have leftover after we ran our of
* bytes. There should be at most one of these set, so it's OK to invoke
* them in series (unset marks will not result in callbacks).
*
* We use the NOADVANCE() variety of callbacks here because 'p' has already
* overflowed 'data' and this allows us to correct for the off-by-one that
* we'd otherwise have (since CALLBACK_DATA() is meant to be run with a 'p'
* value that's in-bounds).
*/
assert(((header_field_mark ? 1 : 0) +
(header_value_mark ? 1 : 0) +
(url_mark ? 1 : 0) +
(body_mark ? 1 : 0) +
(status_mark ? 1 : 0)) <= 1);
CALLBACK_DATA_NOADVANCE(header_field);
CALLBACK_DATA_NOADVANCE(header_value);
CALLBACK_DATA_NOADVANCE(url);
CALLBACK_DATA_NOADVANCE(body);
CALLBACK_DATA_NOADVANCE(status);
RETURN(len);
error:
if (HTTP_PARSER_ERRNO(parser) == HPE_OK) {
SET_ERRNO(HPE_UNKNOWN);
}
RETURN(p - data);
}
/* Does the parser need to see an EOF to find the end of the message? */
int
http_message_needs_eof (const http_parser *parser)
{
if (parser->type == HTTP_REQUEST) {
return 0;
}
/* See RFC 2616 section 4.4 */
if (parser->status_code / 100 == 1 || /* 1xx e.g. Continue */
parser->status_code == 204 || /* No Content */
parser->status_code == 304 || /* Not Modified */
parser->flags & F_SKIPBODY) { /* response to a HEAD request */
return 0;
}
if ((parser->flags & F_CHUNKED) || parser->content_length != ULLONG_MAX) {
return 0;
}
return 1;
}
int
http_should_keep_alive (const http_parser *parser)
{
if (parser->http_major > 0 && parser->http_minor > 0) {
/* HTTP/1.1 */
if (parser->flags & F_CONNECTION_CLOSE) {
return 0;
}
} else {
/* HTTP/1.0 or earlier */
if (!(parser->flags & F_CONNECTION_KEEP_ALIVE)) {
return 0;
}
}
return !http_message_needs_eof(parser);
}
const char *
http_method_str (enum http_method m)
{
return ELEM_AT(method_strings, m, "<unknown>");
}
void
http_parser_init (http_parser *parser, enum http_parser_type t)
{
void *data = parser->data; /* preserve application data */
memset(parser, 0, sizeof(*parser));
parser->data = data;
parser->type = t;
parser->state = (t == HTTP_REQUEST ? s_start_req : (t == HTTP_RESPONSE ? s_start_res : s_start_req_or_res));
parser->http_errno = HPE_OK;
}
void
http_parser_settings_init(http_parser_settings *settings)
{
memset(settings, 0, sizeof(*settings));
}
const char *
http_errno_name(enum http_errno err) {
assert(err < (sizeof(http_strerror_tab)/sizeof(http_strerror_tab[0])));
return http_strerror_tab[err].name;
}
const char *
http_errno_description(enum http_errno err) {
assert(err < (sizeof(http_strerror_tab)/sizeof(http_strerror_tab[0])));
return http_strerror_tab[err].description;
}
static enum http_host_state
http_parse_host_char(enum http_host_state s, const char ch) {
switch(s) {
case s_http_userinfo:
case s_http_userinfo_start:
if (ch == '@') {
return s_http_host_start;
}
if (IS_USERINFO_CHAR(ch)) {
return s_http_userinfo;
}
break;
case s_http_host_start:
if (ch == '[') {
return s_http_host_v6_start;
}
if (IS_HOST_CHAR(ch)) {
return s_http_host;
}
break;
case s_http_host:
if (IS_HOST_CHAR(ch)) {
return s_http_host;
}
/* FALLTHROUGH */
case s_http_host_v6_end:
if (ch == ':') {
return s_http_host_port_start;
}
break;
case s_http_host_v6:
if (ch == ']') {
return s_http_host_v6_end;
}
/* FALLTHROUGH */
case s_http_host_v6_start:
if (IS_HEX(ch) || ch == ':' || ch == '.') {
return s_http_host_v6;
}
break;
case s_http_host_port:
case s_http_host_port_start:
if (IS_NUM(ch)) {
return s_http_host_port;
}
break;
default:
break;
}
return s_http_host_dead;
}
static int
http_parse_host(const char * buf, struct http_parser_url *u, int found_at) {
enum http_host_state s;
const char *p;
size_t buflen = u->field_data[UF_HOST].off + u->field_data[UF_HOST].len;
u->field_data[UF_HOST].len = 0;
s = found_at ? s_http_userinfo_start : s_http_host_start;
for (p = buf + u->field_data[UF_HOST].off; p < buf + buflen; p++) {
enum http_host_state new_s = http_parse_host_char(s, *p);
if (new_s == s_http_host_dead) {
return 1;
}
switch(new_s) {
case s_http_host:
if (s != s_http_host) {
u->field_data[UF_HOST].off = p - buf;
}
u->field_data[UF_HOST].len++;
break;
case s_http_host_v6:
if (s != s_http_host_v6) {
u->field_data[UF_HOST].off = p - buf;
}
u->field_data[UF_HOST].len++;
break;
case s_http_host_port:
if (s != s_http_host_port) {
u->field_data[UF_PORT].off = p - buf;
u->field_data[UF_PORT].len = 0;
u->field_set |= (1 << UF_PORT);
}
u->field_data[UF_PORT].len++;
break;
case s_http_userinfo:
if (s != s_http_userinfo) {
u->field_data[UF_USERINFO].off = p - buf ;
u->field_data[UF_USERINFO].len = 0;
u->field_set |= (1 << UF_USERINFO);
}
u->field_data[UF_USERINFO].len++;
break;
default:
break;
}
s = new_s;
}
/* Make sure we don't end somewhere unexpected */
switch (s) {
case s_http_host_start:
case s_http_host_v6_start:
case s_http_host_v6:
case s_http_host_port_start:
case s_http_userinfo:
case s_http_userinfo_start:
return 1;
default:
break;
}
return 0;
}
int
http_parser_parse_url(const char *buf, size_t buflen, int is_connect,
struct http_parser_url *u)
{
enum state s;
const char *p;
enum http_parser_url_fields uf, old_uf;
int found_at = 0;
u->port = u->field_set = 0;
s = is_connect ? s_req_server_start : s_req_spaces_before_url;
old_uf = UF_MAX;
for (p = buf; p < buf + buflen; p++) {
s = parse_url_char(s, *p);
/* Figure out the next field that we're operating on */
switch (s) {
case s_dead:
return 1;
/* Skip delimeters */
case s_req_schema_slash:
case s_req_schema_slash_slash:
case s_req_server_start:
case s_req_query_string_start:
case s_req_fragment_start:
continue;
case s_req_schema:
uf = UF_SCHEMA;
break;
case s_req_server_with_at:
found_at = 1;
/* FALLTHRU */
case s_req_server:
uf = UF_HOST;
break;
case s_req_path:
uf = UF_PATH;
break;
case s_req_query_string:
uf = UF_QUERY;
break;
case s_req_fragment:
uf = UF_FRAGMENT;
break;
default:
assert(!"Unexpected state");
return 1;
}
/* Nothing's changed; soldier on */
if (uf == old_uf) {
u->field_data[uf].len++;
continue;
}
u->field_data[uf].off = p - buf;
u->field_data[uf].len = 1;
u->field_set |= (1 << uf);
old_uf = uf;
}
/* host must be present if there is a schema */
/* parsing http:///toto will fail */
if ((u->field_set & ((1 << UF_SCHEMA) | (1 << UF_HOST))) != 0) {
if (http_parse_host(buf, u, found_at) != 0) {
return 1;
}
}
/* CONNECT requests can only contain "hostname:port" */
if (is_connect && u->field_set != ((1 << UF_HOST)|(1 << UF_PORT))) {
return 1;
}
if (u->field_set & (1 << UF_PORT)) {
/* Don't bother with endp; we've already validated the string */
unsigned long v = strtoul(buf + u->field_data[UF_PORT].off, NULL, 10);
/* Ports have a max value of 2^16 */
if (v > 0xffff) {
return 1;
}
u->port = (uint16_t) v;
}
return 0;
}
void
http_parser_pause(http_parser *parser, int paused) {
/* Users should only be pausing/unpausing a parser that is not in an error
* state. In non-debug builds, there's not much that we can do about this
* other than ignore it.
*/
if (HTTP_PARSER_ERRNO(parser) == HPE_OK ||
HTTP_PARSER_ERRNO(parser) == HPE_PAUSED) {
SET_ERRNO((paused) ? HPE_PAUSED : HPE_OK);
} else {
assert(0 && "Attempting to pause parser in error state");
}
}
int
http_body_is_final(const struct http_parser *parser) {
return parser->state == s_message_done;
}
unsigned long
http_parser_version(void) {
return HTTP_PARSER_VERSION_MAJOR * 0x10000 |
HTTP_PARSER_VERSION_MINOR * 0x00100 |
HTTP_PARSER_VERSION_PATCH * 0x00001;
}
#endif
================================================
FILE: src/ngx_stream_http_parser.h
================================================
/* Copyright Joyent, Inc. and other Node contributors. All rights reserved.
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to
* deal in the Software without restriction, including without limitation the
* rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
* sell copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
* IN THE SOFTWARE.
*/
#ifndef http_parser_h
#define http_parser_h
#ifdef __cplusplus
extern "C" {
#endif
/* Also update SONAME in the Makefile whenever you change these. */
#define HTTP_PARSER_VERSION_MAJOR 2
#define HTTP_PARSER_VERSION_MINOR 4
#define HTTP_PARSER_VERSION_PATCH 2
#include <sys/types.h>
#if defined(_WIN32) && !defined(__MINGW32__) && (!defined(_MSC_VER) || _MSC_VER<1600)
#include <BaseTsd.h>
#include <stddef.h>
typedef __int8 int8_t;
typedef unsigned __int8 uint8_t;
typedef __int16 int16_t;
typedef unsigned __int16 uint16_t;
typedef __int32 int32_t;
typedef unsigned __int32 uint32_t;
typedef __int64 int64_t;
typedef unsigned __int64 uint64_t;
#else
#include <stdint.h>
#endif
/* Compile with -DHTTP_PARSER_STRICT=0 to make less checks, but run
* faster
*/
#ifndef HTTP_PARSER_STRICT
# define HTTP_PARSER_STRICT 1
#endif
/* Maximium header size allowed. If the macro is not defined
* before including this header then the default is used. To
* change the maximum header size, define the macro in the build
* environment (e.g. -DHTTP_MAX_HEADER_SIZE=<value>). To remove
* the effective limit on the size of the header, define the macro
* to a very large number (e.g. -DHTTP_MAX_HEADER_SIZE=0x7fffffff)
*/
#ifndef HTTP_MAX_HEADER_SIZE
# define HTTP_MAX_HEADER_SIZE (80*1024)
#endif
typedef struct http_parser http_parser;
typedef struct http_parser_settings http_parser_settings;
/* Callbacks should return non-zero to indicate an error. The parser will
* then halt execution.
*
* The one exception is on_headers_complete. In a HTTP_RESPONSE parser
* returning '1' from on_headers_complete will tell the parser that it
* should not expect a body. This is used when receiving a response to a
* HEAD request which may contain 'Content-Length' or 'Transfer-Encoding:
* chunked' headers that indicate the presence of a body.
*
* http_data_cb does not return data chunks. It will be called arbitrarily
* many times for each string. E.G. you might get 10 callbacks for "on_url"
* each providing just a few characters more data.
*/
typedef int (*http_data_cb) (http_parser*, const char *at, size_t length);
typedef int (*http_cb) (http_parser*);
/* Request Methods */
#define HTTP_METHOD_MAP(XX) \
XX(0, DELETE, DELETE) \
XX(1, GET, GET) \
XX(2, HEAD, HEAD) \
XX(3, POST, POST) \
XX(4, PUT, PUT) \
/* pathological */ \
XX(5, CONNECT, CONNECT) \
XX(6, OPTIONS, OPTIONS) \
XX(7, TRACE, TRACE) \
/* webdav */ \
XX(8, COPY, COPY) \
XX(9, LOCK, LOCK) \
XX(10, MKCOL, MKCOL) \
XX(11, MOVE, MOVE) \
XX(12, PROPFIND, PROPFIND) \
XX(13, PROPPATCH, PROPPATCH) \
XX(14, SEARCH, SEARCH) \
XX(15, UNLOCK, UNLOCK) \
/* subversion */ \
XX(16, REPORT, REPORT) \
XX(17, MKACTIVITY, MKACTIVITY) \
XX(18, CHECKOUT, CHECKOUT) \
XX(19, MERGE, MERGE) \
/* upnp */ \
XX(20, MSEARCH, M-SEARCH) \
XX(21, NOTIFY, NOTIFY) \
XX(22, SUBSCRIBE, SUBSCRIBE) \
XX(23, UNSUBSCRIBE, UNSUBSCRIBE) \
/* RFC-5789 */ \
XX(24, PATCH, PATCH) \
XX(25, PURGE, PURGE) \
/* CalDAV */ \
XX(26, MKCALENDAR, MKCALENDAR) \
enum http_method
{
#define XX(num, name, string) HTTP_##name = num,
HTTP_METHOD_MAP(XX)
#undef XX
};
enum http_parser_type { HTTP_REQUEST, HTTP_RESPONSE, HTTP_BOTH };
/* Flag values for http_parser.flags field */
enum flags
{ F_CHUNKED = 1 << 0
, F_CONNECTION_KEEP_ALIVE = 1 << 1
, F_CONNECTION_CLOSE = 1 << 2
, F_CONNECTION_UPGRADE = 1 << 3
, F_TRAILING = 1 << 4
, F_UPGRADE = 1 << 5
, F_SKIPBODY = 1 << 6
};
/* Map for errno-related constants
*
* The provided argument should be a macro that takes 2 arguments.
*/
#define HTTP_ERRNO_MAP(XX) \
/* No error */ \
XX(OK, "success") \
\
/* Callback-related errors */ \
XX(CB_message_begin, "the on_message_begin callback failed") \
XX(CB_url, "the on_url callback failed") \
XX(CB_header_field, "the on_header_field callback failed") \
XX(CB_header_value, "the on_header_value callback failed") \
XX(CB_headers_complete, "the on_headers_complete callback failed") \
XX(CB_body, "the on_body callback failed") \
XX(CB_message_complete, "the on_message_complete callback failed") \
XX(CB_status, "the on_status callback failed") \
\
/* Parsing-related errors */ \
XX(INVALID_EOF_STATE, "stream ended at an unexpected time") \
XX(HEADER_OVERFLOW, \
"too many header bytes seen; overflow detected") \
XX(CLOSED_CONNECTION, \
"data received after completed connection: close message") \
XX(INVALID_VERSION, "invalid HTTP version") \
XX(INVALID_STATUS, "invalid HTTP status code") \
XX(INVALID_METHOD, "invalid HTTP method") \
XX(INVALID_URL, "invalid URL") \
XX(INVALID_HOST, "invalid host") \
XX(INVALID_PORT, "invalid port") \
XX(INVALID_PATH, "invalid path") \
XX(INVALID_QUERY_STRING, "invalid query string") \
XX(INVALID_FRAGMENT, "invalid fragment") \
XX(LF_EXPECTED, "LF character expected") \
XX(INVALID_HEADER_TOKEN, "invalid character in header") \
XX(INVALID_CONTENT_LENGTH, \
"invalid character in content-length header") \
XX(INVALID_CHUNK_SIZE, \
"invalid character in chunk size header") \
XX(INVALID_CONSTANT, "invalid constant string") \
XX(INVALID_INTERNAL_STATE, "encountered unexpected internal state")\
XX(STRICT, "strict mode assertion failed") \
XX(PAUSED, "parser is paused") \
XX(UNKNOWN, "an unknown error occurred")
/* Define HPE_* values for each errno value above */
#define HTTP_ERRNO_GEN(n, s) HPE_##n,
enum http_errno {
HTTP_ERRNO_MAP(HTTP_ERRNO_GEN)
};
#undef HTTP_ERRNO_GEN
/* Get an http_errno value from an http_parser */
#define HTTP_PARSER_ERRNO(p) ((enum http_errno) (p)->http_errno)
struct http_parser {
/** PRIVATE **/
unsigned int type : 2; /* enum http_parser_type */
unsigned int flags : 7; /* F_* values from 'flags' enum; semi-public */
unsigned int state : 7; /* enum state from http_parser.c */
unsigned int header_state : 8; /* enum header_state from http_parser.c */
unsigned int index : 8; /* index into current matcher */
uint32_t nread; /* # bytes read in various scenarios */
uint64_t content_length; /* # bytes in body (0 if no Content-Length header) */
/** READ-ONLY **/
unsigned short http_major;
unsigned short http_minor;
unsigned int status_code : 16; /* responses only */
unsigned int method : 8; /* requests only */
unsigned int http_errno : 7;
/* 1 = Upgrade header was present and the parser has exited because of that.
* 0 = No upgrade header present.
* Should be checked when http_parser_execute() returns in addition to
* error checking.
*/
unsigned int upgrade : 1;
/** PUBLIC **/
void *data; /* A pointer to get hook to the "connection" or "socket" object */
};
struct http_parser_settings {
http_cb on_message_begin;
http_data_cb on_url;
http_data_cb on_status;
http_data_cb on_header_field;
http_data_cb on_header_value;
http_cb on_headers_complete;
http_data_cb on_body;
http_cb on_message_complete;
};
enum http_parser_url_fields
{ UF_SCHEMA = 0
, UF_HOST = 1
, UF_PORT = 2
, UF_PATH = 3
, UF_QUERY = 4
, UF_FRAGMENT = 5
, UF_USERINFO = 6
, UF_MAX = 7
};
/* Result structure for http_parser_parse_url().
*
* Callers should index into field_data[] with UF_* values iff field_set
* has the relevant (1 << UF_*) bit set. As a courtesy to clients (and
* because we probably have padding left over), we convert any port to
* a uint16_t.
*/
struct http_parser_url {
uint16_t field_set; /* Bitmask of (1 << UF_*) values */
uint16_t port; /* Converted UF_PORT string */
struct {
uint16_t off; /* Offset into buffer in which field starts */
uint16_t len; /* Length of run in buffer */
} field_data[UF_MAX];
};
/* Returns the library version. Bits 16-23 contain the major version number,
* bits 8-15 the minor version number and bits 0-7 the patch level.
* Usage example:
*
* unsigned long version = http_parser_version();
* unsigned major = (version >> 16) & 255;
* unsigned minor = (version >> 8) & 255;
* unsigned patch = version & 255;
* printf("http_parser v%u.%u.%u\n", major, minor, patch);
*/
unsigned long http_parser_version(void);
void http_parser_init(http_parser *parser, enum http_parser_type type);
/* Initialize http_parser_settings members to 0
*/
void http_parser_settings_init(http_parser_settings *settings);
/* Executes the parser. Returns number of parsed bytes. Sets
* `parser->http_errno` on error. */
size_t http_parser_execute(http_parser *parser,
const http_parser_settings *settings,
const char *data,
size_t len);
/* If http_should_keep_alive() in the on_headers_complete or
* on_message_complete callback returns 0, then this should be
* the last message on the connection.
* If you are the server, respond with the "Connection: close" header.
* If you are the client, close the connection.
*/
int http_should_keep_alive(const http_parser *parser);
/* Returns a string version of the HTTP method. */
const char *http_method_str(enum http_method m);
/* Return a string name of the given error */
const char *http_errno_name(enum http_errno err);
/* Return a string description of the given error */
const char *http_errno_description(enum http_errno err);
/* Parse a URL; return nonzero on failure */
int http_parser_parse_url(const char *buf, size_t buflen,
int is_connect,
struct http_parser_url *u);
/* Pause or un-pause the parser; a nonzero value pauses */
void http_parser_pause(http_parser *parser, int paused);
/* Checks if this is the final chunk of the body. */
int http_body_is_final(const http_parser *parser);
#ifdef __cplusplus
}
#endif
#endif
================================================
FILE: src/ngx_stream_json.c
================================================
/*
Copyright (c) 2009 Dave Gamble
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
*/
/* cJSON */
/* JSON parser in C. */
#include <ngx_config.h>
#ifndef NGX_HTTP_UPSYNC
#include <string.h>
#include <stdio.h>
#include <math.h>
#include <stdlib.h>
#include <float.h>
#include <limits.h>
#include <ctype.h>
#include "ngx_stream_json.h"
static const char *ep;
const char *cJSON_GetErrorPtr(void) {return ep;}
static int cJSON_strcasecmp(const char *s1,const char *s2)
{
if (!s1) return (s1==s2)?0:1;
if (!s2) return 1;
for(; tolower(*s1) == tolower(*s2); ++s1, ++s2) if(*s1 == 0) return 0;
return tolower(*(const unsigned char *)s1) - tolower(*(const unsigned char *)s2);
}
static void *(*cJSON_malloc)(size_t sz) = malloc;
static void (*cJSON_free)(void *ptr) = free;
static char* cJSON_strdup(const char* str)
{
size_t len;
char* copy;
len = strlen(str) + 1;
if (!(copy = (char*)cJSON_malloc(len))) return 0;
memcpy(copy,str,len);
return copy;
}
void cJSON_InitHooks(cJSON_Hooks* hooks)
{
if (!hooks) { /* Reset hooks */
cJSON_malloc = malloc;
cJSON_free = free;
return;
}
cJSON_malloc = (hooks->malloc_fn)?hooks->malloc_fn:malloc;
cJSON_free = (hooks->free_fn)?hooks->free_fn:free;
}
/* Internal constructor. */
static cJSON *cJSON_New_Item(void)
{
cJSON* node = (cJSON*)cJSON_malloc(sizeof(cJSON));
if (node) memset(node,0,sizeof(cJSON));
return node;
}
/* Delete a cJSON structure. */
void cJSON_Delete(cJSON *c)
{
cJSON *next;
while (c)
{
next=c->next;
if (!(c->type&cJSON_IsReference) && c->child) cJSON_Delete(c->child);
if (!(c->type&cJSON_IsReference) && c->valuestring) cJSON_free(c->valuestring);
if (c->string) cJSON_free(c->string);
cJSON_free(c);
c=next;
}
}
/* Parse the input text to generate a number, and populate the result into item. */
static const char *parse_number(cJSON *item,const char *num)
{
double n=0,sign=1,scale=0;int subscale=0,signsubscale=1;
/* Could use sscanf for this? */
if (*num=='-') sign=-1,num++; /* Has sign? */
if (*num=='0') num++; /* is zero */
if (*num>='1' && *num<='9') do n=(n*10.0)+(*num++ -'0'); while (*num>='0' && *num<='9'); /* Number? */
if (*num=='.' && num[1]>='0' && num[1]<='9') {num++; do n=(n*10.0)+(*num++ -'0'),scale--; while (*num>='0' && *num<='9');} /* Fractional part? */
if (*num=='e' || *num=='E') /* Exponent? */
{ num++;if (*num=='+') num++; else if (*num=='-') signsubscale=-1,num++; /* With sign? */
while (*num>='0' && *num<='9') subscale=(subscale*10)+(*num++ - '0'); /* Number? */
}
n=sign*n*pow(10.0,(scale+subscale*signsubscale)); /* number = +/- number.fraction * 10^+/- exponent */
item->valuedouble=n;
item->valueint=(int)n;
item->type=cJSON_Number;
return num;
}
/* Render the number nicely from the given item into a string. */
static char *print_number(cJSON *item)
{
char *str;
double d=item->valuedouble;
if (fabs(((double)item->valueint)-d)<=DBL_EPSILON && d<=INT_MAX && d>=INT_MIN)
{
str=(char*)cJSON_malloc(21); /* 2^64+1 can be represented in 21 chars. */
if (str) sprintf(str,"%d",item->valueint);
}
else
{
str=(char*)cJSON_malloc(64); /* This is a nice tradeoff. */
if (str)
{
if (fabs(floor(d)-d)<=DBL_EPSILON && fabs(d)<1.0e60)sprintf(str,"%.0f",d);
else if (fabs(d)<1.0e-6 || fabs(d)>1.0e9) sprintf(str,"%e",d);
else sprintf(str,"%f",d);
}
}
return str;
}
/* Parse the input text into an unescaped cstring, and populate item. */
static const unsigned char firstByteMark[7] = { 0x00, 0x00, 0xC0, 0xE0, 0xF0, 0xF8, 0xFC };
static const char *parse_string(cJSON *item,const char *str)
{
const char *ptr=str+1;char *ptr2;char *out;int len=0;unsigned uc,uc2;
if (*str!='\"') {ep=str;return 0;} /* not a string! */
while (*ptr!='\"' && *ptr && ++len) if (*ptr++ == '\\') ptr++; /* Skip escaped quotes. */
out=(char*)cJSON_malloc(len+1); /* This is how long we need for the string, roughly. */
if (!out) return 0;
ptr=str+1;ptr2=out;
while (*ptr!='\"' && *ptr)
{
if (*ptr!='\\') *ptr2++=*ptr++;
else
{
ptr++;
switch (*ptr)
{
case 'b': *ptr2++='\b'; break;
case 'f': *ptr2++='\f'; break;
case 'n': *ptr2++='\n'; break;
case 'r': *ptr2++='\r'; break;
case 't': *ptr2++='\t'; break;
case 'u': /* transcode utf16 to utf8. */
sscanf(ptr+1,"%4x",&uc);ptr+=4; /* get the unicode char. */
if ((uc>=0xDC00 && uc<=0xDFFF) || uc==0) break; /* check for invalid. */
if (uc>=0xD800 && uc<=0xDBFF) /* UTF16 surrogate pairs. */
{
if (ptr[1]!='\\' || ptr[2]!='u') break; /* missing second-half of surrogate. */
sscanf(ptr+3,"%4x",&uc2);ptr+=6;
if (uc2<0xDC00 || uc2>0xDFFF) break; /* invalid second-half of surrogate. */
uc=0x10000 + (((uc&0x3FF)<<10) | (uc2&0x3FF));
}
len=4;if (uc<0x80) len=1;else if (uc<0x800) len=2;else if (uc<0x10000) len=3; ptr2+=len;
switch (len) {
case 4: *--ptr2 =((uc | 0x80) & 0xBF); uc >>= 6; /* FALLTHRU */
case 3: *--ptr2 =((uc | 0x80) & 0xBF); uc >>= 6; /* FALLTHRU */
case 2: *--ptr2 =((uc | 0x80) & 0xBF); uc >>= 6; /* FALLTHRU */
case 1: *--ptr2 =(uc | firstByteMark[len]);
}
ptr2+=len;
break;
default: *ptr2++=*ptr; break;
}
ptr++;
}
}
*ptr2=0;
if (*ptr=='\"') ptr++;
item->valuestring=out;
item->type=cJSON_String;
return ptr;
}
/* Render the cstring provided to an escaped version that can be printed. */
static char *print_string_ptr(const char *str)
{
const char *ptr;char *ptr2,*out;int len=0;unsigned char token;
if (!str) return cJSON_strdup("");
ptr=str;while ((token=*ptr) && ++len) {if (strchr("\"\\\b\f\n\r\t",token)) len++; else if (token<32) len+=5;ptr++;}
out=(char*)cJSON_malloc(len+3);
if (!out) return 0;
ptr2=out;ptr=str;
*ptr2++='\"';
while (*ptr)
{
if ((unsigned char)*ptr>31 && *ptr!='\"' && *ptr!='\\') *ptr2++=*ptr++;
else
{
*ptr2++='\\';
switch (token=*ptr++)
{
case '\\': *ptr2++='\\'; break;
case '\"': *ptr2++='\"'; break;
case '\b': *ptr2++='b'; break;
case '\f': *ptr2++='f'; break;
case '\n': *ptr2++='n'; break;
case '\r': *ptr2++='r'; break;
case '\t': *ptr2++='t'; break;
default: sprintf(ptr2,"u%04x",token);ptr2+=5; break; /* escape and print */
}
}
}
*ptr2++='\"';*ptr2++=0;
return out;
}
/* Invote print_string_ptr (which is useful) on an item. */
static char *print_string(cJSON *item) {return print_string_ptr(item->valuestring);}
/* Predeclare these prototypes. */
static const char *parse_value(cJSON *item,const char *value);
static char *print_value(cJSON *item,int depth,int fmt);
static const char *parse_array(cJSON *item,const char *value);
static char *print_array(cJSON *item,int depth,int fmt);
static const char *parse_object(cJSON *item,const char *value);
static char *print_object(cJSON *item,int depth,int fmt);
/* Utility to jump whitespace and cr/lf */
static const char *skip(const char *in) {while (in && *in && (unsigned char)*in<=32) in++; return in;}
/* Parse an object - create a new root, and populate. */
cJSON *cJSON_ParseWithOpts(const char *value,const char **return_parse_end,int require_null_terminated)
{
const char *end=0;
cJSON *c=cJSON_New_Item();
ep=0;
if (!c) return 0; /* memory fail */
end=parse_value(c,skip(value));
if (!end) {cJSON_Delete(c);return 0;} /* parse failure. ep is set. */
/* if we require null-terminated JSON without appended garbage, skip and then check for a null terminator */
if (require_null_terminated) {end=skip(end);if (*end) {cJSON_Delete(c);ep=end;return 0;}}
if (return_parse_end) *return_parse_end=end;
return c;
}
/* Default options for cJSON_Parse */
cJSON *cJSON_Parse(const char *value) {return cJSON_ParseWithOpts(value,0,0);}
/* Render a cJSON item/entity/structure to text. */
char *cJSON_Print(cJSON *item) {return print_value(item,0,1);}
char *cJSON_PrintUnformatted(cJSON *item) {return print_value(item,0,0);}
/* Parser core - when encountering text, process appropriately. */
static const char *parse_value(cJSON *item,const char *value)
{
if (!value) return 0; /* Fail on null. */
if (!strncmp(value,"null",4)) { item->type=cJSON_NULL; return value+4; }
if (!strncmp(value,"false",5)) { item->type=cJSON_False; return value+5; }
if (!strncmp(value,"true",4)) { item->type=cJSON_True; item->valueint=1; return value+4; }
if (*value=='\"') { return parse_string(item,value); }
if (*value=='-' || (*value>='0' && *value<='9')) { return parse_number(item,value); }
if (*value=='[') { return parse_array(item,value); }
if (*value=='{') { return parse_object(item,value); }
ep=value;return 0; /* failure. */
}
/* Render a value to text. */
static char *print_value(cJSON *item,int depth,int fmt)
{
char *out=0;
if (!item) return 0;
switch ((item->type)&255)
{
case cJSON_NULL: out=cJSON_strdup("null"); break;
case cJSON_False: out=cJSON_strdup("false");break;
case cJSON_True: out=cJSON_strdup("true"); break;
case cJSON_Number: out=print_number(item);break;
case cJSON_String: out=print_string(item);break;
case cJSON_Array: out=print_array(item,depth,fmt);break;
case cJSON_Object: out=print_object(item,depth,fmt);break;
}
return out;
}
/* Build an array from input text. */
static const char *parse_array(cJSON *item,const char *value)
{
cJSON *child;
if (*value!='[') {ep=value;return 0;} /* not an array! */
item->type=cJSON_Array;
value=skip(value+1);
if (*value==']') return value+1; /* empty array. */
item->child=child=cJSON_New_Item();
if (!item->child) return 0; /* memory fail */
value=skip(parse_value(child,skip(value))); /* skip any spacing, get the value. */
if (!value) return 0;
while (*value==',')
{
cJSON *new_item;
if (!(new_item=cJSON_New_Item())) return 0; /* memory fail */
child->next=new_item;new_item->prev=child;child=new_item;
value=skip(parse_value(child,skip(value+1)));
if (!value) return 0; /* memory fail */
}
if (*value==']') return value+1; /* end of array */
ep=value;return 0; /* malformed. */
}
/* Render an array to text */
static char *print_array(cJSON *item,int depth,int fmt)
{
char **entries;
char *out=0,*ptr,*ret;int len=5;
cJSON *child=item->child;
int numentries=0,i=0,fail=0;
/* How many entries in the array? */
while (child) numentries++,child=child->next;
/* Explicitly handle numentries==0 */
if (!numentries)
{
out=(char*)cJSON_malloc(3);
if (out) strcpy(out,"[]");
return out;
}
/* Allocate an array to hold the values for each */
entries=(char**)cJSON_malloc(numentries*sizeof(char*));
if (!entries) return 0;
memset(entries,0,numentries*sizeof(char*));
/* Retrieve all the results: */
child=item->child;
while (child && !fail)
{
ret=print_value(child,depth+1,fmt);
entries[i++]=ret;
if (ret) len+=strlen(ret)+2+(fmt?1:0); else fail=1;
child=child->next;
}
/* If we didn't fail, try to malloc the output string */
if (!fail) out=(char*)cJSON_malloc(len);
/* If that fails, we fail. */
if (!out) fail=1;
/* Handle failure. */
if (fail)
{
for (i=0;i<numentries;i++) if (entries[i]) cJSON_free(entries[i]);
cJSON_free(entries);
return 0;
}
/* Compose the output array. */
*out='[';
ptr=out+1;*ptr=0;
for (i=0;i<numentries;i++)
{
strcpy(ptr,entries[i]);ptr+=strlen(entries[i]);
if (i!=numentries-1) {*ptr++=',';if(fmt)*ptr++=' ';*ptr=0;}
cJSON_free(entries[i]);
}
cJSON_free(entries);
*ptr++=']';*ptr++=0;
return out;
}
/* Build an object from the text. */
static const char *parse_object(cJSON *item,const char *value)
{
cJSON *child;
if (*value!='{') {ep=value;return 0;} /* not an object! */
item->type=cJSON_Object;
value=skip(value+1);
if (*value=='}') return value+1; /* empty array. */
item->child=child=cJSON_New_Item();
if (!item->child) return 0;
value=skip(parse_string(child,skip(value)));
if (!value) return 0;
child->string=child->valuestring;child->valuestring=0;
if (*value!=':') {ep=value;return 0;} /* fail! */
value=skip(parse_value(child,skip(value+1))); /* skip any spacing, get the value. */
if (!value) return 0;
while (*value==',')
{
cJSON *new_item;
if (!(new_item=cJSON_New_Item())) return 0; /* memory fail */
child->next=new_item;new_item->prev=child;child=new_item;
value=skip(parse_string(child,skip(value+1)));
if (!value) return 0;
child->string=child->valuestring;child->valuestring=0;
if (*value!=':') {ep=value;return 0;} /* fail! */
value=skip(parse_value(child,skip(value+1))); /* skip any spacing, get the value. */
if (!value) return 0;
}
if (*value=='}') return value+1; /* end of array */
ep=value;return 0; /* malformed. */
}
/* Render an object to text. */
static char *print_object(cJSON *item,int depth,int fmt)
{
char **entries=0,**names=0;
char *out=0,*ptr,*ret,*str;int len=7,i=0,j;
cJSON *child=item->child;
int numentries=0,fail=0;
/* Count the number of entries. */
while (child) numentries++,child=child->next;
/* Explicitly handle empty object case */
if (!numentries)
{
out=(char*)cJSON_malloc(fmt?depth+3:3);
if (!out) return 0;
ptr=out;*ptr++='{';
if (fmt) {*ptr++='\n';for (i=0;i<depth-1;i++) *ptr++='\t';}
*ptr++='}';*ptr++=0;
return out;
}
/* Allocate space for the names and the objects */
entries=(char**)cJSON_malloc(numentries*sizeof(char*));
if (!entries) return 0;
names=(char**)cJSON_malloc(numentries*sizeof(char*));
if (!names) {cJSON_free(entries);return 0;}
memset(entries,0,sizeof(char*)*numentries);
memset(names,0,sizeof(char*)*numentries);
/* Collect all the results into our arrays: */
child=item->child;depth++;if (fmt) len+=depth;
while (child)
{
names[i]=str=print_string_ptr(child->string);
entries[i++]=ret=print_value(child,depth,fmt);
if (str && ret) len+=strlen(ret)+strlen(str)+2+(fmt?2+depth:0); else fail=1;
child=child->next;
}
/* Try to allocate the output string */
if (!fail) out=(char*)cJSON_malloc(len);
if (!out) fail=1;
/* Handle failure */
if (fail)
{
for (i=0;i<numentries;i++) {if (names[i]) cJSON_free(names[i]);if (entries[i]) cJSON_free(entries[i]);}
cJSON_free(names);cJSON_free(entries);
return 0;
}
/* Compose the output: */
*out='{';ptr=out+1;if (fmt)*ptr++='\n';*ptr=0;
for (i=0;i<numentries;i++)
{
if (fmt) for (j=0;j<depth;j++) *ptr++='\t';
strcpy(ptr,names[i]);ptr+=strlen(names[i]);
*ptr++=':';if (fmt) *ptr++='\t';
strcpy(ptr,entries[i]);ptr+=strlen(entries[i]);
if (i!=numentries-1) *ptr++=',';
if (fmt) *ptr++='\n';
*ptr=0;
cJSON_free(names[i]);cJSON_free(entries[i]);
}
cJSON_free(names);cJSON_free(entries);
if (fmt) for (i=0;i<depth-1;i++) *ptr++='\t';
*ptr++='}';*ptr++=0;
return out;
}
/* Get Array size/item / object item. */
int cJSON_GetArraySize(cJSON *array) {cJSON *c=array->child;int i=0;while(c)i++,c=c->next;return i;}
cJSON *cJSON_GetArrayItem(cJSON *array,int item) {cJSON *c=array->child; while (c && item>0) item--,c=c->next; return c;}
cJSON *cJSON_GetObjectItem(cJSON *object,const char *string) {cJSON *c=object->child; while (c && cJSON_strcasecmp(c->string,string)) c=c->next; return c;}
/* Utility for array list handling. */
static void suffix_object(cJSON *prev,cJSON *item) {prev->next=item;item->prev=prev;}
/* Utility for handling references. */
static cJSON *create_reference(cJSON *item) {cJSON *ref=cJSON_New_Item();if (!ref) return 0;memcpy(ref,item,sizeof(cJSON));ref->string=0;ref->type|=cJSON_IsReference;ref->next=ref->prev=0;return ref;}
/* Add item to array/object. */
void cJSON_AddItemToArray(cJSON *array, cJSON *item) {cJSON *c=array->child;if (!item) return; if (!c) {array->child=item;} else {while (c && c->next) c=c->next; suffix_object(c,item);}}
void cJSON_AddItemToObject(cJSON *object,const char *string,cJSON *item) {if (!item) return; if (item->string) cJSON_free(item->string);item->string=cJSON_strdup(string);cJSON_AddItemToArray(object,item);}
void cJSON_AddItemReferenceToArray(cJSON *array, cJSON *item) {cJSON_AddItemToArray(array,create_reference(item));}
void cJSON_AddItemReferenceToObject(cJSON *object,const char *string,cJSON *item) {cJSON_AddItemToObject(object,string,create_reference(item));}
cJSON *cJSON_DetachItemFromArray(cJSON *array,int which) {cJSON *c=array->child;while (c && which>0) c=c->next,which--;if (!c) return 0;
if (c->prev) c->prev->next=c->next;
if (c->next) c->next->prev=c->prev;
if (c==array->child) array->child=c->next;
c->prev=c->next=0;return c;}
void cJSON_DeleteItemFromArray(cJSON *array,int which) {cJSON_Delete(cJSON_DetachItemFromArray(array,which));}
cJSON *cJSON_DetachItemFromObject(cJSON *object,const char *string) {int i=0;cJSON *c=object->child;while (c && cJSON_strcasecmp(c->string,string)) i++,c=c->next;if (c) return cJSON_DetachItemFromArray(object,i);return 0;}
void cJSON_DeleteItemFromObject(cJSON *object,const char *string) {cJSON_Delete(cJSON_DetachItemFromObject(object,string));}
/* Replace array/object items with new ones. */
void cJSON_ReplaceItemInArray(cJSON *array,int which,cJSON *newitem) {cJSON *c=array->child;while (c && which>0) c=c->next,which--;if (!c) return;
newitem->next=c->next;newitem->prev=c->prev;if (newitem->next) newitem->next->prev=newitem;
if (c==array->child) array->child=newitem; else newitem->prev->next=newitem;c->next=c->prev=0;cJSON_Delete(c);}
void cJSON_ReplaceItemInObject(cJSON *object,const char *string,cJSON *newitem){int i=0;cJSON *c=object->child;while(c && cJSON_strcasecmp(c->string,string))i++,c=c->next;if(c){newitem->string=cJSON_strdup(string);cJSON_ReplaceItemInArray(object,i,newitem);}}
/* Create basic types: */
cJSON *cJSON_CreateNull(void) {cJSON *item=cJSON_New_Item();if(item)item->type=cJSON_NULL;return item;}
cJSON *cJSON_CreateTrue(void) {cJSON *item=cJSON_New_Item();if(item)item->type=cJSON_True;return item;}
cJSON *cJSON_CreateFalse(void) {cJSON *item=cJSON_New_Item();if(item)item->type=cJSON_False;return item;}
cJSON *cJSON_CreateBool(int b) {cJSON *item=cJSON_New_Item();if(item)item->type=b?cJSON_True:cJSON_False;return item;}
cJSON *cJSON_CreateNumber(double num) {cJSON *item=cJSON_New_Item();if(item){item->type=cJSON_Number;item->valuedouble=num;item->valueint=(int)num;}return item;}
cJSON *cJSON_CreateString(const char *string) {cJSON *item=cJSON_New_Item();if(item){item->type=cJSON_String;item->valuestring=cJSON_strdup(string);}return item;}
cJSON *cJSON_CreateArray(void) {cJSON *item=cJSON_New_Item();if(item)item->type=cJSON_Array;return item;}
cJSON *cJSON_CreateObject(void) {cJSON *item=cJSON_New_Item();if(item)item->type=cJSON_Object;return item;}
/* Create Arrays: */
cJSON *cJSON_CreateIntArray(int *numbers,int count) {int i;cJSON *n=0,*p=0,*a=cJSON_CreateArray();for(i=0;a && i<count;i++){n=cJSON_CreateNumber(numbers[i]);if(!i)a->child=n;else suffix_object(p,n);p=n;}return a;}
cJSON *cJSON_CreateFloatArray(float *numbers,int count) {int i;cJSON *n=0,*p=0,*a=cJSON_CreateArray();for(i=0;a && i<count;i++){n=cJSON_CreateNumber(numbers[i]);if(!i)a->child=n;else suffix_object(p,n);p=n;}return a;}
cJSON *cJSON_CreateDoubleArray(double *numbers,int count) {int i;cJSON *n=0,*p=0,*a=cJSON_CreateArray();for(i=0;a && i<count;i++){n=cJSON_CreateNumber(numbers[i]);if(!i)a->child=n;else suffix_object(p,n);p=n;}return a;}
cJSON *cJSON_CreateStringArray(const char **strings,int count) {int i;cJSON *n=0,*p=0,*a=cJSON_CreateArray();for(i=0;a && i<count;i++){n=cJSON_CreateString(strings[i]);if(!i)a->child=n;else suffix_object(p,n);p=n;}return a;}
/* Duplication */
cJSON *cJSON_Duplicate(cJSON *item,int recurse)
{
cJSON *newitem,*cptr,*nptr=0,*newchild;
/* Bail on bad ptr */
if (!item) return 0;
/* Create new item */
newitem=cJSON_New_Item();
if (!newitem) return 0;
/* Copy over all vars */
newitem->type=item->type&(~cJSON_IsReference),newitem->valueint=item->valueint,newitem->valuedouble=item->valuedouble;
if (item->valuestring) {newitem->valuestring=cJSON_strdup(item->valuestring); if (!newitem->valuestring) {cJSON_Delete(newitem);return 0;}}
if (item->string) {newitem->string=cJSON_strdup(item->string); if (!newitem->string) {cJSON_Delete(newitem);return 0;}}
/* If non-recursive, then we're done! */
if (!recurse) return newitem;
/* Walk the ->next chain for the child. */
cptr=item->child;
while (cptr)
{
newchild=cJSON_Duplicate(cptr,1); /* Duplicate (with recurse) each item in the ->next chain */
if (!newchild) {cJSON_Delete(newitem);return 0;}
if (nptr) {nptr->next=newchild,newchild->prev=nptr;nptr=newchild;} /* If newitem->child already set, then crosswire ->prev and ->next and move on */
else {newitem->child=newchild;nptr=newchild;} /* Set newitem->child and move to it */
cptr=cptr->next;
}
return newitem;
}
#endif
================================================
FILE: src/ngx_stream_json.h
================================================
/*
Copyright (c) 2009 Dave Gamble
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
*/
#ifndef cJSON__h
#define cJSON__h
#ifdef __cplusplus
extern "C"
{
#endif
/* cJSON Types: */
#define cJSON_False 0
#define cJSON_True 1
#define cJSON_NULL 2
#define cJSON_Number 3
#define cJSON_String 4
#define cJSON_Array 5
#define cJSON_Object 6
#define cJSON_IsReference 256
/* The cJSON structure: */
typedef struct cJSON {
struct cJSON *next,*prev; /* next/prev allow you to walk array/object chains. Alternatively, use GetArraySize/GetArrayItem/GetObjectItem */
struct cJSON *child; /* An array or object item will have a child pointer pointing to a chain of the items in the array/object. */
int type; /* The type of the item, as above. */
char *valuestring; /* The item's string, if type==cJSON_String */
int valueint; /* The item's number, if type==cJSON_Number */
double valuedouble; /* The item's number, if type==cJSON_Number */
char *string; /* The item's name string, if this item is the child of, or is in the list of subitems of an object. */
} cJSON;
typedef struct cJSON_Hooks {
void *(*malloc_fn)(size_t sz);
void (*free_fn)(void *ptr);
} cJSON_Hooks;
/* Supply malloc, realloc and free functions to cJSON */
extern void cJSON_InitHooks(cJSON_Hooks* hooks);
/* Supply a block of JSON, and this returns a cJSON object you can interrogate. Call cJSON_Delete when finished. */
extern cJSON *cJSON_Parse(const char *value);
/* Render a cJSON entity to text for transfer/storage. Free the char* when finished. */
extern char *cJSON_Print(cJSON *item);
/* Render a cJSON entity to text for transfer/storage without any formatting. Free the char* when finished. */
extern char *cJSON_PrintUnformatted(cJSON *item);
/* Delete a cJSON entity and all subentities. */
extern void cJSON_Delete(cJSON *c);
/* Returns the number of items in an array (or object). */
extern int cJSON_GetArraySize(cJSON *array);
/* Retrieve item number "item" from array "array". Returns NULL if unsuccessful. */
extern cJSON *cJSON_GetArrayItem(cJSON *array,int item);
/* Get item "string" from object. Case insensitive. */
extern cJSON *cJSON_GetObjectItem(cJSON *object,const char *string);
/* For analysing failed parses. This returns a pointer to the parse error. You'll probably need to look a few chars back to make sense of it. Defined when cJSON_Parse() returns 0. 0 when cJSON_Parse() succeeds. */
extern const char *cJSON_GetErrorPtr(void);
/* These calls create a cJSON item of the appropriate type. */
extern cJSON *cJSON_CreateNull(void);
extern cJSON *cJSON_CreateTrue(void);
extern cJSON *cJSON_CreateFalse(void);
extern cJSON *cJSON_CreateBool(int b);
extern cJSON *cJSON_CreateNumber(double num);
extern cJSON *cJSON_CreateString(const char *string);
extern cJSON *cJSON_CreateArray(void);
extern cJSON *cJSON_CreateObject(void);
/* These utilities create an Array of count items. */
extern cJSON *cJSON_CreateIntArray(int *numbers,int count);
extern cJSON *cJSON_CreateFloatArray(float *numbers,int count);
extern cJSON *cJSON_CreateDoubleArray(double *numbers,int count);
extern cJSON *cJSON_CreateStringArray(const char **strings,int count);
/* Append item to the specified array/object. */
extern void cJSON_AddItemToArray(cJSON *array, cJSON *item);
extern void cJSON_AddItemToObject(cJSON *object,const char *string,cJSON *item);
/* Append reference to item to the specified array/object. Use this when you want to add an existing cJSON to a new cJSON, but don't want to corrupt your existing cJSON. */
extern void cJSON_AddItemReferenceToArray(cJSON *array, cJSON *item);
extern void cJSON_AddItemReferenceToObject(cJSON *object,const char *string,cJSON *item);
/* Remove/Detatch items from Arrays/Objects. */
extern cJSON *cJSON_DetachItemFromArray(cJSON *array,int which);
extern void cJSON_DeleteItemFromArray(cJSON *array,int which);
extern cJSON *cJSON_DetachItemFromObject(cJSON *object,const char *string);
extern void cJSON_DeleteItemFromObject(cJSON *object,const char *string);
/* Update array items. */
extern void cJSON_ReplaceItemInArray(cJSON *array,int which,cJSON *newitem);
extern void cJSON_ReplaceItemInObject(cJSON *object,const char *string,cJSON *newitem);
/* Duplicate a cJSON item */
extern cJSON *cJSON_Duplicate(cJSON *item,int recurse);
/* Duplicate will create a new, identical cJSON item to the one you pass, in new memory that will
need to be released. With recurse!=0, it will duplicate any children connected to the item.
The item->next and ->prev pointers are always zero on return from Duplicate. */
/* ParseWithOpts allows you to require (and check) that the JSON is null terminated, and to retrieve the pointer to the final byte parsed. */
extern cJSON *cJSON_ParseWithOpts(const char *value,const char **return_parse_end,int require_null_terminated);
/* Macros for creating things quickly. */
#define cJSON_AddNullToObject(object,name) cJSON_AddItemToObject(object, name, cJSON_CreateNull())
#define cJSON_AddTrueToObject(object,name) cJSON_AddItemToObject(object, name, cJSON_CreateTrue())
#define cJSON_AddFalseToObject(object,name) cJSON_AddItemToObject(object, name, cJSON_CreateFalse())
#define cJSON_AddBoolToObject(object,name,b) cJSON_AddItemToObject(object, name, cJSON_CreateBool(b))
#define cJSON_AddNumberToObject(object,name,n) cJSON_AddItemToObject(object, name, cJSON_CreateNumber(n))
#define cJSON_AddStringToObject(object,name,s) cJSON_AddItemToObject(object, name, cJSON_CreateString(s))
/* When assigning an integer value, it needs to be propagated to valuedouble too. */
#define cJSON_SetIntValue(object,val) ((object)?(object)->valueint=(object)->valuedouble=(val):(val))
#ifdef __cplusplus
}
#endif
#endif
================================================
FILE: src/ngx_stream_upsync_module.c
================================================
/*
* Copyright (C) 2015 Weibo Group Holding Limited
* Copyright (C) 2015 Xiaokai Wang (xiaokai.wang@live.com)
*/
#include <ngx_core.h>
#include <ngx_stream.h>
#include <ngx_config.h>
#include "ngx_stream_upsync_module.h"
/* client for conf service */
typedef struct {
ngx_int_t sd;
ngx_int_t port;
ngx_int_t connected;
char ip[16];
struct sockaddr_in addr;
} ngx_stream_conf_client;
typedef struct {
u_char sockaddr[NGX_SOCKADDRLEN];
ngx_int_t weight;
ngx_uint_t max_fails;
time_t fail_timeout;
unsigned down:1;
unsigned backup:1;
} ngx_stream_upsync_conf_t;
#define NGX_STREAM_UPSYNC_CONSUL 0x0001
#define NGX_STREAM_UPSYNC_CONSUL_SERVICES 0x0002
#define NGX_STREAM_UPSYNC_ETCD 0x0003
typedef ngx_int_t (*ngx_stream_upsync_packet_init_pt)
(void *upsync_server);
typedef ngx_int_t (*ngx_stream_upsync_packet_parse_pt)
(void *upsync_server);
typedef void (*ngx_stream_upsync_packet_clean_pt)
(void *upsync_server);
typedef struct {
ngx_str_t name;
ngx_uint_t upsync_type;
ngx_event_handler_pt send_handler;
ngx_event_handler_pt recv_handler;
ngx_stream_upsync_packet_init_pt init;
ngx_stream_upsync_packet_parse_pt parse;
ngx_stream_upsync_packet_clean_pt clean;
} ngx_upsync_conf_t;
typedef struct {
ngx_pool_t *pool;
ngx_buf_t send;
ngx_buf_t recv;
ngx_buf_t body;
ngx_array_t del_upstream; /* ngx_stream_upsync_conf_t */
ngx_array_t add_upstream;
ngx_array_t upstream_conf;
} ngx_stream_upsync_ctx_t;
typedef struct {
ngx_str_t upsync_host;
ngx_int_t upsync_port;
ngx_msec_t upsync_timeout;
ngx_msec_t upsync_interval;
ngx_int_t upsync_lb;
ngx_uint_t strong_dependency;
ngx_str_t upsync_send;
ngx_str_t upsync_dump_path;
ngx_open_file_t *conf_file;
ngx_upsync_conf_t *upsync_type_conf;
ngx_stream_upstream_server_t conf_server; /* conf server */
} ngx_stream_upsync_srv_conf_t;
/* based on upstream conf, every unit upsync from consul */
typedef struct {
ngx_str_t host;
uint64_t index;
uint64_t update_generation;
ngx_event_t upsync_ev;
ngx_event_t upsync_timeout_ev;
ngx_queue_t delete_ev;
ngx_shmtx_t upsync_accept_mutex;
ngx_peer_connection_t pc;
ngx_stream_upsync_ctx_t ctx;
ngx_stream_upsync_srv_conf_t *upscf;
ngx_stream_upstream_srv_conf_t *uscf;
} ngx_stream_upsync_server_t;
typedef struct {
ngx_event_t delay_delete_ev;
ngx_queue_t queue;
time_t start_sec;
ngx_msec_t start_msec;
void *data;
} ngx_delay_event_t;
typedef struct {
ngx_uint_t upstream_num;
ngx_stream_upsync_server_t *upsync_server;
} ngx_stream_upsync_main_conf_t;
/* http parser state */
typedef struct {
u_char status[3];
char headers[NGX_MAX_HEADERS][2][NGX_MAX_ELEMENT_SIZE];
ngx_uint_t num_headers;
enum { NONE=0, FIELD, VALUE } last_header;
u_char http_body[NGX_PAGE_SIZE * NGX_PAGE_NUMBER];
} ngx_stream_http_state;
static ngx_upsync_conf_t *ngx_stream_upsync_get_type_conf(ngx_str_t *str);
static char *ngx_stream_upsync_set_lb(ngx_conf_t *cf, ngx_command_t *cmd,
void *conf);
static char *ngx_stream_upsync_server(ngx_conf_t *cf,
ngx_command_t *cmd, void *conf);
static char *ngx_stream_upsync_set_conf_dump(ngx_conf_t *cf,
ngx_command_t *cmd, void *conf);
static void *ngx_stream_upsync_create_main_conf(ngx_conf_t *cf);
static void *ngx_stream_upsync_create_srv_conf(ngx_conf_t *cf);
static char *ngx_stream_upsync_init_main_conf(ngx_conf_t *cf, void *conf);
static char *ngx_stream_upsync_init_srv_conf(ngx_conf_t *cf, void *conf,
ngx_uint_t num);
static void ngx_stream_upsync_process(ngx_stream_upsync_server_t *upsync_server);
static ngx_int_t ngx_stream_upsync_init_process(ngx_cycle_t *cycle);
static ngx_int_t ngx_stream_upsync_init_module(ngx_cycle_t *cycle);
static ngx_int_t ngx_stream_upsync_init_shm_mutex(ngx_cycle_t *cycle);
static ngx_int_t ngx_stream_upsync_add_timers(ngx_cycle_t *cycle);
static void ngx_stream_upsync_begin_handler(ngx_event_t *event);
static void ngx_stream_upsync_connect_handler(ngx_event_t *event);
static void ngx_stream_upsync_recv_handler(ngx_event_t *event);
static void ngx_stream_upsync_send_handler(ngx_event_t *event);
static void ngx_stream_upsync_recv_empty_handler(ngx_event_t *event);
static void ngx_stream_upsync_send_empty_handler(ngx_event_t *event);
static void ngx_stream_upsync_timeout_handler(ngx_event_t *event);
static void ngx_stream_upsync_clean_event(void *upsync_server);
static ngx_int_t ngx_stream_upsync_etcd_parse_init(void *upsync_server);
static ngx_int_t ngx_stream_upsync_consul_parse_init(void *upsync_server);
static ngx_int_t ngx_stream_upsync_dump_server(
ngx_stream_upsync_server_t *upsync_server);
static ngx_int_t ngx_stream_upsync_init_server(ngx_event_t *event);
static ngx_int_t ngx_stream_upsync_add_peers(ngx_cycle_t *cycle,
ngx_stream_upsync_server_t *upsync_server);
static ngx_int_t ngx_stream_upsync_del_peers(ngx_cycle_t *cycle,
ngx_stream_upsync_server_t *upsync_server);
static ngx_int_t ngx_stream_upsync_replace_peers(ngx_cycle_t *cycle,
ngx_stream_upsync_server_t *upsync_server);
static void ngx_stream_upsync_update_peer(ngx_stream_upstream_rr_peers_t *peers,
ngx_stream_upstream_rr_peer_t *peer,
ngx_stream_upsync_conf_t *upstream_conf,
ngx_uint_t *updated);
static void ngx_stream_upsync_diff_filter(ngx_cycle_t *cycle,
ngx_stream_upsync_server_t *upsync_server,
ngx_uint_t *diff);
static void ngx_stream_upsync_event_init(ngx_stream_upstream_rr_peer_t *peer,
ngx_stream_upsync_server_t *upsync_server);
static ngx_int_t ngx_stream_http_parser_init();
static int ngx_stream_http_status(http_parser *p, const char *buf, size_t len);
static int ngx_stream_http_header_field_cb(http_parser *p, const char *buf,
size_t len);
static int ngx_stream_http_header_value_cb(http_parser *p, const char *buf,
size_t len);
static int ngx_stream_http_body(http_parser *p, const char *buf, size_t len);
static ngx_int_t ngx_stream_upsync_check_index(
ngx_stream_upsync_server_t *upsync_server);
static ngx_int_t ngx_stream_upsync_consul_parse_json(void *upsync_server);
static ngx_int_t ngx_stream_upsync_consul_services_parse_json(
void *upsync_server);
static ngx_int_t ngx_stream_upsync_etcd_parse_json(void *upsync_server);
static ngx_int_t ngx_stream_upsync_check_key(u_char *key, ngx_str_t host);
static void *ngx_stream_upsync_servers(ngx_cycle_t *cycle,
ngx_stream_upsync_server_t *upsync_server, ngx_flag_t flag);
static void *ngx_stream_upsync_addrs(ngx_pool_t *pool, u_char *sockaddr);
static void ngx_stream_upsync_del_delay_delete(ngx_event_t *event);
static ngx_int_t ngx_stream_upsync_need_exit();
static void ngx_stream_upsync_clear_all_events(ngx_cycle_t *cycle);
static ngx_int_t ngx_stream_upsync_get_upstream(ngx_cycle_t *cycle,
ngx_stream_upsync_server_t *upsync_server, char **conf_value);
static ngx_stream_conf_client *ngx_stream_create_client(ngx_cycle_t *cycle,
ngx_stream_upsync_server_t *upsync_server);
static ngx_int_t ngx_stream_client_conn(ngx_stream_conf_client *client);
static void ngx_stream_client_destroy(ngx_stream_conf_client *client);
static ngx_int_t ngx_stream_client_send(ngx_stream_conf_client *client,
ngx_stream_upsync_server_t *upsync_server);
static ngx_int_t ngx_stream_client_recv(ngx_stream_conf_client *client,
char **data, int size);
static char *ngx_stream_upsync_set(ngx_conf_t *cf, ngx_command_t *cmd,
void *conf);
static void ngx_stream_upsync_show(ngx_stream_session_t *r);
static http_parser_settings settings = {
.on_message_begin = 0,
.on_header_field = ngx_stream_http_header_field_cb,
.on_header_value = ngx_stream_http_header_value_cb,
.on_url = 0,
.on_status = ngx_stream_http_status,
.on_body = ngx_stream_http_body,
.on_headers_complete = 0,
.on_message_complete = 0
};
ngx_atomic_t stream_upsync_shared_created0;
ngx_atomic_t *stream_upsync_shared_created = &stream_upsync_shared_created0;
static http_parser *parser = NULL;
static ngx_stream_http_state state;
static ngx_stream_upsync_main_conf_t *upsync_ctx = NULL;
static ngx_command_t ngx_stream_upsync_commands[] = {
{ ngx_string("upsync"),
NGX_STREAM_UPS_CONF|NGX_CONF_1MORE,
ngx_stream_upsync_server,
NGX_STREAM_SRV_CONF_OFFSET,
0,
NULL },
{ ngx_string("upsync_lb"),
NGX_STREAM_UPS_CONF|NGX_CONF_TAKE1,
ngx_stream_upsync_set_lb,
0,
0,
NULL },
{ ngx_string("upsync_dump_path"),
NGX_STREAM_UPS_CONF|NGX_CONF_TAKE1,
ngx_stream_upsync_set_conf_dump,
NGX_STREAM_SRV_CONF_OFFSET,
0,
NULL },
{ ngx_string("upstream_show"),
NGX_STREAM_SRV_CONF|NGX_CONF_NOARGS,
ngx_stream_upsync_set,
0,
0,
NULL },
ngx_null_command
};
static ngx_stream_module_t ngx_stream_upsync_module_ctx = {
NULL, /* preconfiguration */
NULL, /* postconfiguration */
ngx_stream_upsync_create_main_conf, /* create main configuration */
ngx_stream_upsync_init_main_conf, /* init main configuration */
ngx_stream_upsync_create_srv_conf, /* create server configuration */
NULL, /* merge server configuration */
};
ngx_module_t ngx_stream_upsync_module = {
NGX_MODULE_V1,
&ngx_stream_upsync_module_ctx, /* module context */
ngx_stream_upsync_commands, /* module directives */
NGX_STREAM_MODULE, /* module type */
NULL, /* init master */
ngx_stream_upsync_init_module, /* init module */
ngx_stream_upsync_init_process, /* init process */
NULL, /* init thread */
NULL, /* exit thread */
ngx_stream_upsync_clear_all_events, /* exit process */
NULL, /* exit master */
NGX_MODULE_V1_PADDING
};
static ngx_upsync_conf_t ngx_upsync_types[] = {
{ ngx_string("consul"),
NGX_STREAM_UPSYNC_CONSUL,
ngx_stream_upsync_send_handler,
ngx_stream_upsync_recv_handler,
ngx_stream_upsync_consul_parse_init,
ngx_stream_upsync_consul_parse_json,
ngx_stream_upsync_clean_event },
{ ngx_string("consul_services"),
NGX_STREAM_UPSYNC_CONSUL_SERVICES,
ngx_stream_upsync_send_handler,
ngx_stream_upsync_recv_handler,
ngx_stream_upsync_consul_parse_init,
ngx_stream_upsync_consul_services_parse_json,
ngx_stream_upsync_clean_event },
{ ngx_string("etcd"),
NGX_STREAM_UPSYNC_ETCD,
ngx_stream_upsync_send_handler,
ngx_stream_upsync_recv_handler,
ngx_stream_upsync_etcd_parse_init,
ngx_stream_upsync_etcd_parse_json,
ngx_stream_upsync_clean_event },
{ ngx_null_string,
0,
NULL,
NULL,
NULL,
NULL,
NULL }
};
static char *
ngx_stream_upsync_server(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
{
u_char *p = NULL;
time_t upsync_timeout = 0, upsync_interval = 0;
ngx_str_t *value, s;
ngx_url_t u;
ngx_uint_t i, strong_dependency = 0;
ngx_stream_upstream_server_t *conf_server;
ngx_stream_upsync_srv_conf_t *upscf;
value = cf->args->elts;
upscf = ngx_stream_conf_get_module_srv_conf(cf,
ngx_stream_upsync_module);
conf_server = &upscf->conf_server;
for (i = 2; i < cf->args->nelts; i++) {
if (ngx_strncmp(value[i].data, "upsync_timeout=", 15) == 0) {
s.len = value[i].len - 15;
s.data = &value[i].data[15];
upsync_timeout = ngx_parse_time(&s, 0);
if (upsync_timeout == (time_t) NGX_ERROR) {
ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
"upsync_server: invalid parameter:\"%V\"",
&value[i]);
goto invalid;
}
continue;
}
if (ngx_strncmp(value[i].data, "upsync_interval=", 16) == 0) {
s.len = value[i].len - 16;
s.data = &value[i].data[16];
upsync_interval = ngx_parse_time(&s, 0);
if (upsync_interval == (time_t) NGX_ERROR) {
ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
"upsync_server: invalid parameter: \"%V\"",
&value[i]);
goto invalid;
}
continue;
}
if (ngx_strncmp(value[i].data, "strong_dependency=", 18) == 0) {
s.len = value[i].len - 18;
s.data = value[i].data + 18;
if (ngx_strcasecmp(s.data, (u_char *) "on") == 0) {
strong_dependency = 1;
} else if (ngx_strcasecmp(s.data, (u_char *) "off") == 0) {
strong_dependency = 0;
} else {
ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
"invalid value \"%s\", "
"it must be \"on\" or \"off\"",
value[i].data);
return NGX_CONF_ERROR;
}
continue;
}
if (ngx_strncmp(value[i].data, "upsync_type=", 12) == 0) {
s.len = value[i].len - 12;
s.data = value[i].data + 12;
upscf->upsync_type_conf = ngx_stream_upsync_get_type_conf(&s);
if (upscf->upsync_type_conf == NULL) {
ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
"upsync_server: upsync_type invalid para");
goto invalid;
}
continue;
}
goto invalid;
}
if (upsync_interval != 0) {
upscf->upsync_interval = upsync_interval;
}
if (upsync_timeout != 0) {
upscf->upsync_timeout = upsync_timeout;
}
if (strong_dependency != 0) {
upscf->strong_dependency = strong_dependency;
}
if (upscf->upsync_type_conf == NGX_CONF_UNSET_PTR) {
ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
"upsync_server: upsync_type cannt be null");
goto invalid;
}
ngx_memzero(&u, sizeof(ngx_url_t));
p = (u_char *)ngx_strchr(value[1].data, '/');
if (p == NULL) {
ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "upsync_server: "
"please input conf_server upstream key in upstream");
return NGX_CONF_ERROR;
}
upscf->upsync_send.data = p;
upscf->upsync_send.len = value[1].len - (p - value[1].data);
u.url.data = value[1].data;
u.url.len = p - value[1].data;
p = (u_char *)ngx_strchr(value[1].data, ':');
if (p != NULL) {
upscf->upsync_host.data = value[1].data;
upscf->upsync_host.len = p - value[1].data;
upscf->upsync_port = ngx_atoi(p + 1, upscf->upsync_send.data - p - 1);
if (upscf->upsync_port < 1 || upscf->upsync_port > 65535) {
ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "upsync_server: "
"conf server port is invalid");
return NGX_CONF_ERROR;
}
} else {
upscf->upsync_host.data = value[1].data;
upscf->upsync_host.len = u.url.len;
upscf->upsync_port = 80;
}
u.default_port = 80;
if (ngx_parse_url(cf->pool, &u) != NGX_OK) {
if (u.err) {
ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "upsync_server: "
"%s in upstream \"%V\"", u.err, &u.url);
}
return NGX_CONF_ERROR;
}
conf_server->name = u.url;
conf_server->addrs = u.addrs;
conf_server->naddrs = u.naddrs;
conf_server->weight = 1;
conf_server->max_fails = 1;
conf_server->fail_timeout = 10;
return NGX_CONF_OK;
invalid:
return NGX_CONF_ERROR;
}
static ngx_upsync_conf_t *
ngx_stream_upsync_get_type_conf(ngx_str_t *str)
{
ngx_uint_t i;
for (i = 0; /* void */ ; i++) {
if (ngx_upsync_types[i].upsync_type == 0) {
break;
}
if (str->len != ngx_upsync_types[i].name.len) {
continue;
}
if (ngx_strncmp(str->data, ngx_upsync_types[i].name.data,
str->len) == 0)
{
return &ngx_upsync_types[i];
}
}
return NULL;
}
static char *
ngx_stream_upsync_set_lb(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
{
ngx_str_t *value, *str;
ngx_stream_upsync_srv_conf_t *upscf;
upscf = ngx_stream_conf_get_module_srv_conf(cf,
ngx_stream_upsync_module);
value = cf->args->elts;
str = &value[1];
if (str->len == NGX_CONF_UNSET_SIZE) {
upscf->upsync_lb = NGX_STREAM_LB_DEFAULT;
return NGX_CONF_OK;
}
switch(str->len) {
case 7:
if (ngx_memcmp((char *)str->data, "ip_hash", 7) == 0) {
upscf->upsync_lb = NGX_STREAM_LB_IP_HASH;
return NGX_CONF_OK;
}
break;
case 10:
if (ngx_memcmp((char *)str->data, "roundrobin", 10) == 0) {
upscf->upsync_lb = NGX_STREAM_LB_ROUNDROBIN;
return NGX_CONF_OK;
}
if (ngx_memcmp((char *)str->data, "least_conn", 10) == 0) {
upscf->upsync_lb = NGX_STREAM_LB_LEAST_CONN;
return NGX_CONF_OK;
}
break;
case 11:
if (ngx_memcmp((char *)str->data, "hash_modula", 11) == 0) {
upscf->upsync_lb = NGX_STREAM_LB_HASH_MODULA;
return NGX_CONF_OK;
}
if (ngx_memcmp((char *)str->data, "hash_ketama", 11) == 0) {
upscf->upsync_lb = NGX_STREAM_LB_HASH_KETAMA;
return NGX_CONF_OK;
}
break;
}
return NGX_CONF_OK;
}
static char *
ngx_stream_upsync_set_conf_dump(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
{
ngx_str_t *value;
ngx_stream_upsync_srv_conf_t *upscf;
upscf = ngx_stream_conf_get_module_srv_conf(cf,
ngx_stream_upsync_module);
value = cf->args->elts;
upscf->upsync_dump_path = value[1];
if (upscf->upsync_dump_path.len == NGX_CONF_UNSET_SIZE) {
return NGX_CONF_ERROR;
}
upscf->conf_file = ngx_conf_open_file(cf->cycle, &value[1]);
if (upscf->conf_file == NULL) {
return NGX_CONF_ERROR;
}
return NGX_CONF_OK;
}
static void
ngx_stream_upsync_process(ngx_stream_upsync_server_t *upsync_server)
{
ngx_uint_t diff = 0;
ngx_upsync_conf_t *upsync_type_conf;
ngx_stream_upsync_ctx_t *ctx;
if (ngx_stream_upsync_need_exit()) {
return;
}
ctx = &upsync_server->ctx;
upsync_type_conf = upsync_server->upscf->upsync_type_conf;
if (ngx_stream_upsync_check_index(upsync_server) == NGX_ERROR) {
return;
}
if (upsync_type_conf->parse(upsync_server) == NGX_ERROR) {
if (upsync_server->index != 0) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"upsync_process: parse json error");
}
return;
}
ngx_log_debug0(NGX_LOG_DEBUG, ngx_cycle->log, 0,
"upsync_process: parse json success");
ngx_stream_upsync_diff_filter((ngx_cycle_t *)ngx_cycle, upsync_server, &diff);
if (ctx->add_upstream.nelts > 0) {
if (upsync_server->update_generation != 0) {
if (ngx_stream_upsync_add_peers((ngx_cycle_t *)ngx_cycle,
upsync_server) != NGX_OK) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"upsync_process: upstream add peers failed");
return;
}
} else {
if (ngx_stream_upsync_replace_peers((ngx_cycle_t *)ngx_cycle,
upsync_server) != NGX_OK) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"upsync_process: upstream add/replace peers failed");
return;
}
}
upsync_server->update_generation++;
}
if (ctx->del_upstream.nelts > 0) {
if (upsync_server->update_generation != 0) {
if (ngx_stream_upsync_del_peers((ngx_cycle_t *)ngx_cycle,
upsync_server) != NGX_OK) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"upsync_process: upstream del peers failed");
return;
}
} else {
if (ngx_stream_upsync_replace_peers((ngx_cycle_t *)ngx_cycle,
upsync_server) != NGX_OK) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"upsync_process: upstream del/replace peers failed");
return;
}
}
upsync_server->update_generation++;
}
if (diff) {
if (ngx_shmtx_trylock(&upsync_server->upsync_accept_mutex)) {
ngx_stream_upsync_dump_server(upsync_server);
ngx_shmtx_unlock(&upsync_server->upsync_accept_mutex);
}
}
return;
}
static ngx_int_t
ngx_stream_upsync_check_index(ngx_stream_upsync_server_t *upsync_server)
{
char *p;
ngx_uint_t i;
uint64_t index = 0;
ngx_upsync_conf_t *upsync_type_conf;
upsync_type_conf = upsync_server->upscf->upsync_type_conf;
if (upsync_type_conf->upsync_type == NGX_STREAM_UPSYNC_CONSUL
|| upsync_type_conf->upsync_type == NGX_STREAM_UPSYNC_CONSUL_SERVICES) {
for (i = 0; i < state.num_headers; i++) {
if (ngx_memcmp(state.headers[i][0], NGX_INDEX_HEADER,
NGX_INDEX_HEADER_LEN) == 0) {
p = ngx_strchr(state.headers[i][1], '\r');
*p = '\0';
index = ngx_strtoull((u_char *)state.headers[i][1],
(char **)NULL, 10);
break;
}
}
if (index == upsync_server->index) {
ngx_log_error(NGX_LOG_NOTICE, ngx_cycle->log, 0,
"upsync_check_index: upstream index has not changed: %V",
&upsync_server->upscf->upsync_dump_path);
return NGX_ERROR;
} else {
upsync_server->index = index;
}
}
if (upsync_type_conf->upsync_type == NGX_STREAM_UPSYNC_ETCD) {
for (i = 0; i < state.num_headers; i++) {
if (ngx_memcmp(state.headers[i][0], NGX_INDEX_ETCD_HEADER,
NGX_INDEX_ETCD_HEADER_LEN) == 0) {
p = ngx_strchr(state.headers[i][1], '\r');
*p = '\0';
index = ngx_strtoull((u_char *)state.headers[i][1],
(char **)NULL, 10);
break;
}
}
upsync_server->index = index + 1;
}
return NGX_OK;
}
static ngx_int_t
ngx_stream_upsync_add_peers(ngx_cycle_t *cycle,
ngx_stream_upsync_server_t *upsync_server)
{
ngx_uint_t i=0, n=0, w=0, len=0;
ngx_array_t *servers;
ngx_stream_upstream_server_t *server = NULL;
ngx_stream_upstream_rr_peer_t *peer = NULL;
ngx_stream_upstream_rr_peers_t *peers = NULL;
ngx_stream_upstream_srv_conf_t *uscf;
if (ngx_stream_upsync_need_exit()) {
return NGX_OK;
}
u_char *namep = NULL;
struct sockaddr *saddr = NULL;
len = sizeof(struct sockaddr);
uscf = upsync_server->uscf;
servers = ngx_stream_upsync_servers(cycle, upsync_server, NGX_ADD);
if (servers == NULL) {
return NGX_ERROR;
}
if (servers->nelts < 1) {
ngx_log_error(NGX_LOG_ERR, cycle->log, 0,
"upsync_add_peers: no servers to add \"%V\"", &uscf->host);
return NGX_ERROR;
}
if (uscf->peer.data == NULL) {
return NGX_ERROR;
}
peers = (ngx_stream_upstream_rr_peers_t *)uscf->peer.data;
if (peers && servers->nelts >= 1) {
n = peers->number + servers->nelts;
for (i = 0; i < servers->nelts; i++) {
server = (ngx_stream_upstream_server_t *)servers->elts + i;
//if (server->backup) {
// continue;
//}
//
// FIXME: until backup is fully implemented this causes crashes
// on startup with nodes set backup=1. Let them in for now
peer = ngx_calloc(sizeof(ngx_stream_upstream_rr_peer_t),
cycle->log);
if (peer == NULL) {
goto invalid;
}
if ((saddr = ngx_calloc(len, cycle->log)) == NULL) {
goto invalid;
}
ngx_memcpy(saddr, server->addrs->sockaddr, len);
peer->sockaddr = saddr;
if ((namep = ngx_calloc(server->addrs->name.len,
cycle->log)) == NULL) {
goto invalid;
}
ngx_memcpy(namep, server->addrs->name.data,
server->addrs->name.len);
peer->name.data = namep;
peer->socklen = server->addrs->socklen;
peer->name.len = server->addrs->name.len;
peer->max_fails = server->max_fails;
peer->fail_timeout = server->fail_timeout;
peer->down = server->down;
peer->weight = server->weight;
peer->effective_weight = server->weight;
peer->current_weight = 0;
peer->conns = 0;
peer->next = peers->peer;
peers->peer = peer;
w += server->weight;
}
w += peers->total_weight;
peers->single = (n == 1);
peers->number = n;
peers->weighted = (w != n);
peers->total_weight = w;
if (upsync_server->upscf->upsync_lb == NGX_STREAM_LB_HASH_KETAMA) {
ngx_stream_upsync_chash_init(uscf, peers);
}
}
return NGX_OK;
invalid:
ngx_log_error(NGX_LOG_ERR, cycle->log, 0,
"upsync_add_peers: calloc peer failed \"%V\"",
&uscf->host);
if (peer != NULL) {
if (peer->sockaddr != NULL) {
ngx_free(peer->sockaddr);
}
ngx_free(peer);
peer = NULL;
}
return NGX_ERROR;
}
static void
ngx_stream_upsync_update_peer(ngx_stream_upstream_rr_peers_t *peers,
ngx_stream_upstream_rr_peer_t *peer,
ngx_stream_upsync_conf_t *upstream_conf,
ngx_uint_t *updated)
{
ngx_uint_t w = peers->total_weight, pw = 0;
*updated = 0;
if (peer->max_fails == upstream_conf->max_fails &&
peer->fail_timeout == upstream_conf->fail_timeout &&
peer->down == upstream_conf->down &&
peer->weight == upstream_conf->weight) {
return;
}
pw = peer->weight;
peer->max_fails = upstream_conf->max_fails;
peer->fail_timeout = upstream_conf->fail_timeout;
peer->down = upstream_conf->down;
peer->weight = upstream_conf->weight;
peer->effective_weight = upstream_conf->weight;
peer->current_weight = 0;
w = w + upstream_conf->weight - pw;
peers->weighted = (w != peers->number);
peers->total_weight = w;
*updated = 1;
return;
}
static void
ngx_stream_upsync_diff_filter(ngx_cycle_t *cycle,
ngx_stream_upsync_server_t *upsync_server,
ngx_uint_t *diff)
{
ngx_uint_t i, j, len, updated;
ngx_uint_t *flags = NULL;
ngx_array_t flag_array;
ngx_stream_upsync_ctx_t *ctx;
ngx_stream_upsync_conf_t *upstream_conf;
ngx_stream_upsync_conf_t *add_upstream, *del_upstream;
ngx_stream_upstream_rr_peer_t *peer = NULL;
ngx_stream_upstream_rr_peers_t *peers = NULL;
ngx_stream_upstream_srv_conf_t *uscf;
*diff = 0;
ctx = &upsync_server->ctx;
if (ngx_stream_upsync_need_exit()) {
return;
}
if (ngx_array_init(&ctx->add_upstream, ctx->pool, 16,
sizeof(*add_upstream)) != NGX_OK)
{
ngx_log_error(NGX_LOG_ERR, cycle->log, 0,
"upsync_diff_filter_add: alloc error");
return;
}
if (ngx_array_init(&ctx->del_upstream, ctx->pool, 16,
sizeof(*del_upstream)) != NGX_OK) {
ngx_log_error(NGX_LOG_ERR, cycle->log, 0,
"upsync_diff_filter_del: alloc error");
return;
}
uscf = upsync_server->uscf;
if (uscf->peer.data == NULL) {
return;
}
peers = (ngx_stream_upstream_rr_peers_t *)uscf->peer.data;
if (peers->number != 0) {
if (ngx_array_init(&flag_array, ctx->pool, peers->number,
sizeof(*flags)) != NGX_OK) {
ngx_log_error(NGX_LOG_ERR, cycle->log, 0,
"upsync_diff_filter: alloc error");
return;
}
ngx_memzero(flag_array.elts, sizeof(ngx_uint_t) * flag_array.nalloc);
flags = (ngx_uint_t*)flag_array.elts;
}
len = ctx->upstream_conf.nelts;
for (i = 0; i < len; i++) {
upstream_conf = (ngx_stream_upsync_conf_t *)ctx->upstream_conf.elts + i;
for (peer = peers->peer, j = 0; peer; peer = peer->next, j++) {
if (*(flags + j) == 1) {
continue;
}
if (ngx_memn2cmp(peer->name.data, upstream_conf->sockaddr,
peer->name.len,
ngx_strlen(upstream_conf->sockaddr)) == 0) {
// update peer
ngx_stream_upsync_update_peer(peers, peer, upstream_conf, &updated);
*diff |= updated;
// set flag, not to be deleted
*(flags + j) = 1;
break;
}
}
// add_upstream
if (j == peers->number) {
add_upstream = ngx_array_push(&ctx->add_upstream);
ngx_memcpy(add_upstream, upstream_conf, sizeof(*upstream_conf));
}
}
// del_upstream
for (peer = peers->peer, j = 0; peer; peer = peer->next, j++) {
if (*(flags + j) == 1) {
continue;
}
del_upstream = ngx_array_push(&ctx->del_upstream);
ngx_memzero(del_upstream, sizeof(*del_upstream));
ngx_memcpy(&del_upstream->sockaddr, peer->name.data, peer->name.len);
}
*diff |= (ctx->add_upstream.nelts > 0);
*diff |= (ctx->del_upstream.nelts > 0);
return;
}
static ngx_int_t
ngx_stream_upsync_del_peers(ngx_cycle_t *cycle,
ngx_stream_upsync_server_t *upsync_server)
{
ngx_uint_t i, n=0, w=0, len=0;
ngx_array_t *servers;
ngx_stream_upstream_server_t *server = NULL;
ngx_stream_upstream_rr_peer_t *peer = NULL, *pre_peer = NULL;
ngx_stream_upstream_rr_peer_t *del_peer = NULL, *tmp_del_peer = NULL;
ngx_stream_upstream_rr_peers_t *peers = NULL;
ngx_stream_upstream_srv_conf_t *uscf;
len = sizeof(struct sockaddr);
uscf = upsync_server->uscf;
if (ngx_stream_upsync_need_exit()) {
return NGX_OK;
}
servers = ngx_stream_upsync_servers(cycle, upsync_server, NGX_DEL);
if (servers == NULL) {
return NGX_ERROR;
}
if (servers->nelts < 1) {
ngx_log_error(NGX_LOG_ERR, cycle->log, 0,
"upsync_del_peers: no servers to delete \"%V\"", &uscf->host);
return NGX_ERROR;
}
if (uscf->peer.data == NULL) {
return NGX_ERROR;
}
peers = (ngx_stream_upstream_rr_peers_t *)uscf->peer.data;
if (peers->number <= servers->nelts) {
ngx_log_error(NGX_LOG_ERR, cycle->log, 0,
"upsync_del_peer: upstream \"%V\" cannot delete all peers",
&uscf->host);
return NGX_ERROR;
}
n = peers->number - servers->nelts;
w = peers->total_weight;
pre_peer = peers->peer;
for (peer = peers->peer; peer; peer = peer->next) {
for (i = 0; i < servers->nelts; i++) {
server = (ngx_stream_upstream_server_t *)servers->elts + i;
if (ngx_memn2cmp((u_char *) peer->sockaddr,
(u_char *) server->addrs->sockaddr, len, len) == 0)
{
if (del_peer == NULL) {
del_peer = peer;
tmp_del_peer = peer;
} else {
tmp_del_peer->next = peer;
tmp_del_peer = peer;
}
if (pre_peer == peer) {
peers->peer = peer->next;
pre_peer = peer->next;
} else {
pre_peer->next = peer->next;
}
w -= peer->weight;
break;
}
}
if (i == servers->nelts) {
pre_peer = peer;
}
}
if (tmp_del_peer) {
tmp_del_peer->next = NULL;
}
peers->single = (n == 1);
peers->number = n;
peers->weighted = (w != n);
peers->total_weight = w;
if (upsync_server->upscf->upsync_lb == NGX_STREAM_LB_HASH_KETAMA) {
ngx_stream_upsync_del_chash_peer(uscf);
}
ngx_stream_upsync_event_init(del_peer, upsync_server);
return NGX_OK;
}
static ngx_int_t
ngx_stream_upsync_replace_peers(ngx_cycle_t *cycle,
ngx_stream_upsync_server_t *upsync_server)
{
ngx_uint_t i, len, n=0, w=0;
ngx_array_t *servers;
ngx_stream_upstream_server_t *server = NULL;
ngx_stream_upstream_rr_peer_t *peer = NULL;
ngx_stream_upstream_rr_peers_t *peers = NULL;
ngx_stream_upstream_srv_conf_t *uscf;
uscf = upsync_server->uscf;
u_char *namep = NULL;
struct sockaddr *saddr = NULL;
len = sizeof(struct sockaddr);
if (uscf->peer.data == NULL) {
return NGX_ERROR;
}
peers = (ngx_stream_upstream_rr_peers_t *)uscf->peer.data;
servers = ngx_stream_upsync_servers(cycle, upsync_server, NGX_ALL);
if (servers == NULL) {
return NGX_ERROR;
}
if (servers->nelts < 1) {
ngx_log_error(NGX_LOG_ERR, cycle->log, 0,
"upsync_replace_peers: no servers to replace \"%V\"", &uscf->host);
return NGX_ERROR;
}
//tmp_peer = peers->peer;
if (peers && servers->nelts >= 1) {
n = servers->nelts;
for (i = 0; i < servers->nelts; i++) {
server = (ngx_stream_upstream_server_t *)servers->elts + i;
peer = ngx_calloc(sizeof(ngx_stream_upstream_rr_peer_t),
cycle->log);
if (peer == NULL) {
goto invalid;
}
if ((saddr = ngx_calloc(len, cycle->log)) == NULL) {
goto invalid;
}
ngx_memcpy(saddr, server->addrs->sockaddr, len);
peer->sockaddr = saddr;
if ((namep = ngx_calloc(server->addrs->name.len,
cycle->log)) == NULL) {
goto invalid;
}
ngx_memcpy(namep, server->addrs->name.data,
server->addrs->name.len);
peer->name.data = namep;
peer->socklen = server->addrs->socklen;
peer->name.len = server->addrs->name.len;
peer->max_fails = server->max_fails;
peer->fail_timeout = server->fail_timeout;
peer->down = server->down;
peer->weight = server->weight;
peer->effective_weight = server->weight;
peer->current_weight = 0;
peer->conns = 0;
peer->next = peers->peer;
peers->peer = peer;
w += server->weight;
if(i == 0) {
peer->next = NULL;
}
}
peers->single = (n == 1);
peers->number = n;
peers->weighted = (w != n);
peers->total_weight = w;
if (upsync_server->upscf->upsync_lb == NGX_STREAM_LB_HASH_KETAMA) {
ngx_stream_upsync_chash_init(uscf, NULL);
}
//ngx_pfree(cycle->pool, tmp_peer); not free for caused address invalid.
}
return NGX_OK;
invalid:
ngx_log_error(NGX_LOG_ERR, cycle->log, 0,
"upsync_init_peers: copy failed \"%V\"", &uscf->host);
return NGX_ERROR;
}
static ngx_int_t
ngx_stream_upsync_consul_parse_json(void *data)
{
u_char *p;
ngx_buf_t *buf;
ngx_int_t max_fails=2, backup=0, down=0;
ngx_str_t src, dst;
ngx_stream_upsync_ctx_t *ctx;
ngx_stream_upsync_conf_t *upstream_conf = NULL;
ngx_stream_upsync_server_t *upsync_server = data;
ctx = &upsync_server->ctx;
buf = &ctx->body;
src.len = 0, src.data = NULL;
dst.len = 0, dst.data = NULL;
cJSON *root = cJSON_Parse((char *)buf->pos);
if (root == NULL) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"consul_upsync_parse_json: root error");
return NGX_ERROR;
}
if (ngx_array_init(&ctx->upstream_conf, ctx->pool, 16,
sizeof(*upstream_conf)) != NGX_OK)
{
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"consul_upsync_parse_json: array init error");
cJSON_Delete(root);
return NGX_ERROR;
}
cJSON *server_next;
for (server_next = root->child; server_next != NULL;
server_next = server_next->next)
{
cJSON *temp1 = cJSON_GetObjectItem(server_next, "Key");
if (temp1 != NULL && temp1->valuestring != NULL) {
if (ngx_stream_upsync_check_key((u_char *)temp1->valuestring,
upsync_server->host) != NGX_OK) {
continue;
}
p = (u_char *)ngx_strrchr(temp1->valuestring, '/');
upstream_conf = ngx_array_push(&ctx->upstream_conf);
ngx_memzero(upstream_conf, sizeof(*upstream_conf));
ngx_sprintf(upstream_conf->sockaddr, "%*s", ngx_strlen(p + 1), p + 1);
}
temp1 = NULL;
if (upstream_conf == NULL) {
continue;
}
temp1 = cJSON_GetObjectItem(server_next, "Value");
if (temp1 != NULL && temp1->valuestring != NULL) {
src.data = (u_char *)temp1->valuestring;
src.len = ngx_strlen(temp1->valuestring);
if (dst.data == NULL) {
dst.data = ngx_pcalloc(ctx->pool, 1024);
} else {
ngx_memzero(dst.data, 1024);
}
dst.len = 0;
ngx_decode_base64(&dst, &src);
}
temp1 = NULL;
/* default value, server attribute */
upstream_conf->weight = 1;
upstream_conf->max_fails = 2;
upstream_conf->fail_timeout = 10;
upstream_conf->down = 0;
upstream_conf->backup = 0;
p = NULL;
if (dst.data != NULL && dst.len != 0) {
p = dst.data;
cJSON *sub_root = cJSON_Parse((char *)p);
if (sub_root == NULL) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"consul_upsync_parse_json: parse \'%s\' failed", p);
continue;
}
cJSON *sub_attribute = sub_root;
cJSON *temp1 = cJSON_GetObjectItem(sub_attribute, "weight");
if (temp1 != NULL) {
if (temp1->valuestring != NULL) {
upstream_conf->weight = ngx_atoi((u_char *)temp1->valuestring,
(size_t)ngx_strlen(temp1->valuestring));
} else if (temp1->valueint >= 0) {
upstream_conf->weight = temp1->valueint;
}
}
temp1 = NULL;
temp1 = cJSON_GetObjectItem(sub_attribute, "max_fails");
if (temp1 != NULL) {
if (temp1->valuestring != NULL) {
max_fails = ngx_atoi((u_char *)temp1->valuestring,
(size_t)ngx_strlen(temp1->valuestring));
} else if (temp1->valueint >= 0) {
max_fails = temp1->valueint;
}
}
temp1 = NULL;
temp1 = cJSON_GetObjectItem(sub_attribute, "fail_timeout");
if (temp1 != NULL){
if (temp1->valuestring != NULL) {
upstream_conf->fail_timeout = ngx_atoi((u_char *)temp1->valuestring,
(size_t)ngx_strlen(temp1->valuestring));
} else if (temp1->valueint >= 0) {
upstream_conf->fail_timeout = temp1->valueint;
}
}
temp1 = NULL;
temp1 = cJSON_GetObjectItem(sub_attribute, "down");
if (temp1 != NULL) {
if (temp1->valueint != 0) {
down = temp1->valueint;
} else if (temp1->valuestring != NULL) {
down = ngx_atoi((u_char *)temp1->valuestring,
(size_t)ngx_strlen(temp1->valuestring));
}
}
temp1 = NULL;
temp1 = cJSON_GetObjectItem(sub_attribute, "backup");
if (temp1 != NULL) {
if (temp1->valueint != 0) {
backup = temp1->valueint;
} else if (temp1->valuestring != NULL) {
backup = ngx_atoi((u_char *)temp1->valuestring,
(size_t)ngx_strlen(temp1->valuestring));
}
}
temp1 = NULL;
dst.len = 0;
cJSON_Delete(sub_root);
}
if (upstream_conf->weight <= 0) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"consul_upsync_parse_json: \"weight\" value is "
"invalid, setting default value 1");
upstream_conf->weight = 1;
}
if (max_fails < 0) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"consul_upsync_parse_json: \"max_fails\" value is "
"invalid, setting default value 2");
} else {
upstream_conf->max_fails = (ngx_uint_t)max_fails;
}
if (upstream_conf->fail_timeout < 0) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"consul_upsync_parse_json: \"fail_timeout\" value is "
"invalid, setting default value 10");
upstream_conf->fail_timeout = 10;
}
if (down != 1 && down != 0) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"consul_upsync_parse_json: \"down\" value is invalid,"
"setting default value 0");
} else {
upstream_conf->down = (ngx_uint_t)down;
}
if (backup != 1 && backup != 0) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"upsync_parse_json: \"backup\" value is invalid,"
"setting default value 0");
} else {
upstream_conf->backup = (ngx_uint_t)backup;
}
max_fails=2, backup=0, down=0;
}
cJSON_Delete(root);
return NGX_OK;
}
static ngx_int_t
ngx_stream_upsync_consul_services_parse_json(void *data)
{
ngx_buf_t *buf;
ngx_int_t attr_value;
ngx_stream_upsync_ctx_t *ctx;
ngx_stream_upsync_conf_t *upstream_conf = NULL;
ngx_stream_upsync_server_t *upsync_server = data;
ctx = &upsync_server->ctx;
buf = &ctx->body;
cJSON *root = cJSON_Parse((char *)buf->pos);
if (root == NULL) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"upsync_parse_json: root error");
return NGX_ERROR;
}
if (ngx_array_init(&ctx->upstream_conf, ctx->pool, 16,
sizeof(*upstream_conf)) != NGX_OK)
{
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"upsync_parse_json: array init error");
cJSON_Delete(root);
return NGX_ERROR;
}
cJSON *server_next;
for (server_next = root->child; server_next != NULL;
server_next = server_next->next)
{
cJSON *addr, *port, *tags, *tag_next;
size_t addr_len, port_len;
u_char port_buf[8];
addr = cJSON_GetObjectItem(server_next, "ServiceAddress");
if (addr == NULL || addr->valuestring == NULL
|| addr->valuestring[0] == '\0')
{
addr = cJSON_GetObjectItem(server_next, "Address");
if (addr == NULL || addr->valuestring == NULL) {
continue;
}
}
port = cJSON_GetObjectItem(server_next, "ServicePort");
if (port == NULL || port->valueint < 1 || port->valueint > 65535) {
continue;
}
ngx_memzero(port_buf, 8);
ngx_sprintf(port_buf, "%d", port->valueint);
addr_len = ngx_strlen(addr->valuestring);
port_len = ngx_strlen(port_buf);
if (addr_len + port_len + 2 > NGX_SOCKADDRLEN) {
continue;
}
upstream_conf = ngx_array_push(&ctx->upstream_conf);
if (upstream_conf == NULL) {
cJSON_Delete(root);
return NGX_ERROR;
}
ngx_memzero(upstream_conf, sizeof(*upstream_conf));
ngx_memcpy(upstream_conf->sockaddr, addr->valuestring, addr_len);
ngx_memcpy(upstream_conf->sockaddr + addr_len, ":", 1);
ngx_memcpy(upstream_conf->sockaddr + addr_len + 1, port_buf, port_len);
/* default value, server attribute */
upstream_conf->weight = 1;
upstream_conf->max_fails = 2;
upstream_conf->fail_timeout = 10;
upstream_conf->down = 0;
upstream_conf->backup = 0;
tags = cJSON_GetObjectItem(server_next, "ServiceTags");
if (tags == NULL) {
continue;
}
for (tag_next = tags->child; tag_next != NULL;
tag_next = tag_next->next)
{
u_char *tag = (u_char *) tag_next->valuestring;
if (tag == NULL) {
continue;
}
if (ngx_strncmp(tag, "weight=", 7) == 0) {
attr_value = ngx_atoi(tag + 7, (size_t)ngx_strlen(tag) - 7);
if (attr_value == NGX_ERROR || attr_value <= 0) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"upsync_parse_json: \"weight\" value is "
"invalid, setting default value 1");
continue;
} else {
upstream_conf->weight = attr_value;
}
}
if (ngx_strncmp(tag, "max_fails=", 10) == 0) {
attr_value = ngx_atoi(tag + 10, (size_t)ngx_strlen(tag) - 10);
if (attr_value == NGX_ERROR || attr_value < 0) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"upsync_parse_json: \"max_fails\" value is "
"invalid, setting default value 2");
continue;
} else {
upstream_conf->max_fails = attr_value;
}
}
if (ngx_strncmp(tag, "fail_timeout=", 13) == 0) {
ngx_str_t value = {ngx_strlen(tag) - 13, tag + 13};
attr_value = ngx_parse_time(&value, 1);
if (attr_value == NGX_ERROR || attr_value < 0) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"upsync_parse_json: \"fail_timeout\" value is "
"invalid, setting default value 10");
continue;
} else {
upstream_conf->fail_timeout = attr_value;
}
}
if (ngx_strncmp(tag, "down", 4) == 0 && tag[4] == '\0') {
upstream_conf->down = 1;
}
if (ngx_strncmp(tag, "backup", 6) == 0 && tag[6] == '\0') {
upstream_conf->backup = 1;
}
}
}
cJSON_Delete(root);
return NGX_OK;
}
static ngx_int_t
ngx_stream_upsync_etcd_parse_json(void *data)
{
u_char *p;
ngx_buf_t *buf;
ngx_int_t max_fails=2, backup=0, down=0;
ngx_stream_upsync_ctx_t *ctx;
ngx_stream_upsync_conf_t *upstream_conf = NULL;
ngx_stream_upsync_server_t *upsync_server = data;
ctx = &upsync_server->ctx;
buf = &ctx->body;
cJSON *root = cJSON_Parse((char *)buf->pos);
if (root == NULL) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"etcd_upsync_parse_json: root error");
return NGX_ERROR;
}
cJSON *errorCode = cJSON_GetObjectItem(root, "errorCode");
if (errorCode != NULL) {
if (errorCode->valueint == 401) { // trigger reload, we've gone too far with index
upsync_server->index = 0;
ngx_del_timer(&upsync_server->upsync_timeout_ev);
ngx_add_timer(&upsync_server->upsync_ev, 0);
}
cJSON_Delete(root);
return NGX_ERROR;
}
cJSON *action = cJSON_GetObjectItem(root, "action");
if (action != NULL) {
if (action->valuestring != NULL) {
if (ngx_memcmp(action->valuestring, "get", 3) != 0) {
upsync_server->index = 0;
ngx_del_timer(&upsync_server->upsync_timeout_ev);
ngx_add_timer(&upsync_server->upsync_ev, 0);
cJSON_Delete(root);
return NGX_ERROR;
}
}
}
action = NULL;
if (ngx_array_init(&ctx->upstream_conf, ctx->pool, 16,
sizeof(*upstream_conf)) != NGX_OK)
{
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"etcd_upsync_parse_json: array init error");
cJSON_Delete(root);
return NGX_ERROR;
}
cJSON *node = cJSON_GetObjectItem(root, "node");
if (node == NULL) {
cJSON_Delete(root);
return NGX_ERROR;
}
cJSON *nodes = cJSON_GetObjectItem(node, "nodes");
if (nodes == NULL) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"etcd_upsync_parse_json: nodes is null, no servers");
cJSON_Delete(root);
return NGX_ERROR;
}
cJSON *server_next;
for (server_next = nodes->child; server_next != NULL;
server_next = server_next->next)
{
cJSON *temp0 = cJSON_GetObjectItem(server_next, "key");
if (temp0 != NULL && temp0->valuestring != NULL) {
if (ngx_stream_upsync_check_key((u_char *)temp0->valuestring,
upsync_server->host) != NGX_OK) {
continue;
}
p = (u_char *)ngx_strrchr(temp0->valuestring, '/');
upstream_conf = ngx_array_push(&ctx->upstream_conf);
ngx_memzero(upstream_conf, sizeof(*upstream_conf));
ngx_sprintf(upstream_conf->sockaddr, "%*s", ngx_strlen(p + 1), p + 1);
}
temp0 = NULL;
/* default value, server attribute */
upstream_conf->weight = 1;
upstream_conf->max_fails = 2;
upstream_conf->fail_timeout = 10;
upstream_conf->down = 0;
upstream_conf->backup = 0;
temp0 = cJSON_GetObjectItem(server_next, "value");
if (temp0 != NULL && ngx_strlen(temp0->valuestring) != 0) {
cJSON *sub_attribute = cJSON_Parse((char *)temp0->valuestring);
if (sub_attribute == NULL) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"etcd_upsync_parse_json: \'%s\' is invalid",
temp0->valuestring);
continue;
}
cJSON *temp1 = cJSON_GetObjectItem(sub_attribute, "weight");
if (temp1 != NULL) {
if (temp1->valuestring != NULL) {
upstream_conf->weight = ngx_atoi((u_char *)temp1->valuestring,
(size_t)ngx_strlen(temp1->valuestring));
} else if (temp1->valueint >= 0) {
upstream_conf->weight = temp1->valueint;
}
}
temp1 = NULL;
temp1 = cJSON_GetObjectItem(sub_attribute, "max_fails");
if (temp1 != NULL) {
if (temp1->valuestring != NULL) {
max_fails = ngx_atoi((u_char *)temp1->valuestring,
(size_t)ngx_strlen(temp1->valuestring));
} else if (temp1->valueint >= 0) {
max_fails = temp1->valueint;
}
}
temp1 = NULL;
temp1 = cJSON_GetObjectItem(sub_attribute, "fail_timeout");
if (temp1 != NULL){
if (temp1->valuestring != NULL) {
upstream_conf->fail_timeout = ngx_atoi((u_char *)temp1->valuestring,
(size_t)ngx_strlen(temp1->valuestring));
} else if (temp1->valueint >= 0) {
upstream_conf->fail_timeout = temp1->valueint;
}
}
temp1 = NULL;
temp1 = cJSON_GetObjectItem(sub_attribute, "down");
if (temp1 != NULL) {
if (temp1->valueint != 0) {
down = temp1->valueint;
} else if (temp1->valuestring != NULL) {
down = ngx_atoi((u_char *)temp1->valuestring,
(size_t)ngx_strlen(temp1->valuestring));
}
}
temp1 = NULL;
temp1 = cJSON_GetObjectItem(sub_attribute, "backup");
if (temp1 != NULL) {
if (temp1->valueint != 0) {
backup = temp1->valueint;
} else if (temp1->valuestring != NULL) {
backup = ngx_atoi((u_char *)temp1->valuestring,
(size_t)ngx_strlen(temp1->valuestring));
}
}
temp1 = NULL;
} else {
continue;
}
if (upstream_conf->weight <= 0) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"etcd_upsync_parse_json: \"weight\" value is invalid,"
" setting default value 1");
upstream_conf->weight = 1;
}
if (max_fails < 0) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"etcd_upsync_parse_json: \"max_fails\" value is invalid,"
" setting default value 2");
} else {
upstream_conf->max_fails = (ngx_uint_t)max_fails;
}
if (upstream_conf->fail_timeout < 0) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"etcd_upsync_parse_json: \"fail_timeout\" value is "
"invalid, setting default value 10");
upstream_conf->fail_timeout = 10;
}
if (down != 1 && down != 0) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"etcd_upsync_parse_json: \"down\" value is invalid"
", setting default value 0");
} else {
upstream_conf->down = (ngx_uint_t)down;
}
if (backup != 1 && backup != 0) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"etcd_upsync_parse_json: \"backup\" value is invalid"
", setting default value 0");
} else {
upstream_conf->backup = (ngx_uint_t)backup;
}
max_fails=2, backup=0, down=0;
}
cJSON_Delete(root);
return NGX_OK;
}
static ngx_int_t
ngx_stream_upsync_check_key(u_char *key, ngx_str_t host)
{
u_char *last, *ip_p, *port_p, *s_p; // *u_p;
ngx_int_t port;
/*
u_p = (u_char *)ngx_strstr(key, host.data);
if (u_p == NULL) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"upsync_parse_json: %s is illegal, "
"dont contains subkey %V", key, &host);
return NGX_ERROR;
}
if (*(u_p + host.len) != '/' || *(u_p - 1) != '/') {
return NGX_ERROR;
}
*/
s_p = (u_char *)ngx_strrchr(key, '/');
if (s_p == NULL) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"upsync_parse_json: %s key format is illegal, "
"contains no slash ('/')", key);
return NGX_ERROR;
}
port_p = (u_char *)ngx_strchr(s_p, ':');
if (port_p == NULL) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"upsync_check_key: has no port in %s", s_p);
return NGX_ERROR;
}
ip_p = s_p + 1;
if (ngx_inet_addr(ip_p, port_p - ip_p) == INADDR_NONE) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"upsync_check_key: invalid ip in %s", s_p);
return NGX_ERROR;
}
last = ip_p + ngx_strlen(ip_p);
port = ngx_atoi(port_p + 1, last - port_p - 1);
if (port < 1 || port > 65535) {
ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0,
"upsync_check_key: invalid port in %s", s_p);
return NGX_ERROR;
}
return NGX_OK;
}
static void *
ngx_stream_upsync_servers(ngx_cycle_t *cycle,
ngx_stream_upsync_server_t *upsync_server, ngx_flag_t flag)
{
ngx_uint_t i;
ngx_addr_t *addrs;
ngx_array_t *servers; /* ngx_stream_upstream_server_t */
ngx_stream_upsync_ctx_t *ctx;
ngx_stream_upsync_conf_t *conf;
ngx_stream_upstream_server_t *server;
ctx = &upsync_server->ctx;
servers = ngx_pcalloc(ctx->pool, sizeof(ngx_array_t));
if (servers == NULL) {
ngx_log_error(NGX_LOG_ERR, cycle->log, 0,
"upsync_servers: alloc error");
return NULL;
}
if (ngx_array_init(servers, ctx->pool, 16, sizeof(*server)) != NGX_OK) {
ngx_log_error(NGX_LOG_ERR, cycle->log, 0,
"upsync_servers: alloc error");
return NULL;
}
if (flag == NGX_ADD) {
for (i = 0; i < ctx->add_upstream.nelts; i++) {
conf = (ngx_stream_upsync_conf_t *)ctx->add_upstream.elts + i;
addrs = ngx_stream_upsync_addrs(ctx->pool, conf->sockaddr);
if (addrs == NULL) {
continue;
}
server = ngx_array_push(servers);
ngx_memzero(server, sizeof(ngx_stream_upstream_server_t));
server->addrs = addrs;
server->naddrs = 1;
server->down = conf->down;
server->backup = conf->backup;
server->weight = conf->weight;
server->max_fails = conf->max_fails;
server->fail_timeout = conf->fail_timeout;
}
} else if (flag == NGX_DEL) {
for (i = 0; i < ctx->del_upstream.nelts; i++) {
conf = (ngx_stream_upsync_conf_t *)ctx->del_upstream.elts + i;
addrs = ngx_stream_upsync_addrs(ctx->pool, conf->sockaddr);
if (addrs == NULL) {
continue;
}
server = ngx_array_push(servers);
ngx_memzero(server, sizeof(ngx_stream_upstream_server_t));
server->addrs = addrs;
server->naddrs = 1;
server->down = conf->down;
server->backup = conf->backup;
server->weight = conf->weight;
server->max_fails = conf->max_fails;
server->fail_timeout = conf->fail_timeout;
}
} else {
for (i = 0; i < ctx->upstream_conf.nelts; i++) {
conf = (ngx_stream_upsync_conf_t *)ctx->upstream_conf.elts + i;
addrs = ngx_stream_upsync_addrs(ctx->pool, conf->sockaddr);
if (addrs == NULL) {
continue;
}
server = ngx_array_push(servers);
ngx_memzero(server, sizeof(ngx_stream_upstream_server_t));
server->addrs = addrs;
server->naddrs = 1;
server->down = conf->down;
server->backup = conf->backup;
server->weight = conf->weight;
server->max_fails = conf->max_fails;
server->fail_timeout = conf->fail_timeout;
}
}
return servers;
}
static void *
ngx_stream_upsync_addrs(ngx_pool_t *pool, u_char *sockaddr)
{
u_char *port_p, *p, *last, *pp;
ngx_int_t port;
ngx_addr_t *addrs;
struct sockaddr_in *sin;
p = sockaddr;
last = p + ngx_strlen(p);
port_p = ngx_strlchr(p, last, ':');
if (port_p == NULL) {
ngx_log_error(NGX_LOG_ERR, pool->log, 0,
"upsync_addrs: has no port in %s", p);
return NULL;
}
port = ngx_atoi(port_p + 1, last - port_p - 1);
if (port < 1 || port > 65535) {
ngx_log_error(NGX_LOG_ERR, pool->log, 0,
"upsync_addrs: invalid port in %s", p);
return NULL;
}
sin = ngx_pcalloc(pool, sizeof(struct sockaddr_in));
if (sin == NULL) {
return NULL;
}
sin->sin_family = AF_INET;
sin->sin_port = htons((in_port_t) port);
sin->sin_addr.s_addr = ngx_inet_addr(p, port_p - p);
if (sin->sin_addr.s_addr == INADDR_NONE) {
ngx_log_error(NGX_LOG_ERR, pool->log, 0,
"upsync_addrs: invalid ip in %s", p);
return NULL;
}
addrs = ngx_pcalloc(pool, sizeof(ngx_addr_t));
if (addrs == NULL) {
return NULL;
}
addrs->sockaddr = (struct sockaddr *) sin;
addrs->socklen = sizeof(struct sockaddr_in);
pp = ngx_pcalloc(pool, last - p);
if (pp == NULL) {
return NULL;
}
addrs->name.len = ngx_sprintf(pp, "%s", p) - pp;
addrs->name.data = pp;
return addrs;
}
static void *
ngx_stream_upsync_create_main_conf(ngx_conf_t *cf)
{
ngx_stream_upsync_main_conf_t *upmcf;
upmcf = ngx_pcalloc(cf->pool, sizeof(ngx_stream_upsync_main_conf_t));
if (upmcf == NULL) {
return NULL;
}
upmcf->upstream_num = NGX_CONF_UNSET_UINT;
upmcf->upsync_server = NGX_CONF_UNSET_PTR;
return upmcf;
}
static char *
ngx_stream_upsync_init_main_conf(ngx_conf_t *cf, void *conf)
{
ngx_uint_t i;
ngx_stream_upsync_main_conf_t *upmcf = conf;
ngx_stream_upstream_srv_conf_t **uscfp;
ngx_stream_upstream_main_conf_t *umcf;
umcf = ngx_stream_conf_get_module_main_conf(cf, ngx_stream_upstream_module);
upmcf->upsync_server = ngx_pcalloc(cf->pool,
umcf->upstreams.nelts * sizeof(ngx_stream_upsync_server_t));
if (upmcf->upsync_server == NULL) {
return NGX_CONF_ERROR;
}
upmcf->upstream_num = 0;
upsync_ctx = upmcf;
uscfp = umcf->upstreams.elts;
for (i = 0; i < umcf->upstreams.nelts; i++) {
if (ngx_stream_upsync_init_srv_conf(cf, uscfp[i], i) != NGX_OK) {
return NGX_CONF_ERROR;
}
}
return NGX_CONF_OK;
}
static void *
ngx_stream_upsync_create_srv_conf(ngx_conf_t *cf)
{
ngx_stream_upsync_srv_conf_t *upscf;
upscf = ngx_pcalloc(cf->pool, sizeof(ngx_stream_upsync_srv_conf_t));
if (upscf == NULL) {
return NULL;
}
upscf->upsync_host.len = NGX_CONF_UNSET_SIZE;
upscf->upsync_host.data = NGX_CONF_UNSET_PTR;
upscf->upsync_port = NGX_CONF_UNSET;
upscf->upsync_dump_path.len = NGX_CONF_UNSET_SIZE;
upscf->upsync_dump_path.data = NGX_CONF_UNSET_PTR;
upscf->upsync_timeout = NGX_CONF_UNSET_MSEC;
upscf->upsync_interval = NGX_CONF_UNSET_MSEC;
upscf->upsync_lb = NGX_CONF_UNSET;
upscf->strong_dependency = NGX_CONF_UNSET_UINT;
upscf->conf_file = NGX_CONF_UNSET_PTR;
upscf->upsync_type_conf = NGX_CONF_UNSET_PTR;
ngx_memzero(&upscf->conf_server, sizeof(upscf->conf_server));
return upscf;
}
static char *
ngx_stream_upsync_init_srv_conf(ngx_conf_t *cf, void *conf, ngx_uint_t num)
{
u_char *buf;
ngx_stream_upsync_server_t *upsync_server;
ngx_stream_upsync_srv_conf_t *upscf;
ngx_stream_upstream_srv_conf_t *uscf = conf;
if (uscf->srv_conf == NULL) {
return NGX_CONF_OK;
}
upscf = ngx_stream_conf_upstream_srv_conf(uscf, ngx_stream_upsync_module);
if (upscf->upsync_host.data == NGX_CONF_UNSET_PTR
&& upscf->upsync_host.len == NGX_CONF_UNSET_SIZE) {
return NGX_CONF_OK;
}
upsync_ctx->upstream_num++;
upsync_server = &upsync_ctx->upsync_server[upsync_ctx->upstream_num - 1];
if (upsync_server == NULL) {
return NGX_CONF_ERROR;
}
if (upscf->upsync_timeout == NGX_CONF_UNSET_MSEC) {
upscf->upsync_timeout = 1000 * 60 * 6;
}
if (upscf->upsync_interval == NGX_CONF_UNSET_MSEC) {
upscf->upsync_interval = 1000 * 5;
}
if (upscf->upsync_lb == NGX_CONF_UNSET) {
upscf->upsync_lb = NGX_STREAM_LB_DEFAULT;
}
if (upscf->strong_dependency == NGX_CONF_UNSET_UINT) {
upscf->strong_dependency = 0;
}
if (upscf->upsync_dump_path.len == NGX_CONF_UNSET_SIZE) {
buf = ngx_pcalloc(cf->pool,
ngx_strlen("/tmp/servers_.conf") + uscf->host.len + 1);
ngx_sprintf(buf, "/tmp/servers_%V.conf", &uscf->host);
upscf->upsync_dump_path.data = buf;
upscf->upsync_dump_path.len = ngx_strlen("/tmp/servers_.conf")
+ uscf->host.len;
}
upscf->conf_file = ngx_pcalloc(cf->pool, sizeof(ngx_open_file_t));
if (upscf->conf_file == NULL) {
return NGX_CONF_ERROR;
}
upscf->conf_file->fd = NGX_INVALID_FILE;
upscf->conf_file->name = upscf->upsync_dump_path;
upscf->conf_file->flush = NULL;
upscf->conf_file->data = NULL;
upsync_server->index = 0;
upsync_server->update_generation = 0;
upsync_server->upscf = upscf;
upsync_server->uscf = uscf;
upsync_server->host.len = uscf->host.len;
upsync_server->host.data = uscf->host.data;
return NGX_CONF_OK;
}
static ngx_int_t
ngx_stream_upsync_init_module(ngx_cycle_t *cycle)
{
ngx_uint_t i;
ngx_stream_upsync_server_t *upsync_server;
ngx_stream_upsync_srv_conf_t *upscf;
// no stream {} block found
if (upsync_ctx == NULL) {
return NGX_OK;
}
upsync_server = upsync_ctx->upsync_server;
if (ngx_stream_upsync_init_shm_mutex(cycle) != NGX_OK) {
ngx_log_error(NGX_LOG_ERR, cycle->log, 0, "upsync_init_module:"
" init shm mutex failed");
return NGX_ERROR;
}
for (i = 0; i < upsync_ctx->upstream_num; i++) {
upscf = upsync_server[i].upscf;
if (upscf->conf_file->fd != NGX_INVALID_FILE) {
ngx_close_file(upscf->conf_file->fd);
upscf->conf_file->fd = NGX_INVALID_FILE;
}
ngx_change_file_access(upscf->upsync_dump_path.data,
S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH|S_IWOTH);
}
return NGX_OK;
}
static ngx_int_t
ngx_stream_upsync_init_shm_mutex(ngx_cycle_t *cycle)
{
u_char *shared, *file;
size_t size, cl;
ngx_shm_t shm;
ngx_uint_t i;
ngx_stream_upsync_server_t *upsync_server;
upsync_server = upsync_ctx->upsync_server;
if (*stream_upsync_shared_created) {
shm.size = 128 * (*stream_upsync_shared_created);
shm.log = cycle->log;
shm.addr = (u_char *)(stream_upsync_shared_created);
shm.name.len = sizeof("ngx_upsync_shared_zone");
shm.name.data = (u_char *)"ngx_upsync_shared_zone";
ngx_shm_free(&shm);
}
/* cl should be equal to or greater than cache line size
shared created flag
upsync_accept_mutex for every upstream
*/
cl = 128;
size = cl
+ cl * upsync_ctx->upstream_num;
shm.size = size;
shm.log = cycle->log;
shm.name.len = sizeof("ngx_upsync_shared_zone");
shm.name.data = (u_char *)"ngx_upsync_shared_zone";
if (ngx_shm_alloc(&shm) != NGX_OK) {
return NGX_ERROR;
}
shared = shm.addr;
stream_upsync_shared_created = (ngx_atomic_t *)shared;
for (i = 0; i < upsync_ctx->upstream_num; i++) {
#if (NGX_HAVE_ATOMIC_OPS)
file = NULL;
#else
file = ngx_pcalloc(cycle->pool,
cycle->lock_file.len + ngx_strlen("upsync") + 3);
if (file == NULL) {
return NGX_ERROR;
}
(void) ngx_sprintf(file, "%V%s%d%Z", &ngx_cycle->lock_file, "upsync", i);
#endif
if (ngx_shmtx_create(&upsync_server[i].upsync_accept_mutex,
(ngx_shmtx_sh_t *)(shared + (i + 1) * cl), file)
!= NGX_OK)
{
return NGX_ERROR;
}
}
ngx_atomic_cmp_set(stream_upsync_shared_created, *stream_upsync_shared_created,
upsync_ctx->upstream_num);
return NGX_OK;
}
static ngx_int_t
ngx_stream_upsync_init_process(ngx_cycle_t *cycle)
{
char *conf_value = NULL;
ngx_int_t status = 0;
ngx_uint_t i, j;
ngx_pool_t *pool;
ngx_upsync_conf_t *upsync_type_conf;
ngx_stream_upsync_ctx_t *ctx;
ngx_stream_upsync_server_t *upsync_server;
// no stream {} block found
if (upsync_ctx == NULL) {
return NGX_OK;
}
upsync_server = upsync_ctx->upsync_server;
for (i = 0; i < upsync_ctx->upstream_num; i++) {
ngx_queue_init(&upsync_server[i].delete_ev);
if (upsync_server[i].upscf->strong_dependency == 0) {
continue;
}
ctx = &upsync_server[i].ctx;
ngx_memzero(ctx, sizeof(*ctx));
upsync_type_conf = upsync_server[i].upscf->upsync_type_conf;
pool = ngx_create_pool(NGX_DEFAULT_POOL_SIZE, ngx_cycle->log);
if (pool == NULL) {
ngx_log_error(NGX_LOG_ERR, cycle->log, 0,
"upsync_init_process: recv error, "
"server no enough memory");
return NGX_ERROR;
}
ctx->pool = pool;
for (j = 0; j < NGX_STREAM_RETRY_TIMES; j++) {
status = ngx_stream_upsync_get_upstream(cycle,
&upsync_server[i], &conf_value);
if (status == NGX_OK) {
break;
}
}
if (status != NGX_OK) {
ngx_log_error(NGX_LOG_ERR, cycle->log, 0,
"u
gitextract_yli_f_d8/
├── README.md
├── config
├── src/
│ ├── ngx_stream_http_parser.c
│ ├── ngx_stream_http_parser.h
│ ├── ngx_stream_json.c
│ ├── ngx_stream_json.h
│ ├── ngx_stream_upsync_module.c
│ └── ngx_stream_upsync_module.h
└── test/
├── README
├── conf-server.sh
├── t/
│ ├── lib/
│ │ ├── Test/
│ │ │ ├── Nginx/
│ │ │ │ ├── IMAP.pm
│ │ │ │ ├── POP3.pm
│ │ │ │ └── SMTP.pm
│ │ │ └── Nginx.pm
│ │ └── Time/
│ │ ├── Parse.pm
│ │ └── Zone.pm
│ └── upsync.t
└── test.sh
SYMBOL INDEX (142 symbols across 6 files)
FILE: src/ngx_stream_http_parser.c
type state (line 282) | enum state
type header_states (line 366) | enum header_states
type http_host_state (line 396) | enum http_host_state
function parse_url_char (line 481) | static enum state
function http_parser_execute (line 629) | size_t http_parser_execute (http_parser *parser,
function http_message_needs_eof (line 2082) | int
function http_should_keep_alive (line 2105) | int
type http_method (line 2125) | enum http_method
function http_parser_init (line 2131) | void
function http_parser_settings_init (line 2142) | void
type http_errno (line 2149) | enum http_errno
type http_errno (line 2155) | enum http_errno
function http_parse_host_char (line 2160) | static enum http_host_state
function http_parse_host (line 2225) | static int
function http_parser_parse_url (line 2298) | int
function http_parser_pause (line 2397) | void
function http_body_is_final (line 2411) | int
function http_parser_version (line 2416) | unsigned long
FILE: src/ngx_stream_http_parser.h
type __int8 (line 36) | typedef __int8 int8_t;
type __int16 (line 38) | typedef __int16 int16_t;
type __int32 (line 40) | typedef __int32 int32_t;
type __int64 (line 42) | typedef __int64 int64_t;
type http_parser (line 66) | typedef struct http_parser http_parser;
type http_parser_settings (line 67) | typedef struct http_parser_settings http_parser_settings;
type http_method (line 123) | enum http_method
type http_parser_type (line 131) | enum http_parser_type { HTTP_REQUEST, HTTP_RESPONSE, HTTP_BOTH }
type flags (line 135) | enum flags
type http_errno (line 194) | enum http_errno {
type http_parser (line 204) | struct http_parser {
type http_parser_settings (line 234) | struct http_parser_settings {
type http_parser_url_fields (line 246) | enum http_parser_url_fields
type http_parser_url (line 265) | struct http_parser_url {
type http_parser_type (line 288) | enum http_parser_type
type http_method (line 313) | enum http_method
type http_errno (line 316) | enum http_errno
type http_errno (line 319) | enum http_errno
type http_parser_url (line 324) | struct http_parser_url
FILE: src/ngx_stream_json.c
function cJSON_strcasecmp (line 42) | static int cJSON_strcasecmp(const char *s1,const char *s2)
function cJSON_InitHooks (line 64) | void cJSON_InitHooks(cJSON_Hooks* hooks)
function cJSON (line 77) | static cJSON *cJSON_New_Item(void)
function cJSON_Delete (line 85) | void cJSON_Delete(cJSON *c)
function cJSON (line 256) | cJSON *cJSON_ParseWithOpts(const char *value,const char **return_parse_e...
function cJSON (line 272) | cJSON *cJSON_Parse(const char *value) {return cJSON_ParseWithOpts(value,...
function cJSON_GetArraySize (line 503) | int cJSON_GetArraySize(cJSON *array) {cJSON *c=array->child;int...
function cJSON (line 504) | cJSON *cJSON_GetArrayItem(cJSON *array,int item) {cJSON *c=array->chi...
function cJSON (line 505) | cJSON *cJSON_GetObjectItem(cJSON *object,const char *string) {cJSON *c=o...
function suffix_object (line 508) | static void suffix_object(cJSON *prev,cJSON *item) {prev->next=item;item...
function cJSON (line 510) | static cJSON *create_reference(cJSON *item) {cJSON *ref=cJSON_New_Item()...
function cJSON_AddItemToArray (line 513) | void cJSON_AddItemToArray(cJSON *array, cJSON *item) {cJSON *c=ar...
function cJSON_AddItemToObject (line 514) | void cJSON_AddItemToObject(cJSON *object,const char *string,cJSON *ite...
function cJSON_AddItemReferenceToArray (line 515) | void cJSON_AddItemReferenceToArray(cJSON *array, cJSON *item) {cJSO...
function cJSON_AddItemReferenceToObject (line 516) | void cJSON_AddItemReferenceToObject(cJSON *object,const char *string,cJS...
function cJSON (line 518) | cJSON *cJSON_DetachItemFromArray(cJSON *array,int which) {cJSON *c=arr...
function cJSON_DeleteItemFromArray (line 523) | void cJSON_DeleteItemFromArray(cJSON *array,int which) {cJSON_Delete...
function cJSON (line 524) | cJSON *cJSON_DetachItemFromObject(cJSON *object,const char *string) {int...
function cJSON_DeleteItemFromObject (line 525) | void cJSON_DeleteItemFromObject(cJSON *object,const char *string) {cJS...
function cJSON_ReplaceItemInArray (line 528) | void cJSON_ReplaceItemInArray(cJSON *array,int which,cJSON *newitem) ...
function cJSON_ReplaceItemInObject (line 531) | void cJSON_ReplaceItemInObject(cJSON *object,const char *string,cJSON ...
function cJSON (line 534) | cJSON *cJSON_CreateNull(void) {cJSON *item=cJSON_New_Item();if(item)...
function cJSON (line 535) | cJSON *cJSON_CreateTrue(void) {cJSON *item=cJSON_New_Item();if(item)...
function cJSON (line 536) | cJSON *cJSON_CreateFalse(void) {cJSON *item=cJSON_New_Item();if(item...
function cJSON (line 537) | cJSON *cJSON_CreateBool(int b) {cJSON *item=cJSON_New_Item();if(item...
function cJSON (line 538) | cJSON *cJSON_CreateNumber(double num) {cJSON *item=cJSON_New_Item();if...
function cJSON (line 539) | cJSON *cJSON_CreateString(const char *string) {cJSON *item=cJSON_New_Ite...
function cJSON (line 540) | cJSON *cJSON_CreateArray(void) {cJSON *item=cJSON_New_Item();if(item...
function cJSON (line 541) | cJSON *cJSON_CreateObject(void) {cJSON *item=cJSON_New_Item();if(ite...
function cJSON (line 544) | cJSON *cJSON_CreateIntArray(int *numbers,int count) {int i;cJSON *n=0...
function cJSON (line 545) | cJSON *cJSON_CreateFloatArray(float *numbers,int count) {int i;cJSON *...
function cJSON (line 546) | cJSON *cJSON_CreateDoubleArray(double *numbers,int count) {int i;cJSON ...
function cJSON (line 547) | cJSON *cJSON_CreateStringArray(const char **strings,int count) {int i;cJ...
function cJSON (line 550) | cJSON *cJSON_Duplicate(cJSON *item,int recurse)
FILE: src/ngx_stream_json.h
type cJSON (line 43) | typedef struct cJSON {
type cJSON_Hooks (line 56) | typedef struct cJSON_Hooks {
FILE: src/ngx_stream_upsync_module.c
type ngx_stream_conf_client (line 15) | typedef struct {
type ngx_stream_upsync_conf_t (line 25) | typedef struct {
type ngx_int_t (line 42) | typedef ngx_int_t (*ngx_stream_upsync_packet_init_pt)
type ngx_int_t (line 44) | typedef ngx_int_t (*ngx_stream_upsync_packet_parse_pt)
type ngx_upsync_conf_t (line 50) | typedef struct {
type ngx_stream_upsync_ctx_t (line 64) | typedef struct {
type ngx_stream_upsync_srv_conf_t (line 79) | typedef struct {
type ngx_stream_upsync_server_t (line 101) | typedef struct {
type ngx_delay_event_t (line 124) | typedef struct {
type ngx_stream_upsync_main_conf_t (line 136) | typedef struct {
type ngx_stream_http_state (line 144) | typedef struct {
function ngx_upsync_conf_t (line 526) | static ngx_upsync_conf_t *
function ngx_stream_upsync_process (line 638) | static void
function ngx_int_t (line 725) | static ngx_int_t
function ngx_int_t (line 780) | static ngx_int_t
function ngx_stream_upsync_update_peer (line 899) | static void
function ngx_stream_upsync_diff_filter (line 935) | static void
function ngx_int_t (line 1038) | static ngx_int_t
function ngx_int_t (line 1138) | static ngx_int_t
function ngx_int_t (line 1240) | static ngx_int_t
function ngx_int_t (line 1449) | static ngx_int_t
function ngx_int_t (line 1592) | static ngx_int_t
function ngx_int_t (line 1819) | static ngx_int_t
type sockaddr_in (line 1971) | struct sockaddr_in
type sockaddr_in (line 1990) | struct sockaddr_in
type sockaddr (line 2010) | struct sockaddr
type sockaddr_in (line 2011) | struct sockaddr_in
function ngx_int_t (line 2181) | static ngx_int_t
function ngx_int_t (line 2216) | static ngx_int_t
function ngx_int_t (line 2291) | static ngx_int_t
function ngx_int_t (line 2372) | static ngx_int_t
function ngx_stream_upsync_begin_handler (line 2415) | static void
function ngx_stream_upsync_connect_handler (line 2453) | static void
function ngx_stream_upsync_send_handler (line 2508) | static void
function ngx_stream_upsync_recv_handler (line 2603) | static void
function ngx_stream_upsync_send_empty_handler (line 2711) | static void
function ngx_stream_upsync_recv_empty_handler (line 2718) | static void
function ngx_int_t (line 2725) | static ngx_int_t
function ngx_int_t (line 2788) | static ngx_int_t
function ngx_int_t (line 2852) | static ngx_int_t
function ngx_int_t (line 2942) | static ngx_int_t
function ngx_stream_upsync_event_init (line 3073) | static void
function ngx_stream_upsync_del_delay_delete (line 3105) | static void
function ngx_stream_upsync_strnlen (line 3173) | static size_t
function ngx_strlncat (line 3187) | static size_t
function ngx_int_t (line 3209) | static ngx_int_t
function ngx_stream_http_status (line 3232) | static int
function ngx_stream_http_header_field_cb (line 3247) | static int
function ngx_stream_http_header_value_cb (line 3271) | static int
function ngx_stream_http_body (line 3291) | static int
function ngx_stream_upsync_timeout_handler (line 3312) | static void
function ngx_stream_upsync_clean_event (line 3331) | static void
function ngx_int_t (line 3387) | static ngx_int_t
function ngx_stream_upsync_clear_all_events (line 3400) | static void
function ngx_int_t (line 3465) | static ngx_int_t
function ngx_stream_conf_client (line 3521) | static ngx_stream_conf_client *
function ngx_int_t (line 3573) | static ngx_int_t
function ngx_stream_client_destroy (line 3586) | static void
function ngx_int_t (line 3596) | static ngx_int_t
function ngx_int_t (line 3644) | static ngx_int_t
function ngx_stream_upsync_show_send (line 3706) | static void
function ngx_stream_upsync_show_upstream (line 3759) | static void
function ngx_stream_upsync_show (line 3802) | static void
FILE: src/ngx_stream_upsync_module.h
type ngx_stream_upstream_chash_point_t (line 57) | typedef struct {
type ngx_stream_upstream_chash_points_t (line 63) | typedef struct {
type ngx_stream_upstream_hash_srv_conf_t (line 69) | typedef struct {
function ngx_stream_upsync_chash_cmp_points (line 85) | static int ngx_libc_cdecl
function ngx_int_t (line 105) | static ngx_int_t
function ngx_int_t (line 250) | static ngx_int_t
Condensed preview — 18 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (326K chars).
[
{
"path": "README.md",
"chars": 13486,
"preview": "Name\n====\n\nnginx-stream-upsync-module - Nginx C module, sync upstreams from consul or others, dynamically modify backend"
},
{
"path": "config",
"chars": 691,
"preview": "ngx_addon_name=ngx_stream_upsync_module\n\nngx_feature_libs=\"-lm\"\n\nngx_module_incs=$ngx_addon_dir/src\n\n_STREAM_UPSYNC_SRCS"
},
{
"path": "src/ngx_stream_http_parser.c",
"chars": 69892,
"preview": "/* Based on src/http/ngx_http_parse.c from NGINX copyright Igor Sysoev\n *\n * Additional changes are licensed under the s"
},
{
"path": "src/ngx_stream_http_parser.h",
"chars": 12673,
"preview": "/* Copyright Joyent, Inc. and other Node contributors. All rights reserved.\n *\n * Permission is hereby granted, free of "
},
{
"path": "src/ngx_stream_json.c",
"chars": 21815,
"preview": "/*\n Copyright (c) 2009 Dave Gamble\n\n Permission is hereby granted, free of charge, to any person obtaining a copy\n of"
},
{
"path": "src/ngx_stream_json.h",
"chars": 6749,
"preview": "/*\n Copyright (c) 2009 Dave Gamble\n \n Permission is hereby granted, free of charge, to any person obtaining a copy\n o"
},
{
"path": "src/ngx_stream_upsync_module.c",
"chars": 116903,
"preview": "/*\n * Copyright (C) 2015 Weibo Group Holding Limited\n * Copyright (C) 2015 Xiaokai Wang (xiaokai.wang@live.com)\n */\n\n\n#i"
},
{
"path": "src/ngx_stream_upsync_module.h",
"chars": 10919,
"preview": "#ifndef _NGX_HTTP_UPSYNC_MODELE_H_INCLUDED_\n#define _NGX_HTTP_UPSYNC_MODELE_H_INCLUDED_\n\n\n#include <ngx_core.h>\n#include"
},
{
"path": "test/README",
"chars": 9889,
"preview": "NAME\n Test::Nginx - Testing modules for Nginx C module development\n\nDESCRIPTION\n This distribution provides two te"
},
{
"path": "test/conf-server.sh",
"chars": 125,
"preview": "#!/bin/sh\n\n/usr/local/bin/consul agent -server -bootstrap -data-dir=/tmp/consul\n\n#/usr/local/bin/etcd --peers 127.0.0.1:"
},
{
"path": "test/t/lib/Test/Nginx/IMAP.pm",
"chars": 1965,
"preview": "package Test::Nginx::IMAP;\n\n# (C) Maxim Dounin\n\n# Module for nginx imap tests.\n\n########################################"
},
{
"path": "test/t/lib/Test/Nginx/POP3.pm",
"chars": 1959,
"preview": "package Test::Nginx::POP3;\n\n# (C) Maxim Dounin\n\n# Module for nginx pop3 tests.\n\n########################################"
},
{
"path": "test/t/lib/Test/Nginx/SMTP.pm",
"chars": 2445,
"preview": "package Test::Nginx::SMTP;\n\n# (C) Maxim Dounin\n\n# Module for nginx smtp tests.\n\n########################################"
},
{
"path": "test/t/lib/Test/Nginx.pm",
"chars": 13784,
"preview": "package Test::Nginx;\n\n# (C) Maxim Dounin\n\n# Generic module for nginx tests.\n\n###########################################"
},
{
"path": "test/t/lib/Time/Parse.pm",
"chars": 5919,
"preview": "# Date::Parse\n#\n# Copyright (c) 1995 Graham Barr. All rights reserved. This program is free\n# software; you can redistri"
},
{
"path": "test/t/lib/Time/Zone.pm",
"chars": 7056,
"preview": "\npackage Time::Zone;\n\n=head1 NAME\n\nTime::Zone -- miscellaneous timezone manipulations routines\n\n=head1 SYNOPSIS\n\n\tuse Ti"
},
{
"path": "test/t/upsync.t",
"chars": 13755,
"preview": "use warnings;\nuse strict;\n\nuse Test::More;\n\nBEGIN { use FindBin; chdir($FindBin::Bin); }\n\n\nuse lib 'lib';\nuse File::Path"
},
{
"path": "test/test.sh",
"chars": 89,
"preview": "#!/bin/sh\n\nTEST_NGINX_USE_HUP=1 TEST_NGINX_BINARY=/usr/local/nginx/sbin/nginx prove -r t\n"
}
]
About this extraction
This page contains the full source code of the xiaokai-wang/nginx-stream-upsync-module GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 18 files (302.8 KB), approximately 85.1k tokens, and a symbol index with 142 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.