[ { "path": ".gitattributes", "content": "*.t linguist-language=Text\n" }, { "path": ".gitignore", "content": "*.mobi\ngenmobi.sh\n.libs\n*.swp\n*.slo\n*.la\n*.swo\n*.lo\n*~\n*.o\nprint.txt\n.rsync\n*.tar.gz\ndist\nbuild[789]\nbuild\nbuild10\ntags\nupdate-readme\n*.tmp\ngo\nt/t.sh\nreleng\nreset\n*.t_\nctags\nsrc/stream.h\nnginx\nkeepalive\nreindex\nsrc/module.[ch]\nall\nt/servroot/\nbuildroot/\nbuild1[0-9]\nre\n*.plist\nMakefile\nsrc/script.[ch]\nsrc/parse.[ch]\nsrc/util.[ch]\n*.patch\n" }, { "path": ".travis.yml", "content": "sudo: required\ndist: focal\n\nos: linux\n\nlanguage: c\n\ncompiler:\n - gcc\n\naddons:\n apt:\n packages:\n - axel\n - cpanminus\n - libgd-dev\n - libpcre3-dev\n\ncache:\n apt: true\n\nenv:\n global:\n - JOBS=3\n - NGX_BUILD_JOBS=$JOBS\n - LUAJIT_PREFIX=/opt/luajit21\n - LUAJIT_LIB=$LUAJIT_PREFIX/lib\n - LUAJIT_INC=$LUAJIT_PREFIX/include/luajit-2.1\n - LUA_INCLUDE_DIR=$LUAJIT_INC\n - LD_LIBRARY_PATH=$LUAJIT_LIB:$LD_LIBRARY_PATH\n jobs:\n - NGINX_VERSION=1.21.4\n - NGINX_VERSION=1.25.1 NGX_EXTRA_OPT=--without-pcre2\n\nbefore_install:\n - sudo cpanm --notest Test::Nginx > build.log 2>&1 || (cat build.log && exit 1)\n\ninstall:\n - git clone https://github.com/openresty/openresty.git ../openresty\n - git clone https://github.com/openresty/no-pool-nginx.git ../no-pool-nginx\n - git clone https://github.com/openresty/openresty-devel-utils.git\n - git clone https://github.com/openresty/lua-nginx-module.git ../lua-nginx-module\n - git clone https://github.com/openresty/echo-nginx-module.git ../echo-nginx-module\n - git clone https://github.com/simpl/ngx_devel_kit.git ../ndk-nginx-module\n - git clone https://github.com/openresty/lua-resty-core.git ../lua-resty-core\n - git clone https://github.com/openresty/lua-resty-lrucache.git ../lua-resty-lrucache\n - git clone -b v2.1-agentzh https://github.com/openresty/luajit2.git luajit2\n - git clone https://github.com/openresty/sregex.git\n\nscript:\n - cd sregex && sudo make PREFIX=/usr install && cd ..\n - cd luajit2/\n - make -j$JOBS CCDEBUG=-g Q= PREFIX=$LUAJIT_PREFIX CC=$CC XCFLAGS='-DLUA_USE_APICHECK -DLUA_USE_ASSERT -msse4.2' > build.log 2>&1 || (cat build.log && exit 1)\n - sudo make install PREFIX=$LUAJIT_PREFIX > build.log 2>&1 || (cat build.log && exit 1)\n - cd ..\n - export PATH=$PWD/work/nginx/sbin:$PWD/openresty-devel-utils:$PATH\n - sh util/build.sh $NGINX_VERSION > build.log 2>&1 || (cat build.log && exit 1)\n - nginx -V\n - prove -r t\n" }, { "path": "README.markdown", "content": "Name\n====\n\nngx_replace_filter - Streaming regular expression replacement in response bodies.\n\n*This module is not distributed with the Nginx source.* See [the installation instructions](#installation).\n\nTable of Contents\n=================\n\n* [Name](#name)\n* [Status](#status)\n* [Synopsis](#synopsis)\n* [Description](#description)\n* [Directives](#directives)\n * [replace_filter](#replace_filter)\n * [replace_filter_types](#replace_filter_types)\n * [replace_filter_max_buffered_size](#replace_filter_max_buffered_size)\n * [replace_filter_last_modified](#replace_filter_last_modified)\n * [replace_filter_skip](#replace_filter_skip)\n* [Installation](#installation)\n* [Trouble Shooting](#trouble-shooting)\n* [TODO](#todo)\n* [Community](#community)\n * [English Mailing List](#english-mailing-list)\n * [Chinese Mailing List](#chinese-mailing-list)\n* [Bugs and Patches](#bugs-and-patches)\n* [Author](#author)\n* [Copyright and License](#copyright-and-license)\n* [See Also](#see-also)\n\nStatus\n======\n\nThis module is already quite usable though still at the early phase of development\nand is considered experimental.\n\nSynopsis\n========\n\n```nginx\n location /t {\n default_type text/html;\n echo abc;\n replace_filter 'ab|abc' X;\n }\n\n location / {\n # proxy_pass/fastcgi_pass/...\n\n # caseless global substitution:\n replace_filter '\\d+' 'blah blah' 'ig';\n replace_filter_types text/plain text/css;\n }\n\n location /a {\n # proxy_pass/fastcgi_pass/root/...\n\n # remove line-leading spaces and line-trailing spaces,\n # as well as blank lines:\n replace_filter '^\\s+|\\s+$' '' g;\n }\n\n location /b {\n # proxy_pass/fastcgi_pass/root/...\n\n # only remove line-leading spaces and line-trailing spaces:\n replace_filter '^[ \\f\\t]+|[ \\f\\t]+$' '' g;\n }\n\n location ~ '\\.cpp$' {\n # proxy_pass/fastcgi_pass/root/...\n\n replace_filter_types text/plain;\n\n # skip C/C++ string literals:\n replace_filter \"'(?:\\\\\\\\[^\\n]|[^'\\n])*'\" $& g;\n replace_filter '\"(?:\\\\\\\\[^\\n]|[^\"\\n])*\"' $& g;\n\n # remove all those ugly C/C++ comments:\n replace_filter '/\\*.*?\\*/|//[^\\n]*' '' g;\n }\n```\n\nDescription\n===========\n\nThis Nginx output filter module tries to do regular expression substitutions in\na non-buffered manner wherever possible.\n\nThis module does *not* use traditional backtracking regular expression engines like PCRE, rather,\nit uses the new [sregex](https://github.com/agentzh/sregex) library implemented by the author himself, which was designed with streaming processing in mind from the very beginning:\n\nA good common subset of Perl 5 regular expressions is supported by `sregex`. For the complete\nfeature list, check out sregex's documentation:\n\nhttps://github.com/agentzh/sregex#syntax-supported\n\nResponse body data is only buffered when absolutely necessary, like facing an incomplete capture that belongs to a possible match near the data chunk boundaries.\n\n[Back to TOC](#table-of-contents)\n\nDirectives\n==========\n\n[Back to TOC](#table-of-contents)\n\nreplace_filter\n--------------\n**syntax:** *replace_filter <regex> <replace>*\n\n**syntax:** *replace_filter <regex> <replace> <options>*\n\n**default:** *no*\n\n**context:** *http, server, location, location if*\n\n**phase:** *output body filter*\n\nSpecifies the regex pattern and text to be replaced, with optional regex flags.\n\nBy default, the filter stops matching after the first match is found. This behavior can be changed by specifying the `g` regex option.\n\nThe following regex options are supported:\n\n* `g`\n\n for global search and substitution (default off)\n* `i`\n\n for case-insensitive matching (default off)\n\nMultiple options can be combined in a single string argument, for example:\n\n```nginx\n replace_filter hello hiya ig;\n```\n\nNginx variables can be interpolated into the text to be replaced, for example:\n\n```nginx\n replace_filter \\w+ \"[$foo,$bar]\";\n```\n\nIf you want to use the literal dollar sign character (`$`), use the `$$` sequence for that,\nfor instance:\n\n```nginx\n replace_filter \\w \"$$\";\n```\n\nUse of submatch capturing variables like `$&`, `$1`, `$2`, and etc are also supported, for example,\n\n```nginx\n replace_filter [bc]|d [$&-$1-$2] g;\n```\n\nThe semantics of the submatch capturing variables is exactly the same as in the Perl 5 language.\n\nMultiple `replace_filter` directives in the same scope is also supported.\nAll the patterns will be applied at the same time as in a tokenizer.\nWe will *not* use the longest token match semantics, but rather, patterns will be prioritized according to their order in\nthe configure file.\n\nHere is an example for removing all the C/C++ comments from a C/C++ source code file:\n\n```nginx\n replace_filter \"'(?:\\\\\\\\[^\\n]|[^'\\n])*'\" $& g;\n replace_filter '\"(?:\\\\\\\\[^\\n]|[^\"\\n])*\"' $& g;\n replace_filter '/\\*.*?\\*/|//[^\\n]*' '' g;\n```\n\nWhen the `Content-Encoding` response header is not empty (like `gzip`), the response\nbody will always remain intact. So usually you want to disable the gzip compression\nin your backend servers' responses by adding the following line to your `nginx.conf`\nif you are the ngx_proxy module:\n\n```nginx\n proxy_set_header Accept-Encoding '';\n```\n\nYour responses can still be gzip compressed on the Nginx server level though.\n\n[Back to TOC](#table-of-contents)\n\nreplace_filter_types\n--------------------\n\n**syntax:** *replace_filter_types <mime-type> ...*\n\n**default:** *replace_filter_types text/html*\n\n**context:** *http, server, location, location if*\n\n**phase:** *output body filter*\n\nSpecify one or more MIME types (in the `Content-Type` response header) to be processed.\n\nBy default, only `text/html` typed responses are processed.\n\n[Back to TOC](#table-of-contents)\n\nreplace_filter_max_buffered_size\n---------------------------------\n**syntax:** *replace_filter_max_buffered_size <size>*\n\n**default:** *replace_filter_max_buffered_size 8k*\n\n**context:** *http, server, location, location if*\n\n**phase:** *output body filter*\n\nLimits the total size of the data buffered by the module at runtime. Default to `8k`.\n\nWhen the limit is reached, `replace_filter` will immediately stop processing and\nleave all the remaining response body data intact.\n\n[Back to TOC](#table-of-contents)\n\nreplace_filter_last_modified\n----------------------------\n\n**syntax:** *replace_filter_last_modifiled keep | clear*\n\n**default:** *replace_filter_last_modified clear*\n\n**context:** *http, server, location, location if*\n\n**phase:** *output body filter*\n\nControls how to deal with the existing Last-Modified response header.\n\nBy default, this module will clear the `Last-Modified` response header if there is any. You can specify\n\n```nginx\n replace_filter_last_modified keep;\n```\n\nto always keep the original `Last-Modified` response header.\n\n[Back to TOC](#table-of-contents)\n\nreplace_filter_skip\n-------------------\n\n**syntax:** *replace_filter_skip $var*\n\n**default:** *no*\n\n**context:** *http, server, location, location if*\n\n**phase:** *output header filter*\n\nThis directive controls whether to skip all the `replace_filter` rules on a per-request basis.\n\nBoth constant values or strings containing NGINX variables are supported.\n\nWhen the value is evaluated to an empty value (\"\") or the value \"0\" in the request output header phase, no `replace_filter` rules will be skipped for the current request. Otherwise all the `replace_filter` rules will be skipped for the current request.\n\nBelow is a trivial example for this:\n\n```nginx\nset $skip '';\nlocation /t {\n content_by_lua '\n ngx.var.skip = 1\n ngx.say(\"abcabd\")\n ';\n replace_filter_skip $skip;\n replace_filter abcabd X;\n}\n```\n\n[Back to TOC](#table-of-contents)\n\nInstallation\n============\n\nYou need to install the sregex library first:\n\nhttps://github.com/agentzh/sregex\n\nAnd then rebuild your Nginx like this:\n\n```bash\n ./configure --add-module=/path/to/replace-filter-nginx-module\n```\n\nIf sregex is not installed to the default prefix (i.e., `/usr/local`), then\nyou should specify the locations of your sregex installation via\nthe `SREGEX_INC` and `SREGEX_LIB` environments before running the\n`./configure` script, as in\n\n```bash\n export SREGEX_INC=/opt/sregex/include\n export SREGEX_LIB=/opt/sregex/lib\n```\n\nassuming that your sregex is installed to the prefix `/opt/sregex`.\n\nStarting from NGINX 1.9.11, you can also compile this module as a dynamic module, by using the `--add-dynamic-module=PATH` option instead of `--add-module=PATH` on the\n`./configure` command line above. And then you can explicitly load the module in your `nginx.conf` via the [load_module](http://nginx.org/en/docs/ngx_core_module.html#load_module)\ndirective, for example,\n\n```nginx\nload_module /path/to/modules/ngx_http_replace_filter_module.so;\n```\n\n[Back to TOC](#table-of-contents)\n\nTrouble Shooting\n================\n\n* If you are seeing the error \"error while loading shared libraries: libsregex.so.0: cannot open shared object file: No such file or directory\"\nwhile starting nginx, then it means that the installation path of your libsregex library\nis not in your system's default library search path. You can solve this issue by passing the option `--with-ld-opt='-Wl,-rpath,/usr/local/lib'` to nginx's `./configure` command. Alternatively, you can just add the path of your libsregex.so.0 to the `LD_LIBRARY_PATH` environment value before starting your nginx server.\n\n[Back to TOC](#table-of-contents)\n\nTODO\n====\n\n* optimize the special case for verbatim substitutions, i.e., `replace_filter $&;`.\n* implement the `replace_filter_skip $var` directive to control whether to enable the filter on the fly.\n* reduce the amount of data that has to be buffered for when an partial match is already found.\n* recycle the memory blocks used to buffer the pending capture data and \"complex values\" for replacement.\n* allow use of inlined Lua code as the `replacement` argument of the `replace_filter` directive to generate the text to be replaced on-the-fly.\n\n[Back to TOC](#table-of-contents)\n\nCommunity\n=========\n\n[Back to TOC](#table-of-contents)\n\nEnglish Mailing List\n--------------------\n\nThe [openresty-en](https://groups.google.com/group/openresty-en) mailing list is for English speakers.\n\n[Back to TOC](#table-of-contents)\n\nChinese Mailing List\n--------------------\n\nThe [openresty](https://groups.google.com/group/openresty) mailing list is for Chinese speakers.\n\n[Back to TOC](#table-of-contents)\n\nBugs and Patches\n================\n\nPlease submit bug reports, wishlists, or patches by\n\n1. creating a ticket on the [GitHub Issue Tracker](http://github.com/agentzh/replace-filter-nginx-module/issues),\n1. or posting to the [OpenResty community](http://wiki.nginx.org/HttpLuaModule#Community).\n\n[Back to TOC](#table-of-contents)\n\nAuthor\n======\n\nYichun \"agentzh\" Zhang (章亦春) , OpenResty Inc.\n\n[Back to TOC](#table-of-contents)\n\nCopyright and License\n=====================\n\nThis module is licensed under the BSD license.\n\nCopyright (C) 2012-2017, by Yichun \"agentzh\" Zhang (章亦春), OpenResty Inc.\n\nAll rights reserved.\n\nRedistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:\n\n* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.\n\n* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.\n\nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n\n[Back to TOC](#table-of-contents)\n\nSee Also\n========\n\n* agentzh's sregex library: https://github.com/agentzh/sregex\n* The standard ngx_sub_filter module: http://nginx.org/en/docs/http/ngx_http_sub_module.html\n* Slides for my talk \"sregex: matching Perl 5 regexes on data streams\": http://agentzh.org/misc/slides/yapc-na-2013-sregex.pdf\n[Back to TOC](#table-of-contents)\n\n" }, { "path": "config", "content": "ngx_feature=\"agentzh's sregex library\"\nngx_feature_libs=\"-lsregex\"\nngx_feature_name=\nngx_feature_run=no\nngx_feature_incs=\"#include \"\nngx_feature_path=\nngx_feature_test=\"sre_regex_t *re;\n sre_program_t *prog;\n sre_pool_t *pool;\n u_char s[] = {'a', 'b', 'c'};\n sre_int_t rc, err_offset, ovector;\n sre_int_t *pending_matched;\n sre_uint_t ncaps;\n sre_vm_pike_ctx_t *pctx;\n pool = sre_create_pool(1024);\n re = sre_regex_parse(pool, s, &ncaps, 0, &err_offset);\n prog = sre_regex_compile(pool, re);\n pctx = sre_vm_pike_create_ctx(pool, prog, &ovector, 0);\n rc = sre_vm_pike_exec(pctx, s, 32, 1, &pending_matched);\n sre_destroy_pool(pool)\"\n\nif [ -n \"$SREGEX_INC\" -o -n \"$SREGEX_LIB\" ]; then\n # explicitly set sregex lib path\n ngx_feature=\"agentzh's sregex library in $SREGEX_LIB and $SREGEX_INC (specified by the SREGEX_LIB and SREGEX_INC env)\"\n ngx_feature_path=\"$SREGEX_INC\"\n if [ $NGX_RPATH = YES ]; then\n ngx_feature_libs=\"-R$SREGEX_LIB -L$SREGEX_LIB -lsregex\"\n else\n ngx_feature_libs=\"-L$SREGEX_LIB -lsregex\"\n fi\n\n . auto/feature\n\n if [ $ngx_found = no ]; then\n cat << END\n $0: error: ngx_http_replace_filter_module requires agentzh's sregex library and SREGEX_LIB is defined as $SREGEX_LIB and SREGEX_INC (path for sregex/sregex.h) $SREGEX_INC, but we cannot find sregex there.\nEND\n exit 1\n fi\n\nelse\n # auto-discovery\n ngx_feature=\"agentzh's sregex library\"\n ngx_feature_libs=\"-lsregex\"\n . auto/feature\n\n if [ $ngx_found = no ]; then\n # default installation prefix in sregex\n ngx_feature=\"agentzh's sregex library in /usr/local/\"\n ngx_feature_path=\"/usr/local/include\"\n if [ $NGX_RPATH = YES ]; then\n ngx_feature_libs=\"-R/usr/local/lib -L/usr/local/lib -lsregex\"\n else\n ngx_feature_libs=\"-L/usr/local/lib -lsregex\"\n fi\n . auto/feature\n fi\nfi\n\nif [ $ngx_found = no ]; then\n cat << END\n $0: error: ngx_http_replace_filter_module requires agentzh's sregex library.\nEND\n exit 1\nfi\n\nREPLACE_FILTER_SRCS=\"$ngx_addon_dir/src/ngx_http_replace_filter_module.c \\\n $ngx_addon_dir/src/ngx_http_replace_script.c \\\n $ngx_addon_dir/src/ngx_http_replace_parse.c \\\n $ngx_addon_dir/src/ngx_http_replace_util.c\"\nREPLACE_FILTER_DEPS=\"$ngx_addon_dir/src/ngx_http_replace_filter_module.h \\\n $ngx_addon_dir/src/ngx_http_replace_script.h \\\n $ngx_addon_dir/src/ngx_http_replace_parse.h \\\n $ngx_addon_dir/src/ngx_http_replace_util.h\"\n\nngx_addon_name=ngx_http_replace_filter_module\nif test -n \"$ngx_module_link\"; then\n ngx_module_type=HTTP_AUX_FILTER\n ngx_module_name=$ngx_addon_name\n ngx_module_srcs=\"$REPLACE_FILTER_SRCS\"\n ngx_module_deps=\"$REPLACE_FILTER_DEPS\"\n ngx_module_incs=$ngx_feature_path\n ngx_module_libs=$ngx_feature_libs\n\n . auto/module\n\nelse\n HTTP_AUX_FILTER_MODULES=\"$HTTP_AUX_FILTER_MODULES $ngx_addon_name\"\n NGX_ADDON_SRCS=\"$NGX_ADDON_SRCS $REPLACE_FILTER_SRCS\"\n NGX_ADDON_DEPS=\"$NGX_ADDON_DEPS $REPLACE_FILTER_DEPS\"\n CORE_INCS=\"$CORE_INCS $ngx_feature_path\"\n CORE_LIBS=\"$CORE_LIBS $ngx_feature_libs\"\nfi\n" }, { "path": "src/ddebug.h", "content": "#ifndef DDEBUG_H\n#define DDEBUG_H\n\n\n#include \n#include \n#include \n\n\n#if defined(DDEBUG) && (DDEBUG)\n\n# if (NGX_HAVE_VARIADIC_MACROS)\n\n# define dd(...) fprintf(stderr, \"replace_filter *** %s: \", __func__); \\\n fprintf(stderr, __VA_ARGS__); \\\n fprintf(stderr, \" at %s line %d.\\n\", __FILE__, __LINE__)\n\n# else\n\n#include \n#include \n\n#include \n\nstatic void dd(const char * fmt, ...) {\n}\n\n# endif\n\n# if DDEBUG > 1\n\n# define dd_enter() dd_enter_helper(r, __func__)\n\nstatic void dd_enter_helper(ngx_http_request_t *r, const char *func) {\n ngx_http_posted_request_t *pr;\n\n fprintf(stderr, \">enter %s %.*s %.*s?%.*s c:%d m:%p r:%p ar:%p pr:%p\",\n func,\n (int) r->method_name.len, r->method_name.data,\n (int) r->uri.len, r->uri.data,\n (int) r->args.len, r->args.data,\n 0/*(int) r->main->count*/, r->main,\n r, r->connection->data, r->parent);\n\n if (r->posted_requests) {\n fprintf(stderr, \" posted:\");\n\n for (pr = r->posted_requests; pr; pr = pr->next) {\n fprintf(stderr, \"%p,\", pr);\n }\n }\n\n fprintf(stderr, \"\\n\");\n}\n\n# else\n\n# define dd_enter()\n\n# endif\n\n#else\n\n# if (NGX_HAVE_VARIADIC_MACROS)\n\n# define dd(...)\n\n# define dd_enter()\n\n# else\n\n#include \n\nstatic void dd(const char * fmt, ...) {\n}\n\nstatic void dd_enter() {\n}\n\n# endif\n\n#endif\n\n#if defined(DDEBUG) && (DDEBUG)\n\n#define dd_check_read_event_handler(r) \\\n dd(\"r->read_event_handler = %s\", \\\n r->read_event_handler == ngx_http_block_reading ? \\\n \"ngx_http_block_reading\" : \\\n r->read_event_handler == ngx_http_test_reading ? \\\n \"ngx_http_test_reading\" : \\\n r->read_event_handler == ngx_http_request_empty_handler ? \\\n \"ngx_http_request_empty_handler\" : \"UNKNOWN\")\n\n#define dd_check_write_event_handler(r) \\\n dd(\"r->write_event_handler = %s\", \\\n r->write_event_handler == ngx_http_handler ? \\\n \"ngx_http_handler\" : \\\n r->write_event_handler == ngx_http_core_run_phases ? \\\n \"ngx_http_core_run_phases\" : \\\n r->write_event_handler == ngx_http_request_empty_handler ? \\\n \"ngx_http_request_empty_handler\" : \"UNKNOWN\")\n\n#else\n\n#define dd_check_read_event_handler(r)\n#define dd_check_write_event_handler(r)\n\n#endif\n\n#endif /* DDEBUG_H */\n\n" }, { "path": "src/ngx_http_replace_filter_module.c", "content": "\n/*\n * Copyright (C) Yichun Zhang (agentzh)\n * Copyright (C) Igor Sysoev\n * Copyright (C) Nginx, Inc.\n */\n\n\n#ifndef DDEBUG\n#define DDEBUG 0\n#endif\n#include \"ddebug.h\"\n\n\n#include \"ngx_http_replace_filter_module.h\"\n#include \"ngx_http_replace_parse.h\"\n#include \"ngx_http_replace_script.h\"\n#include \"ngx_http_replace_util.h\"\n\n\nenum {\n SREGEX_COMPILER_POOL_SIZE = 4096\n};\n\n\nstatic ngx_int_t ngx_http_replace_output(ngx_http_request_t *r,\n ngx_http_replace_ctx_t *ctx);\nstatic char *ngx_http_replace_filter(ngx_conf_t *cf, ngx_command_t *cmd,\n void *conf);\nstatic void *ngx_http_replace_create_loc_conf(ngx_conf_t *cf);\nstatic char *ngx_http_replace_merge_loc_conf(ngx_conf_t *cf,\n void *parent, void *child);\nstatic ngx_int_t ngx_http_replace_filter_init(ngx_conf_t *cf);\nstatic void ngx_http_replace_cleanup_pool(void *data);\nstatic void *ngx_http_replace_create_main_conf(ngx_conf_t *cf);\n\n\n#define ngx_http_replace_regex_is_disabled(ctx) \\\n ((ctx)->disabled[(ctx)->regex_id / 8] & (1 << ((ctx)->regex_id % 8)))\n\n\n#define ngx_http_replace_regex_set_disabled(ctx) \\\n (ctx)->disabled[(ctx)->regex_id / 8] |= (1 << ((ctx)->regex_id % 8))\n\n\nstatic volatile ngx_cycle_t *ngx_http_replace_prev_cycle = NULL;\n\n\n#define NGX_HTTP_REPLACE_CLEAR_LAST_MODIFIED 0\n#define NGX_HTTP_REPLACE_KEEP_LAST_MODIFIED 1\n\n\nstatic ngx_conf_enum_t ngx_http_replace_filter_last_modified[] = {\n { ngx_string(\"clear\"), NGX_HTTP_REPLACE_CLEAR_LAST_MODIFIED },\n { ngx_string(\"keep\"), NGX_HTTP_REPLACE_KEEP_LAST_MODIFIED },\n { ngx_null_string, 0 }\n};\n\n\nstatic ngx_command_t ngx_http_replace_filter_commands[] = {\n\n { ngx_string(\"replace_filter\"),\n NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_HTTP_LIF_CONF\n |NGX_CONF_TAKE23,\n ngx_http_replace_filter,\n NGX_HTTP_LOC_CONF_OFFSET,\n 0,\n NULL },\n\n { ngx_string(\"replace_filter_types\"),\n NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_HTTP_LIF_CONF\n |NGX_CONF_1MORE,\n ngx_http_types_slot,\n NGX_HTTP_LOC_CONF_OFFSET,\n offsetof(ngx_http_replace_loc_conf_t, types_keys),\n &ngx_http_html_default_types[0] },\n\n { ngx_string(\"replace_filter_max_buffered_size\"),\n NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_HTTP_LIF_CONF\n |NGX_CONF_TAKE1,\n ngx_conf_set_size_slot,\n NGX_HTTP_LOC_CONF_OFFSET,\n offsetof(ngx_http_replace_loc_conf_t, max_buffered_size),\n NULL },\n\n { ngx_string(\"replace_filter_last_modified\"),\n NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_HTTP_LIF_CONF\n |NGX_CONF_1MORE,\n ngx_conf_set_enum_slot,\n NGX_HTTP_LOC_CONF_OFFSET,\n offsetof(ngx_http_replace_loc_conf_t, last_modified),\n &ngx_http_replace_filter_last_modified },\n\n { ngx_string(\"replace_filter_skip\"),\n NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_HTTP_LIF_CONF\n |NGX_CONF_TAKE1,\n ngx_http_set_complex_value_slot,\n NGX_HTTP_LOC_CONF_OFFSET,\n offsetof(ngx_http_replace_loc_conf_t, skip),\n NULL },\n\n ngx_null_command\n};\n\n\nstatic ngx_http_module_t ngx_http_replace_filter_module_ctx = {\n NULL, /* preconfiguration */\n ngx_http_replace_filter_init, /* postconfiguration */\n\n ngx_http_replace_create_main_conf, /* create main configuration */\n NULL, /* init main configuration */\n\n NULL, /* create server configuration */\n NULL, /* merge server configuration */\n\n ngx_http_replace_create_loc_conf, /* create location configuration */\n ngx_http_replace_merge_loc_conf /* merge location configuration */\n};\n\n\nngx_module_t ngx_http_replace_filter_module = {\n NGX_MODULE_V1,\n &ngx_http_replace_filter_module_ctx, /* module context */\n ngx_http_replace_filter_commands, /* module directives */\n NGX_HTTP_MODULE, /* module type */\n NULL, /* init master */\n NULL, /* init module */\n NULL, /* init process */\n NULL, /* init thread */\n NULL, /* exit thread */\n NULL, /* exit process */\n NULL, /* exit master */\n NGX_MODULE_V1_PADDING\n};\n\n\nstatic ngx_http_output_header_filter_pt ngx_http_next_header_filter;\nstatic ngx_http_output_body_filter_pt ngx_http_next_body_filter;\n\n\nstatic ngx_int_t\nngx_http_replace_header_filter(ngx_http_request_t *r)\n{\n size_t size;\n ngx_str_t skip;\n ngx_pool_cleanup_t *cln;\n ngx_http_replace_ctx_t *ctx;\n ngx_http_replace_loc_conf_t *rlcf;\n\n rlcf = ngx_http_get_module_loc_conf(r, ngx_http_replace_filter_module);\n\n dd(\"replace header filter\");\n\n if (rlcf->regexes.nelts == 0\n || r->headers_out.content_length_n == 0\n || (r->headers_out.content_encoding\n && r->headers_out.content_encoding->value.len)\n || ngx_http_test_content_type(r, &rlcf->types) == NULL)\n {\n return ngx_http_next_header_filter(r);\n }\n\n dd(\"skip: %p\", rlcf->skip);\n\n if (rlcf->skip != NULL) {\n if (ngx_http_complex_value(r, rlcf->skip, &skip) != NGX_OK) {\n return NGX_ERROR;\n }\n\n if (skip.len && (skip.len != 1 || skip.data[0] != '0')) {\n return ngx_http_next_header_filter(r);\n }\n }\n\n ctx = ngx_pcalloc(r->pool, sizeof(ngx_http_replace_ctx_t));\n if (ctx == NULL) {\n return NGX_ERROR;\n }\n\n ctx->last_special = &ctx->special;\n ctx->last_pending = &ctx->pending;\n ctx->last_pending2 = &ctx->pending2;\n ctx->last_captured = &ctx->captured;\n\n ctx->sub = ngx_pcalloc(r->pool,\n rlcf->multi_replace.nelts * sizeof(ngx_str_t));\n if (ctx->sub == NULL) {\n return NGX_ERROR;\n }\n\n ctx->ovector = ngx_palloc(r->pool, rlcf->ovecsize);\n if (ctx->ovector == NULL) {\n return NGX_ERROR;\n }\n\n size = ngx_align(rlcf->regexes.nelts, 8) / 8;\n ctx->disabled = ngx_pcalloc(r->pool, size);\n if (ctx->disabled == NULL) {\n return NGX_ERROR;\n }\n\n ctx->vm_pool = sre_create_pool(1024);\n if (ctx->vm_pool == NULL) {\n return NGX_ERROR;\n }\n\n dd(\"created vm pool %p\", ctx->vm_pool);\n\n cln = ngx_pool_cleanup_add(r->pool, 0);\n if (cln == NULL) {\n sre_destroy_pool(ctx->vm_pool);\n return NGX_ERROR;\n }\n\n cln->data = ctx->vm_pool;\n cln->handler = ngx_http_replace_cleanup_pool;\n\n ctx->vm_ctx = sre_vm_pike_create_ctx(ctx->vm_pool, rlcf->program,\n ctx->ovector, rlcf->ovecsize);\n if (ctx->vm_ctx == NULL) {\n return NGX_ERROR;\n }\n\n ngx_http_set_ctx(r, ctx, ngx_http_replace_filter_module);\n\n ctx->last_out = &ctx->out;\n\n r->filter_need_in_memory = 1;\n\n if (r == r->main) {\n ngx_http_clear_content_length(r);\n\n if (rlcf->last_modified == NGX_HTTP_REPLACE_CLEAR_LAST_MODIFIED) {\n ngx_http_clear_last_modified(r);\n }\n }\n\n return ngx_http_next_header_filter(r);\n}\n\n\nstatic void\nngx_http_replace_cleanup_pool(void *data)\n{\n sre_pool_t *pool = data;\n\n if (pool) {\n dd(\"destroy sre pool %p\", pool);\n sre_destroy_pool(pool);\n }\n}\n\n\nstatic ngx_int_t\nngx_http_replace_body_filter(ngx_http_request_t *r, ngx_chain_t *in)\n{\n ngx_int_t rc;\n ngx_buf_t *b;\n ngx_str_t *sub;\n ngx_chain_t *cl, *cur = NULL, *rematch = NULL;\n\n ngx_http_replace_ctx_t *ctx;\n ngx_http_replace_loc_conf_t *rlcf;\n\n rlcf = ngx_http_get_module_loc_conf(r, ngx_http_replace_filter_module);\n\n ctx = ngx_http_get_module_ctx(r, ngx_http_replace_filter_module);\n\n if (ctx == NULL) {\n return ngx_http_next_body_filter(r, in);\n }\n\n if ((in == NULL\n && ctx->buf == NULL\n && ctx->in == NULL\n && ctx->busy == NULL))\n {\n return ngx_http_next_body_filter(r, in);\n }\n\n if ((ctx->once || ctx->vm_done) && (ctx->buf == NULL || ctx->in == NULL)) {\n\n if (ctx->busy) {\n if (ngx_http_replace_output(r, ctx) == NGX_ERROR) {\n return NGX_ERROR;\n }\n }\n\n return ngx_http_next_body_filter(r, in);\n }\n\n /* add the incoming chain to the chain ctx->in */\n\n if (in) {\n if (ngx_chain_add_copy(r->pool, &ctx->in, in) != NGX_OK) {\n return NGX_ERROR;\n }\n }\n\n ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,\n \"http sub filter \\\"%V\\\"\", &r->uri);\n\n while (ctx->in || ctx->buf) {\n\n if (ctx->buf == NULL) {\n cur = ctx->in;\n ctx->buf = cur->buf;\n ctx->in = cur->next;\n\n ctx->pos = ctx->buf->pos;\n ctx->special_buf = ngx_buf_special(ctx->buf);\n ctx->last_buf = (ctx->buf->last_buf || ctx->buf->last_in_chain);\n\n dd(\"=== new incoming buf: size=%d, special=%u, last=%u\",\n (int) ngx_buf_size(ctx->buf), ctx->special_buf,\n ctx->last_buf);\n }\n\n b = NULL;\n\n while (ctx->pos < ctx->buf->last\n || (ctx->special_buf && ctx->last_buf))\n {\n rc = rlcf->parse_buf(r, ctx, rematch);\n\n ngx_log_debug3(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,\n \"replace filter parse: %d, %p-%p\",\n rc, ctx->copy_start, ctx->copy_end);\n\n if (rc == NGX_ERROR) {\n return rc;\n }\n\n if (rc == NGX_DECLINED) {\n\n if (ctx->pending) {\n *ctx->last_out = ctx->pending;\n ctx->last_out = ctx->last_pending;\n\n ctx->pending = NULL;\n ctx->last_pending = &ctx->pending;\n }\n\n if (!ctx->special_buf) {\n ctx->copy_start = ctx->pos;\n ctx->copy_end = ctx->buf->last;\n ctx->pos = ctx->buf->last;\n\n } else {\n ctx->copy_start = NULL;\n ctx->copy_end = NULL;\n }\n\n sre_reset_pool(ctx->vm_pool);\n ctx->vm_done = 1;\n }\n\n dd(\"copy_end - copy_start: %d, special: %u\",\n (int) (ctx->copy_end - ctx->copy_start), ctx->special_buf);\n\n if (ctx->copy_start != ctx->copy_end && !ctx->special_buf) {\n dd(\"copy: %.*s\", (int) (ctx->copy_end - ctx->copy_start),\n ctx->copy_start);\n\n cl = ngx_http_replace_get_free_buf(r->pool, &ctx->free);\n if (cl == NULL) {\n return NGX_ERROR;\n }\n\n b = cl->buf;\n\n b->memory = 1;\n b->pos = ctx->copy_start;\n b->last = ctx->copy_end;\n\n *ctx->last_out = cl;\n ctx->last_out = &cl->next;\n }\n\n if (rc == NGX_AGAIN) {\n if (ctx->special_buf && ctx->last_buf) {\n break;\n }\n\n continue;\n }\n\n if (rc == NGX_DECLINED) {\n break;\n }\n\n /* rc == NGX_OK || rc == NGX_BUSY */\n\n cl = ngx_http_replace_get_free_buf(r->pool, &ctx->free);\n if (cl == NULL) {\n return NGX_ERROR;\n }\n\n b = cl->buf;\n\n dd(\"free data buf: %p\", b);\n\n sub = &ctx->sub[ctx->regex_id];\n\n if (sub->data == NULL\n || rlcf->parse_buf == ngx_http_replace_capturing_parse)\n {\n ngx_http_replace_complex_value_t *cv;\n\n if (ngx_http_replace_regex_is_disabled(ctx)) {\n cv = &rlcf->verbatim;\n\n } else {\n cv = rlcf->multi_replace.elts;\n cv = &cv[ctx->regex_id];\n }\n\n if (ngx_http_replace_complex_value(r, ctx->captured,\n rlcf->ncaps,\n ctx->ovector,\n cv, sub)\n != NGX_OK)\n {\n return NGX_ERROR;\n }\n\n /* release ctx->captured */\n if (ctx->captured) {\n dd(\"release ctx captured: %p\", ctx->captured);\n *ctx->last_captured = ctx->free;\n ctx->free = ctx->captured;\n\n ctx->captured = NULL;\n ctx->last_captured = &ctx->captured;\n }\n }\n\n dd(\"emit replaced value: \\\"%.*s\\\"\", (int) sub->len, sub->data);\n\n if (sub->len) {\n b->memory = 1;\n b->pos = sub->data;\n b->last = sub->data + sub->len;\n\n } else {\n b->sync = 1;\n }\n\n cl->buf = b;\n cl->next = NULL;\n\n *ctx->last_out = cl;\n ctx->last_out = &cl->next;\n\n if (!ctx->once && !ngx_http_replace_regex_is_disabled(ctx)) {\n uint8_t *once;\n\n once = rlcf->multi_once.elts;\n\n if (rlcf->regexes.nelts == 1) {\n ctx->once = once[0];\n\n } else {\n if (once[ctx->regex_id]) {\n ngx_http_replace_regex_set_disabled(ctx);\n if (!rlcf->seen_global\n && ++ctx->disabled_count == rlcf->regexes.nelts)\n {\n ctx->once = 1;\n }\n }\n }\n }\n\n if (rc == NGX_BUSY) {\n dd(\"goto rematch\");\n goto rematch;\n }\n\n if (ctx->special_buf) {\n break;\n }\n\n continue;\n }\n\n if ((ctx->buf->flush || ctx->last_buf || ngx_buf_in_memory(ctx->buf))\n && cur)\n {\n if (b == NULL) {\n cl = ngx_http_replace_get_free_buf(r->pool, &ctx->free);\n if (cl == NULL) {\n return NGX_ERROR;\n }\n\n b = cl->buf;\n b->sync = 1;\n\n *ctx->last_out = cl;\n ctx->last_out = &cl->next;\n }\n\n dd(\"setting shadow and last buf: %d\", (int) ctx->buf->last_buf);\n b->last_buf = ctx->buf->last_buf;\n b->last_in_chain = ctx->buf->last_in_chain;\n b->flush = ctx->buf->flush;\n b->shadow = ctx->buf;\n b->recycled = ctx->buf->recycled;\n }\n\n if (!ctx->special_buf) {\n ctx->stream_pos += ctx->buf->last - ctx->buf->pos;\n }\n\n if (rematch) {\n rematch->next = ctx->free;\n ctx->free = rematch;\n rematch = NULL;\n }\n\nrematch:\n\n dd(\"ctx->rematch: %p\", ctx->rematch);\n\n if (ctx->rematch == NULL) {\n ctx->buf = NULL;\n cur = NULL;\n\n } else {\n\n if (cur) {\n ctx->in = cur;\n cur = NULL;\n }\n\n ctx->buf = ctx->rematch->buf;\n\n dd(\"ctx->buf set to rematch buf %p, len=%d, next=%p\",\n ctx->buf, (int) ngx_buf_size(ctx->buf), ctx->rematch->next);\n\n rematch = ctx->rematch;\n ctx->rematch = rematch->next;\n\n ctx->pos = ctx->buf->pos;\n ctx->special_buf = ngx_buf_special(ctx->buf);\n ctx->last_buf = (ctx->buf->last_buf || ctx->buf->last_in_chain);\n ctx->stream_pos = ctx->buf->file_pos;\n }\n\n#if (DDEBUG)\n /*\n ngx_http_replace_dump_chain(\"ctx->pending\", &ctx->pending,\n ctx->last_pending);\n ngx_http_replace_dump_chain(\"ctx->pending2\", &ctx->pending2,\n ctx->last_pending2);\n */\n#endif\n } /* while */\n\n if (ctx->out == NULL && ctx->busy == NULL) {\n return NGX_OK;\n }\n\n return ngx_http_replace_output(r, ctx);\n}\n\n\nstatic ngx_int_t\nngx_http_replace_output(ngx_http_request_t *r, ngx_http_replace_ctx_t *ctx)\n{\n ngx_int_t rc;\n ngx_buf_t *b;\n ngx_chain_t *cl;\n\n#if (DDEBUG)\n b = NULL;\n for (cl = ctx->out; cl; cl = cl->next) {\n ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,\n \"replace out: %p %p\", cl->buf, cl->buf->pos);\n if (cl->buf == b) {\n ngx_log_error(NGX_LOG_ALERT, r->connection->log, 0,\n \"the same buf was used in sub\");\n ngx_debug_point();\n return NGX_ERROR;\n }\n b = cl->buf;\n }\n\n /* ngx_http_replace_dump_chain(\"ctx->out\", &ctx->out, ctx->last_out); */\n#endif\n\n rc = ngx_http_next_body_filter(r, ctx->out);\n\n /* we are essentially duplicating the logic of\n * ngx_chain_update_chains below,\n * with our own optimizations */\n\n if (ctx->busy == NULL) {\n ctx->busy = ctx->out;\n\n } else {\n for (cl = ctx->busy; cl->next; cl = cl->next) { /* void */ }\n cl->next = ctx->out;\n }\n\n ctx->out = NULL;\n ctx->last_out = &ctx->out;\n\n while (ctx->busy) {\n\n cl = ctx->busy;\n b = cl->buf;\n\n if (ngx_buf_size(b) != 0) {\n break;\n }\n\n if (cl->buf->tag != (ngx_buf_tag_t) &ngx_http_replace_filter_module) {\n ctx->busy = cl->next;\n ngx_free_chain(r->pool, cl);\n continue;\n }\n\n if (b->shadow) {\n b->shadow->pos = b->shadow->last;\n b->shadow->file_pos = b->shadow->file_last;\n }\n\n ctx->busy = cl->next;\n\n if (ngx_buf_special(b)) {\n\n /* collect special bufs to ctx->special, which may still be busy */\n\n cl->next = NULL;\n *ctx->last_special = cl;\n ctx->last_special = &cl->next;\n\n } else {\n\n /* add ctx->special to ctx->free because they cannot be busy at\n * this point */\n\n *ctx->last_special = ctx->free;\n ctx->free = ctx->special;\n ctx->special = NULL;\n ctx->last_special = &ctx->special;\n\n#if 0\n /* free the temporary buf's data block if it is big enough */\n if (b->temporary\n && b->start != NULL\n && b->end - b->start > (ssize_t) r->pool->max)\n {\n ngx_pfree(r->pool, b->start);\n }\n#endif\n\n /* add the data buf itself to the free buf chain */\n\n cl->next = ctx->free;\n ctx->free = cl;\n }\n }\n\n if (ctx->in || ctx->buf) {\n r->buffered |= NGX_HTTP_SUB_BUFFERED;\n\n } else {\n r->buffered &= ~NGX_HTTP_SUB_BUFFERED;\n }\n\n return rc;\n}\n\n\nstatic char *\nngx_http_replace_filter(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)\n{\n ngx_http_replace_loc_conf_t *rlcf = conf;\n ngx_http_replace_main_conf_t *rmcf;\n\n int *flags;\n u_char *p, **re;\n ngx_str_t *value;\n ngx_uint_t i;\n uint8_t *once;\n\n ngx_pool_cleanup_t *cln;\n ngx_http_replace_complex_value_t *cv;\n ngx_http_replace_compile_complex_value_t ccv;\n\n value = cf->args->elts;\n\n re = ngx_array_push(&rlcf->regexes);\n if (re == NULL) {\n return NGX_CONF_ERROR;\n }\n\n *re = value[1].data;\n\n cv = ngx_array_push(&rlcf->multi_replace);\n if (cv == NULL) {\n return NGX_CONF_ERROR;\n }\n\n ngx_memzero(cv, sizeof(ngx_http_replace_complex_value_t));\n ngx_memzero(&ccv, sizeof(ngx_http_replace_compile_complex_value_t));\n\n ccv.cf = cf;\n ccv.value = &value[2];\n ccv.complex_value = cv;\n\n if (ngx_http_replace_compile_complex_value(&ccv) != NGX_OK) {\n return NGX_CONF_ERROR;\n }\n\n /* check variable usage in the \"replace\" argument */\n\n if (cv->capture_variables) {\n rlcf->parse_buf = ngx_http_replace_capturing_parse;\n\n } else if (rlcf->parse_buf == NULL) {\n rlcf->parse_buf = ngx_http_replace_non_capturing_parse;\n }\n\n#if 0\n rlcf->parse_buf = ngx_http_replace_capturing_parse;\n#endif\n\n flags = ngx_array_push(&rlcf->multi_flags);\n if (flags == NULL) {\n return NGX_CONF_ERROR;\n }\n *flags = 0;\n\n once = ngx_array_push(&rlcf->multi_once);\n if (once == NULL) {\n return NGX_CONF_ERROR;\n }\n *once = 1; /* default to once */\n\n if (cf->args->nelts == 4) {\n /* 3 user args */\n\n p = value[3].data;\n\n for (i = 0; i < value[3].len; i++) {\n switch (p[i]) {\n case 'i':\n *flags |= SRE_REGEX_CASELESS;\n break;\n\n case 'g':\n *once = 0;\n break;\n\n default:\n return \"specifies an unrecognized regex flag\";\n }\n }\n }\n\n if (*once) {\n rlcf->seen_once = 1;\n\n } else {\n rlcf->seen_global = 1;\n }\n\n if (rlcf->seen_once && rlcf->regexes.nelts > 1) {\n rlcf->parse_buf = ngx_http_replace_capturing_parse;\n\n if (rlcf->verbatim.value.data == NULL) {\n ngx_str_t v = ngx_string(\"$&\");\n\n ngx_memzero(&ccv, sizeof(ngx_http_replace_compile_complex_value_t));\n\n ccv.cf = cf;\n ccv.value = &v;\n ccv.complex_value = &rlcf->verbatim;\n\n if (ngx_http_replace_compile_complex_value(&ccv) != NGX_OK) {\n return NGX_CONF_ERROR;\n }\n }\n }\n\n rmcf =\n ngx_http_conf_get_module_main_conf(cf, ngx_http_replace_filter_module);\n\n if (rmcf->compiler_pool == NULL) {\n rmcf->compiler_pool = sre_create_pool(SREGEX_COMPILER_POOL_SIZE);\n if (rmcf->compiler_pool == NULL) {\n return NGX_CONF_ERROR;\n }\n\n cln = ngx_pool_cleanup_add(cf->pool, 0);\n if (cln == NULL) {\n sre_destroy_pool(rmcf->compiler_pool);\n rmcf->compiler_pool = NULL;\n return NGX_CONF_ERROR;\n }\n\n cln->data = rmcf->compiler_pool;\n cln->handler = ngx_http_replace_cleanup_pool;\n }\n\n return NGX_CONF_OK;\n}\n\n\nstatic void *\nngx_http_replace_create_loc_conf(ngx_conf_t *cf)\n{\n ngx_http_replace_loc_conf_t *conf;\n\n conf = ngx_pcalloc(cf->pool, sizeof(ngx_http_replace_loc_conf_t));\n if (conf == NULL) {\n return NULL;\n }\n\n /*\n * set by ngx_pcalloc():\n *\n * conf->types = { NULL };\n * conf->types_keys = NULL;\n * conf->program = NULL;\n * conf->ncaps = 0;\n * conf->ovecsize = 0;\n * conf->parse_buf = NULL;\n * conf->verbatim = { {0, NULL}, NULL, NULL, 0 };\n * conf->seen_once = 0;\n * conf->seen_global = 0;\n * conf->skip = NULL;\n */\n\n conf->max_buffered_size = NGX_CONF_UNSET_SIZE;\n conf->last_modified = NGX_CONF_UNSET_UINT;\n\n ngx_array_init(&conf->multi_replace, cf->pool, 4,\n sizeof(ngx_http_replace_complex_value_t));\n\n ngx_array_init(&conf->multi_flags, cf->pool, 4, sizeof(int));\n\n ngx_array_init(&conf->regexes, cf->pool, 4, sizeof(u_char *));\n\n ngx_array_init(&conf->multi_once, cf->pool, 4, sizeof(uint8_t));\n\n return conf;\n}\n\n\nstatic char *\nngx_http_replace_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child)\n{\n u_char **value;\n sre_int_t err_offset, err_regex_id;\n ngx_str_t prefix, suffix;\n sre_pool_t *ppool; /* parser pool */\n sre_regex_t *re;\n sre_program_t *prog;\n\n ngx_http_replace_main_conf_t *rmcf;\n\n ngx_http_replace_loc_conf_t *prev = parent;\n ngx_http_replace_loc_conf_t *conf = child;\n\n ngx_conf_merge_size_value(conf->max_buffered_size,\n prev->max_buffered_size,\n 8192);\n\n ngx_conf_merge_uint_value(conf->last_modified,\n prev->last_modified,\n NGX_HTTP_REPLACE_CLEAR_LAST_MODIFIED);\n\n if (ngx_http_merge_types(cf, &conf->types_keys, &conf->types,\n &prev->types_keys, &prev->types,\n ngx_http_html_default_types)\n != NGX_OK)\n {\n return NGX_CONF_ERROR;\n }\n\n if (conf->skip == NULL) {\n conf->skip = prev->skip;\n }\n\n if (conf->regexes.nelts > 0 && conf->program == NULL) {\n\n dd(\"parsing and compiling %d regexes\", (int) conf->regexes.nelts);\n\n ppool = sre_create_pool(1024);\n if (ppool == NULL) {\n return NGX_CONF_ERROR;\n }\n\n value = conf->regexes.elts;\n\n re = sre_regex_parse_multi(ppool, value, conf->regexes.nelts,\n &conf->ncaps, conf->multi_flags.elts,\n &err_offset, &err_regex_id);\n\n if (re == NULL) {\n\n if (err_offset >= 0) {\n prefix.data = value[err_regex_id];\n prefix.len = err_offset;\n\n suffix.data = value[err_regex_id] + err_offset;\n suffix.len = ngx_strlen(value[err_regex_id]) - err_offset;\n\n ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,\n \"failed to parse regex at offset %i: \"\n \"syntax error; marked by <-- HERE in \"\n \"\\\"%V <-- HERE %V\\\"\",\n (ngx_int_t) err_offset, &prefix, &suffix);\n\n } else {\n\n if (err_regex_id >= 0) {\n ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,\n \"failed to parse regex \\\"%s\\\"\",\n value[err_regex_id]);\n\n } else {\n ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,\n \"failed to parse regex \\\"%s\\\" \"\n \"and its siblings\",\n value[0]);\n }\n }\n\n sre_destroy_pool(ppool);\n return NGX_CONF_ERROR;\n }\n\n rmcf = ngx_http_conf_get_module_main_conf(cf,\n ngx_http_replace_filter_module);\n\n prog = sre_regex_compile(rmcf->compiler_pool, re);\n\n sre_destroy_pool(ppool);\n\n if (prog == NULL) {\n ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,\n \"failed to compile regex \\\"%s\\\" and its \"\n \"siblings\", value[0]);\n\n return NGX_CONF_ERROR;\n }\n\n conf->program = prog;\n conf->ovecsize = 2 * (conf->ncaps + 1) * sizeof(sre_int_t);\n\n } else {\n\n conf->regexes = prev->regexes;\n conf->multi_once = prev->multi_once;\n conf->multi_flags = prev->multi_flags;\n conf->multi_replace = prev->multi_replace;\n conf->parse_buf = prev->parse_buf;\n conf->verbatim = prev->verbatim;\n conf->program = prev->program;\n conf->ncaps = prev->ncaps;\n conf->ovecsize = prev->ovecsize;\n conf->seen_once = prev->seen_once;\n conf->seen_global = prev->seen_global;\n }\n\n return NGX_CONF_OK;\n}\n\n\nstatic ngx_int_t\nngx_http_replace_filter_init(ngx_conf_t *cf)\n{\n int multi_http_blocks;\n ngx_http_replace_main_conf_t *rmcf;\n\n rmcf =\n ngx_http_conf_get_module_main_conf(cf, ngx_http_replace_filter_module);\n\n if (ngx_http_replace_prev_cycle != ngx_cycle) {\n ngx_http_replace_prev_cycle = ngx_cycle;\n multi_http_blocks = 0;\n\n } else {\n multi_http_blocks = 1;\n }\n\n if (multi_http_blocks || rmcf->compiler_pool != NULL) {\n ngx_http_next_header_filter = ngx_http_top_header_filter;\n ngx_http_top_header_filter = ngx_http_replace_header_filter;\n\n ngx_http_next_body_filter = ngx_http_top_body_filter;\n ngx_http_top_body_filter = ngx_http_replace_body_filter;\n\n return NGX_OK;\n }\n\n return NGX_OK;\n}\n\n\nstatic void *\nngx_http_replace_create_main_conf(ngx_conf_t *cf)\n{\n ngx_http_replace_main_conf_t *rmcf;\n\n rmcf = ngx_pcalloc(cf->pool, sizeof(ngx_http_replace_main_conf_t));\n if (rmcf == NULL) {\n return NULL;\n }\n\n /* set by ngx_pcalloc:\n * rmcf->compiler_pool = NULL;\n */\n\n return rmcf;\n}\n" }, { "path": "src/ngx_http_replace_filter_module.h", "content": "#ifndef _NGX_HTTP_REPLACE_FILTER_MODULE_H_INCLUDED_\n#define _NGX_HTTP_REPLACE_FILTER_MODULE_H_INCLUDED_\n\n\n#include \"ngx_http_replace_script.h\"\n#include \n#include \n#include \n#include \n\n\nextern ngx_module_t ngx_http_replace_filter_module;\n\n\ntypedef struct {\n sre_int_t regex_id;\n sre_int_t stream_pos;\n sre_int_t *ovector;\n sre_pool_t *vm_pool;\n sre_vm_pike_ctx_t *vm_ctx;\n\n ngx_chain_t *pending; /* pending data before the\n pending matched capture */\n ngx_chain_t **last_pending;\n\n ngx_chain_t *pending2; /* pending data after the pending\n matched capture */\n ngx_chain_t **last_pending2;\n\n ngx_buf_t *buf;\n\n ngx_str_t *sub;\n\n u_char *pos;\n u_char *copy_start;\n u_char *copy_end;\n\n ngx_chain_t *in;\n ngx_chain_t *out;\n ngx_chain_t **last_out;\n ngx_chain_t *busy;\n ngx_chain_t *free;\n ngx_chain_t *special;\n ngx_chain_t **last_special;\n ngx_chain_t *rematch;\n ngx_chain_t *captured;\n ngx_chain_t **last_captured;\n uint8_t *disabled;\n sre_uint_t disabled_count;\n\n size_t total_buffered;\n\n unsigned once:1;\n unsigned vm_done:1;\n unsigned special_buf:1;\n unsigned last_buf:1;\n} ngx_http_replace_ctx_t;\n\n\ntypedef ngx_int_t (*ngx_http_replace_parse_buf_pt)(ngx_http_request_t *r,\n ngx_http_replace_ctx_t *ctx, ngx_chain_t *rematch);\n\n\ntypedef struct {\n sre_pool_t *compiler_pool;\n} ngx_http_replace_main_conf_t;\n\n\ntypedef struct {\n sre_uint_t ncaps;\n size_t ovecsize;\n\n ngx_array_t multi_once; /* of uint8_t */\n ngx_array_t regexes; /* of u_char* */\n ngx_array_t multi_flags; /* of int */\n ngx_array_t multi_replace;\n /* of ngx_http_replace_complex_value_t */\n\n sre_program_t *program;\n\n ngx_hash_t types;\n ngx_array_t *types_keys;\n\n size_t max_buffered_size;\n\n ngx_uint_t last_modified;\n /* replace_filter_last_modified */\n\n ngx_http_replace_parse_buf_pt parse_buf;\n ngx_http_replace_complex_value_t verbatim;\n\n ngx_http_complex_value_t *skip;\n\n unsigned seen_once; /* :1 */\n unsigned seen_global; /* :1 */\n} ngx_http_replace_loc_conf_t;\n\n\n#endif /* _NGX_HTTP_REPLACE_FILTER_MODULE_H_INCLUDED_ */\n" }, { "path": "src/ngx_http_replace_parse.c", "content": "\n/*\n * Copyright (C) Yichun Zhang (agentzh)\n * Copyright (C) Igor Sysoev\n * Copyright (C) Nginx, Inc.\n */\n\n\n#ifndef DDEBUG\n#define DDEBUG 0\n#endif\n#include \"ddebug.h\"\n\n\n#include \"ngx_http_replace_parse.h\"\n#include \"ngx_http_replace_util.h\"\n\n\nstatic void ngx_http_replace_check_total_buffered(ngx_http_request_t *r,\n ngx_http_replace_ctx_t *ctx, sre_int_t len, sre_int_t mlen);\n\n\nngx_int_t\nngx_http_replace_capturing_parse(ngx_http_request_t *r,\n ngx_http_replace_ctx_t *ctx, ngx_chain_t *rematch)\n{\n sre_int_t ret, from, to;\n ngx_int_t rc;\n ngx_chain_t *new_rematch = NULL;\n ngx_chain_t *cl;\n ngx_chain_t **last_rematch, **last;\n size_t len;\n\n dd(\"replace capturing parse\");\n\n if (ctx->once || ctx->vm_done) {\n ctx->copy_start = ctx->pos;\n ctx->copy_end = ctx->buf->last;\n ctx->pos = ctx->buf->last;\n\n ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, \"once\");\n\n return NGX_AGAIN;\n }\n\n len = ctx->buf->last - ctx->pos;\n\n dd(\"=== process data chunk %p len=%d, pos=%u, special=%u, \"\n \"last=%u, \\\"%.*s\\\"\", ctx->buf, (int) (ctx->buf->last - ctx->pos),\n (int) (ctx->pos - ctx->buf->pos + ctx->stream_pos),\n ctx->special_buf, ctx->last_buf,\n (int) (ctx->buf->last - ctx->pos), ctx->pos);\n\n ret = sre_vm_pike_exec(ctx->vm_ctx, ctx->pos, len, ctx->last_buf, NULL);\n\n dd(\"vm pike exec: %d\", (int) ret);\n\n if (ret >= 0) {\n ctx->regex_id = ret;\n ctx->total_buffered = 0;\n\n from = ctx->ovector[0];\n to = ctx->ovector[1];\n\n dd(\"pike vm ok: (%d, %d)\", (int) from, (int) to);\n\n if (from >= ctx->stream_pos) {\n /* the match is completely on the current buf */\n\n if (ctx->pending) {\n *ctx->last_out = ctx->pending;\n ctx->last_out = ctx->last_pending;\n\n ctx->pending = NULL;\n ctx->last_pending = &ctx->pending;\n }\n\n /* prepare ctx->captured */\n\n cl = ngx_http_replace_get_free_buf(r->pool, &ctx->free);\n if (cl == NULL) {\n return NGX_ERROR;\n }\n\n cl->buf->pos = ctx->buf->pos;\n cl->buf->last = ctx->buf->last;\n cl->buf->memory = 1;\n cl->buf->file_pos = ctx->stream_pos;\n cl->buf->file_last = ctx->stream_pos\n + (cl->buf->last - cl->buf->pos);\n\n *ctx->last_captured = cl;\n ctx->last_captured = &cl->next;\n\n dd(\"ctx captured: %p\", ctx->captured);\n\n /* prepare copy-out data */\n\n ctx->copy_start = ctx->pos;\n ctx->copy_end = ctx->buf->pos + (from - ctx->stream_pos);\n\n dd(\"copy len: %d\", (int) (ctx->copy_end - ctx->copy_start));\n\n ctx->pos = ctx->buf->pos + (to - ctx->stream_pos);\n return NGX_OK;\n }\n\n /* from < ctx->stream_pos */\n\n if (ctx->pending) {\n\n if (ngx_http_replace_split_chain(r, ctx, &ctx->pending,\n &ctx->last_pending, from,\n &cl, &last, 1)\n != NGX_OK)\n {\n return NGX_ERROR;\n }\n\n if (ctx->pending) {\n *ctx->last_out = ctx->pending;\n ctx->last_out = ctx->last_pending;\n\n ctx->pending = NULL;\n ctx->last_pending = &ctx->pending;\n }\n\n if (cl) {\n if (to >= ctx->stream_pos) {\n /* no pending data to be rematched */\n\n if (to == ctx->stream_pos) {\n *ctx->last_captured = cl;\n ctx->last_captured = &cl->next;\n\n } else {\n *ctx->last_captured = cl;\n ctx->last_captured = last;\n\n cl = ngx_http_replace_get_free_buf(r->pool, &ctx->free);\n if (cl == NULL) {\n return NGX_ERROR;\n }\n\n cl->buf->pos = ctx->buf->pos;\n cl->buf->last = ctx->buf->last;\n cl->buf->memory = 1;\n cl->buf->file_pos = ctx->stream_pos;\n cl->buf->file_last = ctx->stream_pos\n + (cl->buf->last - cl->buf->pos);\n\n *ctx->last_captured = cl;\n ctx->last_captured = &cl->next;\n }\n\n } else {\n /* there's pending data to be rematched */\n\n if (ngx_http_replace_split_chain(r, ctx, &cl,\n &last,\n to, &new_rematch,\n &last_rematch, 1)\n != NGX_OK)\n {\n return NGX_ERROR;\n }\n\n if (cl) {\n *ctx->last_captured = cl;\n ctx->last_captured = last;\n }\n\n if (new_rematch) {\n if (rematch) {\n ctx->rematch = rematch;\n }\n\n /* prepend cl to ctx->rematch */\n *last_rematch = ctx->rematch;\n ctx->rematch = new_rematch;\n }\n }\n }\n }\n\n#if (DDEBUG)\n ngx_http_replace_dump_chain(\"ctx->rematch\", &ctx->rematch, NULL);\n#endif\n\n ctx->copy_start = NULL;\n ctx->copy_end = NULL;\n\n ctx->pos = ctx->buf->pos + (to - ctx->stream_pos);\n\n return new_rematch ? NGX_BUSY : NGX_OK;\n }\n\n switch (ret) {\n case SRE_AGAIN:\n from = ctx->ovector[0];\n to = ctx->ovector[1];\n\n dd(\"pike vm again: (%d, %d)\", (int) from, (int) to);\n\n if (from == -1) {\n from = ctx->stream_pos + (ctx->buf->last - ctx->buf->pos);\n }\n\n if (to == -1) {\n to = ctx->stream_pos + (ctx->buf->last - ctx->buf->pos);\n }\n\n dd(\"pike vm again (adjusted): stream pos:%d, (%d, %d)\",\n (int) ctx->stream_pos, (int) from, (int) to);\n\n if (from > to) {\n ngx_log_error(NGX_LOG_ALERT, r->connection->log, 0,\n \"invalid capture range: %i > %i\", (ngx_int_t) from,\n (ngx_int_t) to);\n return NGX_ERROR;\n }\n\n if (from == to) {\n if (ctx->pending) {\n ctx->total_buffered = 0;\n\n dd(\"output pending\");\n *ctx->last_out = ctx->pending;\n ctx->last_out = ctx->last_pending;\n\n ctx->pending = NULL;\n ctx->last_pending = &ctx->pending;\n }\n\n ctx->copy_start = ctx->pos;\n ctx->copy_end = ctx->buf->pos + (from - ctx->stream_pos);\n ctx->pos = ctx->copy_end;\n\n return NGX_AGAIN;\n }\n\n /*\n * append the existing ctx->pending data right before\n * the $& capture to ctx->out.\n */\n\n if (from >= ctx->stream_pos) {\n /* the match is completely on the current buf */\n\n ctx->copy_start = ctx->pos;\n ctx->copy_end = ctx->buf->pos + (from - ctx->stream_pos);\n\n if (ctx->pending) {\n ctx->total_buffered = 0;\n\n *ctx->last_out = ctx->pending;\n ctx->last_out = ctx->last_pending;\n\n ctx->pending = NULL;\n ctx->last_pending = &ctx->pending;\n }\n\n dd(\"create ctx->pending as (%ld, %ld)\", (long) from, (long) to);\n rc = ngx_http_replace_new_pending_buf(r, ctx, from, to, &cl);\n if (rc == NGX_ERROR) {\n return NGX_ERROR;\n }\n\n#if 1\n if (rc == NGX_BUSY) {\n dd(\"stop processing because of buffer size limit reached\");\n ctx->once = 1;\n ctx->copy_start = ctx->pos;\n ctx->copy_end = ctx->buf->last;\n ctx->pos = ctx->buf->last;\n return NGX_AGAIN;\n }\n#endif\n\n *ctx->last_pending = cl;\n ctx->last_pending = &cl->next;\n\n ctx->pos = ctx->buf->last;\n\n return NGX_AGAIN;\n }\n\n dd(\"from < ctx->stream_pos\");\n\n if (ctx->pending) {\n /* split ctx->pending into ctx->out and ctx->pending */\n\n if (ngx_http_replace_split_chain(r, ctx, &ctx->pending,\n &ctx->last_pending, from, &cl,\n &last, 1)\n != NGX_OK)\n {\n return NGX_ERROR;\n }\n\n if (ctx->pending) {\n dd(\"adjust pending: pos=%d, from=%d\",\n (int) ctx->pending->buf->file_pos, (int) from);\n\n ctx->total_buffered -= (size_t)\n (from - ctx->pending->buf->file_pos);\n\n *ctx->last_out = ctx->pending;\n ctx->last_out = ctx->last_pending;\n\n ctx->pending = NULL;\n ctx->last_pending = &ctx->pending;\n }\n\n if (cl) {\n dd(\"splitted ctx->pending into ctx->out and ctx->pending: %d\",\n (int) ctx->total_buffered);\n\n ctx->pending = cl;\n ctx->last_pending = last;\n }\n }\n\n /* new pending data to buffer to ctx->pending */\n\n rc = ngx_http_replace_new_pending_buf(r, ctx, ctx->pos\n - ctx->buf->pos\n + ctx->stream_pos, to, &cl);\n if (rc == NGX_ERROR) {\n return NGX_ERROR;\n }\n\n#if 1\n if (rc == NGX_BUSY) {\n ctx->once = 1;\n\n if (ctx->pending) {\n *ctx->last_out = ctx->pending;\n ctx->last_out = ctx->last_pending;\n\n ctx->pending = NULL;\n ctx->last_pending = &ctx->pending;\n }\n\n ctx->copy_start = ctx->pos;\n ctx->copy_end = ctx->buf->last;\n ctx->pos = ctx->buf->last;\n\n return NGX_AGAIN;\n }\n#endif\n\n *ctx->last_pending = cl;\n ctx->last_pending = &cl->next;\n\n ctx->copy_start = NULL;\n ctx->copy_end = NULL;\n\n ctx->pos = ctx->buf->last;\n\n return NGX_AGAIN;\n\n case SRE_DECLINED:\n ctx->total_buffered = 0;\n\n return NGX_DECLINED;\n\n default:\n /* SRE_ERROR */\n return NGX_ERROR;\n }\n\n /* cannot reach here */\n}\n\n\nngx_int_t\nngx_http_replace_non_capturing_parse(ngx_http_request_t *r,\n ngx_http_replace_ctx_t *ctx, ngx_chain_t *rematch)\n{\n sre_int_t ret, from, to, mfrom = -1, mto = -1;\n ngx_int_t rc;\n ngx_chain_t *new_rematch = NULL;\n ngx_chain_t *cl;\n ngx_chain_t **last_rematch, **last;\n size_t len;\n sre_int_t *pending_matched;\n\n dd(\"replace non capturing parse\");\n\n if (ctx->once || ctx->vm_done) {\n ctx->copy_start = ctx->pos;\n ctx->copy_end = ctx->buf->last;\n ctx->pos = ctx->buf->last;\n\n ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, \"once\");\n\n return NGX_AGAIN;\n }\n\n len = ctx->buf->last - ctx->pos;\n\n dd(\"=== process data chunk %p len=%d, pos=%u, special=%u, \"\n \"last=%u, \\\"%.*s\\\"\", ctx->buf, (int) (ctx->buf->last - ctx->pos),\n (int) (ctx->pos - ctx->buf->pos + ctx->stream_pos),\n ctx->special_buf, ctx->last_buf,\n (int) (ctx->buf->last - ctx->pos), ctx->pos);\n\n ret = sre_vm_pike_exec(ctx->vm_ctx, ctx->pos, len, ctx->last_buf,\n &pending_matched);\n\n dd(\"vm pike exec: %d\", (int) ret);\n\n if (ret >= 0) {\n ctx->regex_id = ret;\n ctx->total_buffered = 0;\n\n from = ctx->ovector[0];\n to = ctx->ovector[1];\n\n dd(\"pike vm ok: (%d, %d)\", (int) from, (int) to);\n\n if (from >= ctx->stream_pos) {\n /* the match is completely on the current buf */\n\n if (ctx->pending) {\n *ctx->last_out = ctx->pending;\n ctx->last_out = ctx->last_pending;\n\n ctx->pending = NULL;\n ctx->last_pending = &ctx->pending;\n }\n\n if (ctx->pending2) {\n ngx_log_error(NGX_LOG_ALERT, r->connection->log, 0,\n \"assertion failed: ctx->pending2 is not NULL \"\n \"when the match is completely on the current \"\n \"buf\");\n return NGX_ERROR;\n }\n\n ctx->copy_start = ctx->pos;\n ctx->copy_end = ctx->buf->pos + (from - ctx->stream_pos);\n\n dd(\"copy len: %d\", (int) (ctx->copy_end - ctx->copy_start));\n\n ctx->pos = ctx->buf->pos + (to - ctx->stream_pos);\n return NGX_OK;\n }\n\n /* from < ctx->stream_pos */\n\n if (ctx->pending) {\n\n if (ngx_http_replace_split_chain(r, ctx, &ctx->pending,\n &ctx->last_pending, from,\n &cl, &last, 0)\n != NGX_OK)\n {\n return NGX_ERROR;\n }\n\n if (ctx->pending) {\n *ctx->last_out = ctx->pending;\n ctx->last_out = ctx->last_pending;\n\n ctx->pending = NULL;\n ctx->last_pending = &ctx->pending;\n }\n\n if (cl) {\n *last = ctx->free;\n ctx->free = cl;\n }\n }\n\n if (ctx->pending2) {\n\n if (ngx_http_replace_split_chain(r, ctx, &ctx->pending2,\n &ctx->last_pending2,\n to, &new_rematch, &last_rematch, 1)\n != NGX_OK)\n {\n return NGX_ERROR;\n }\n\n if (ctx->pending2) {\n *ctx->last_pending2 = ctx->free;\n ctx->free = ctx->pending2;\n\n ctx->pending2 = NULL;\n ctx->last_pending2 = &ctx->pending2;\n }\n\n if (new_rematch) {\n if (rematch) {\n ctx->rematch = rematch;\n }\n\n /* prepend cl to ctx->rematch */\n *last_rematch = ctx->rematch;\n ctx->rematch = new_rematch;\n }\n }\n\n#if (DDEBUG)\n ngx_http_replace_dump_chain(\"ctx->rematch\", &ctx->rematch, NULL);\n#endif\n\n ctx->copy_start = NULL;\n ctx->copy_end = NULL;\n\n ctx->pos = ctx->buf->pos + (to - ctx->stream_pos);\n\n return new_rematch ? NGX_BUSY : NGX_OK;\n }\n\n switch (ret) {\n case SRE_AGAIN:\n from = ctx->ovector[0];\n to = ctx->ovector[1];\n\n dd(\"pike vm again: (%d, %d)\", (int) from, (int) to);\n\n if (from == -1) {\n from = ctx->stream_pos + (ctx->buf->last - ctx->buf->pos);\n }\n\n if (to == -1) {\n to = ctx->stream_pos + (ctx->buf->last - ctx->buf->pos);\n }\n\n dd(\"pike vm again (adjusted): stream pos:%d, (%d, %d)\",\n (int) ctx->stream_pos, (int) from, (int) to);\n\n if (from > to) {\n ngx_log_error(NGX_LOG_ALERT, r->connection->log, 0,\n \"invalid capture range: %i > %i\", (ngx_int_t) from,\n (ngx_int_t) to);\n return NGX_ERROR;\n }\n\n if (pending_matched) {\n mfrom = pending_matched[0];\n mto = pending_matched[1];\n\n dd(\"pending matched: (%ld, %ld)\", (long) mfrom, (long) mto);\n }\n\n if (from == to) {\n if (ctx->pending) {\n ctx->total_buffered = 0;\n\n dd(\"output pending\");\n *ctx->last_out = ctx->pending;\n ctx->last_out = ctx->last_pending;\n\n ctx->pending = NULL;\n ctx->last_pending = &ctx->pending;\n }\n\n ctx->copy_start = ctx->pos;\n ctx->copy_end = ctx->buf->pos + (from - ctx->stream_pos);\n ctx->pos = ctx->copy_end;\n\n ngx_http_replace_check_total_buffered(r, ctx, to - from,\n mto - mfrom);\n return NGX_AGAIN;\n }\n\n /*\n * append the existing ctx->pending data right before\n * the $& capture to ctx->out.\n */\n\n if (from >= ctx->stream_pos) {\n /* the match is completely on the current buf */\n\n ctx->copy_start = ctx->pos;\n ctx->copy_end = ctx->buf->pos + (from - ctx->stream_pos);\n\n if (ctx->pending) {\n ctx->total_buffered = 0;\n\n *ctx->last_out = ctx->pending;\n ctx->last_out = ctx->last_pending;\n\n ctx->pending = NULL;\n ctx->last_pending = &ctx->pending;\n }\n\n if (ctx->pending2) {\n ngx_log_error(NGX_LOG_ALERT, r->connection->log, 0,\n \"assertion failed: ctx->pending2 is not NULL \"\n \"when the match is completely on the current \"\n \"buf\");\n return NGX_ERROR;\n }\n\n if (pending_matched) {\n\n if (from < mfrom) {\n /* create ctx->pending as (from, mfrom) */\n\n rc = ngx_http_replace_new_pending_buf(r, ctx, from, mfrom,\n &cl);\n if (rc == NGX_ERROR) {\n return NGX_ERROR;\n }\n\n if (rc == NGX_BUSY) {\n dd(\"stop processing because of buffer size limit \"\n \"reached\");\n ctx->once = 1;\n ctx->copy_start = ctx->pos;\n ctx->copy_end = ctx->buf->last;\n ctx->pos = ctx->buf->last;\n return NGX_AGAIN;\n }\n\n *ctx->last_pending = cl;\n ctx->last_pending = &cl->next;\n }\n\n if (mto < to) {\n /* create ctx->pending2 as (mto, to) */\n rc = ngx_http_replace_new_pending_buf(r, ctx, mto, to, &cl);\n if (rc == NGX_ERROR) {\n return NGX_ERROR;\n }\n\n#if 1\n if (rc == NGX_BUSY) {\n dd(\"stop processing because of buffer size limit \"\n \"reached\");\n ctx->once = 1;\n ctx->copy_start = ctx->pos;\n ctx->copy_end = ctx->buf->last;\n ctx->pos = ctx->buf->last;\n return NGX_AGAIN;\n }\n#endif\n\n *ctx->last_pending2 = cl;\n ctx->last_pending2 = &cl->next;\n }\n\n } else {\n dd(\"create ctx->pending as (%ld, %ld)\", (long) from, (long) to);\n rc = ngx_http_replace_new_pending_buf(r, ctx, from, to, &cl);\n if (rc == NGX_ERROR) {\n return NGX_ERROR;\n }\n\n#if 1\n if (rc == NGX_BUSY) {\n dd(\"stop processing because of buffer size limit reached\");\n ctx->once = 1;\n ctx->copy_start = ctx->pos;\n ctx->copy_end = ctx->buf->last;\n ctx->pos = ctx->buf->last;\n return NGX_AGAIN;\n }\n#endif\n\n *ctx->last_pending = cl;\n ctx->last_pending = &cl->next;\n }\n\n ctx->pos = ctx->buf->last;\n\n ngx_http_replace_check_total_buffered(r, ctx, to - from,\n mto - mfrom);\n\n return NGX_AGAIN;\n }\n\n dd(\"from < ctx->stream_pos\");\n\n if (ctx->pending) {\n /* split ctx->pending into ctx->out and ctx->pending */\n\n if (ngx_http_replace_split_chain(r, ctx, &ctx->pending,\n &ctx->last_pending, from, &cl,\n &last, 1)\n != NGX_OK)\n {\n return NGX_ERROR;\n }\n\n if (ctx->pending) {\n dd(\"adjust pending: pos=%d, from=%d\",\n (int) ctx->pending->buf->file_pos, (int) from);\n\n ctx->total_buffered -= (size_t)\n (from - ctx->pending->buf->file_pos);\n\n *ctx->last_out = ctx->pending;\n ctx->last_out = ctx->last_pending;\n\n ctx->pending = NULL;\n ctx->last_pending = &ctx->pending;\n }\n\n if (cl) {\n dd(\"splitted ctx->pending into ctx->out and ctx->pending: %d\",\n (int) ctx->total_buffered);\n ctx->pending = cl;\n ctx->last_pending = last;\n }\n\n if (pending_matched && !ctx->pending2 && mto >= ctx->stream_pos) {\n dd(\"splitting ctx->pending into ctx->pending and ctx->free\");\n\n if (ngx_http_replace_split_chain(r, ctx, &ctx->pending,\n &ctx->last_pending, mfrom, &cl,\n &last, 0)\n != NGX_OK)\n {\n return NGX_ERROR;\n }\n\n if (cl) {\n ctx->total_buffered -= (size_t) (ctx->stream_pos - mfrom);\n\n dd(\"splitted ctx->pending into ctx->pending and ctx->free\");\n *last = ctx->free;\n ctx->free = cl;\n }\n }\n }\n\n if (ctx->pending2) {\n\n if (pending_matched) {\n dd(\"splitting ctx->pending2 into ctx->free and ctx->pending2\");\n\n if (ngx_http_replace_split_chain(r, ctx, &ctx->pending2,\n &ctx->last_pending2,\n mto, &cl, &last, 1)\n != NGX_OK)\n {\n return NGX_ERROR;\n }\n\n if (ctx->pending2) {\n\n dd(\"total buffered reduced by %d (was %d)\",\n (int) (mto - ctx->pending2->buf->file_pos),\n (int) ctx->total_buffered);\n\n ctx->total_buffered -= (size_t)\n (mto - ctx->pending2->buf->file_pos);\n\n *ctx->last_pending2 = ctx->free;\n ctx->free = ctx->pending2;\n\n ctx->pending2 = NULL;\n ctx->last_pending2 = &ctx->pending2;\n }\n\n if (cl) {\n ctx->pending2 = cl;\n ctx->last_pending2 = last;\n }\n }\n\n if (mto < to) {\n dd(\"new pending data to buffer to ctx->pending2: (%ld, %ld)\",\n (long) mto, (long) to);\n\n rc = ngx_http_replace_new_pending_buf(r, ctx, mto, to, &cl);\n if (rc == NGX_ERROR) {\n return NGX_ERROR;\n }\n\n#if 1\n if (rc == NGX_BUSY) {\n ctx->once = 1;\n\n if (ctx->pending) {\n *ctx->last_out = ctx->pending;\n ctx->last_out = ctx->last_pending;\n\n ctx->pending = NULL;\n ctx->last_pending = &ctx->pending;\n }\n\n ctx->copy_start = NULL;\n ctx->copy_end = NULL;\n\n if (ctx->pending2) {\n new_rematch = ctx->pending2;\n last_rematch = ctx->last_pending2;\n\n if (rematch) {\n ctx->rematch = rematch;\n }\n\n /* prepend cl to ctx->rematch */\n *last_rematch = ctx->rematch;\n ctx->rematch = new_rematch;\n\n ctx->pending2 = NULL;\n ctx->last_pending2 = &ctx->pending2;\n }\n\n ctx->pos = ctx->buf->pos + (mto - ctx->stream_pos);\n return new_rematch ? NGX_BUSY : NGX_OK;\n }\n#endif\n\n *ctx->last_pending2 = cl;\n ctx->last_pending2 = &cl->next;\n }\n\n ctx->copy_start = NULL;\n ctx->copy_end = NULL;\n\n ctx->pos = ctx->buf->last;\n\n ngx_http_replace_check_total_buffered(r, ctx, to - from,\n mto - mfrom);\n\n return NGX_AGAIN;\n }\n\n /* ctx->pending2 == NULL */\n\n if (pending_matched) {\n\n if (mto < to) {\n /* new pending data to buffer to ctx->pending2 */\n rc = ngx_http_replace_new_pending_buf(r, ctx, mto, to, &cl);\n if (rc == NGX_ERROR) {\n return NGX_ERROR;\n }\n\n if (rc == NGX_BUSY) {\n ctx->once = 1;\n\n if (ctx->pending) {\n *ctx->last_out = ctx->pending;\n ctx->last_out = ctx->last_pending;\n\n ctx->pending = NULL;\n ctx->last_pending = &ctx->pending;\n }\n\n ctx->copy_start = NULL;\n ctx->copy_end = NULL;\n ctx->pos = ctx->buf->pos + mto - ctx->stream_pos;\n\n return NGX_OK;\n }\n\n *ctx->last_pending2 = cl;\n ctx->last_pending2 = &cl->next;\n }\n\n /* otherwise no new data to buffer */\n\n } else {\n\n /* new pending data to buffer to ctx->pending */\n rc = ngx_http_replace_new_pending_buf(r, ctx, ctx->pos\n - ctx->buf->pos\n + ctx->stream_pos, to, &cl);\n if (rc == NGX_ERROR) {\n return NGX_ERROR;\n }\n\n#if 1\n if (rc == NGX_BUSY) {\n ctx->once = 1;\n\n if (ctx->pending) {\n *ctx->last_out = ctx->pending;\n ctx->last_out = ctx->last_pending;\n\n ctx->pending = NULL;\n ctx->last_pending = &ctx->pending;\n }\n\n ctx->copy_start = ctx->pos;\n ctx->copy_end = ctx->buf->last;\n ctx->pos = ctx->buf->last;\n\n return NGX_AGAIN;\n }\n#endif\n\n *ctx->last_pending = cl;\n ctx->last_pending = &cl->next;\n }\n\n ctx->copy_start = NULL;\n ctx->copy_end = NULL;\n\n ctx->pos = ctx->buf->last;\n\n ngx_http_replace_check_total_buffered(r, ctx, to - from,\n mto - mfrom);\n\n return NGX_AGAIN;\n\n case SRE_DECLINED:\n ctx->total_buffered = 0;\n\n return NGX_DECLINED;\n\n default:\n /* SRE_ERROR */\n return NGX_ERROR;\n }\n\n /* cannot reach here */\n}\n\n\nstatic void\nngx_http_replace_check_total_buffered(ngx_http_request_t *r,\n ngx_http_replace_ctx_t *ctx, sre_int_t len, sre_int_t mlen)\n{\n dd(\"total buffered: %d\", (int) ctx->total_buffered);\n\n if ((ssize_t) ctx->total_buffered != len - mlen) {\n ngx_log_error(NGX_LOG_ALERT, r->connection->log, 0,\n \"replace filter: ctx->total_buffered out of \"\n \"sync: it is %i but should be %uz\",\n ctx->total_buffered, (ngx_int_t) (len - mlen));\n\n#if (DDEBUG)\n assert(0);\n#endif\n }\n}\n" }, { "path": "src/ngx_http_replace_parse.h", "content": "#ifndef _NGX_HTTP_REPLACE_PARSE_H_INCLUDED_\n#define _NGX_HTTP_REPLACE_PARSE_H_INCLUDED_\n\n\n#include \"ngx_http_replace_filter_module.h\"\n\n\nngx_int_t ngx_http_replace_non_capturing_parse(ngx_http_request_t *r,\n ngx_http_replace_ctx_t *ctx, ngx_chain_t *rematch);\nngx_int_t ngx_http_replace_capturing_parse(ngx_http_request_t *r,\n ngx_http_replace_ctx_t *ctx, ngx_chain_t *rematch);\n\n\n#endif /* _NGX_HTTP_REPLACE_PARSE_H_INCLUDED_ */\n" }, { "path": "src/ngx_http_replace_script.c", "content": "\n/*\n * Copyright (C) Yichun Zhang (agentzh)\n */\n\n\n#ifndef DDEBUG\n#define DDEBUG 0\n#endif\n#include \"ddebug.h\"\n\n\n#include \"ngx_http_replace_script.h\"\n\n\nstatic void *ngx_http_replace_script_add_code(ngx_array_t *codes, size_t size);\nstatic size_t ngx_http_replace_script_copy_len_code(\n ngx_http_replace_script_engine_t *e);\nstatic size_t\n ngx_http_replace_script_copy_code(ngx_http_replace_script_engine_t *e);\nstatic ngx_int_t ngx_http_replace_script_add_copy_code(\n ngx_http_replace_script_compile_t *sc, ngx_str_t *value, ngx_uint_t last);\nstatic ngx_int_t\n ngx_http_replace_script_compile(ngx_http_replace_script_compile_t *sc);\nstatic ngx_int_t ngx_http_replace_script_add_capture_code(\n ngx_http_replace_script_compile_t *sc, ngx_uint_t n);\nstatic size_t ngx_http_replace_script_copy_capture_len_code(\n ngx_http_replace_script_engine_t *e);\nstatic size_t ngx_http_replace_script_copy_capture_code(\n ngx_http_replace_script_engine_t *e);\nstatic ngx_int_t\n ngx_http_replace_script_done(ngx_http_replace_script_compile_t *sc);\nstatic ngx_int_t ngx_http_replace_script_init_arrays(\n ngx_http_replace_script_compile_t *sc);\nstatic ngx_int_t\n ngx_http_replace_script_add_var_code(ngx_http_replace_script_compile_t *sc,\n ngx_str_t *name);\nstatic size_t\n ngx_http_replace_script_copy_var_len_code(\n ngx_http_replace_script_engine_t *e);\nstatic size_t\n ngx_http_replace_script_copy_var_code(ngx_http_replace_script_engine_t *e);\nstatic void ngx_http_replace_count_variables(u_char *src, size_t len,\n ngx_uint_t *ngxvars, ngx_uint_t *capvars);\n\n\nngx_int_t\nngx_http_replace_compile_complex_value(\n ngx_http_replace_compile_complex_value_t *ccv)\n{\n ngx_str_t *v;\n ngx_uint_t n, ngxvars, capvars;\n ngx_array_t lengths, values, *pl, *pv;\n\n ngx_http_replace_script_compile_t sc;\n\n v = ccv->value;\n\n ngx_http_replace_count_variables(v->data, v->len, &ngxvars, &capvars);\n\n ccv->complex_value->value = *v;\n ccv->complex_value->lengths = NULL;\n ccv->complex_value->values = NULL;\n\n if (capvars == 0 && ngxvars == 0) {\n return NGX_OK;\n }\n\n n = capvars * (2 * sizeof(ngx_http_replace_script_copy_code_t)\n + sizeof(ngx_http_replace_script_capture_code_t))\n + ngxvars * (2 * sizeof(ngx_http_replace_script_copy_code_t)\n + sizeof(ngx_http_replace_script_var_code_t))\n + sizeof(uintptr_t);\n\n if (ngx_array_init(&lengths, ccv->cf->pool, n, 1) != NGX_OK) {\n return NGX_ERROR;\n }\n\n n = capvars * (2 * sizeof(ngx_http_replace_script_copy_code_t)\n + sizeof(ngx_http_replace_script_capture_code_t))\n + ngxvars * (2 * sizeof(ngx_http_replace_script_var_code_t)\n + sizeof(ngx_http_replace_script_var_code_t))\n + sizeof(uintptr_t);\n\n if (ngx_array_init(&values, ccv->cf->pool, n, 1) != NGX_OK) {\n return NGX_ERROR;\n }\n\n pl = &lengths;\n pv = &values;\n\n ngx_memzero(&sc, sizeof(ngx_http_replace_script_compile_t));\n\n sc.cf = ccv->cf;\n sc.source = v;\n sc.lengths = &pl;\n sc.values = &pv;\n\n if (ngx_http_replace_script_compile(&sc) != NGX_OK) {\n ngx_array_destroy(&lengths);\n ngx_array_destroy(&values);\n return NGX_ERROR;\n }\n\n ccv->complex_value->lengths = lengths.elts;\n ccv->complex_value->values = values.elts;\n ccv->complex_value->capture_variables = sc.capture_variables;\n\n return NGX_OK;\n}\n\n\nngx_int_t\nngx_http_replace_complex_value(ngx_http_request_t *r,\n ngx_chain_t *captured, sre_uint_t ncaps, sre_int_t *cap,\n ngx_http_replace_complex_value_t *val, ngx_str_t *value)\n{\n size_t len;\n ngx_http_replace_script_code_pt code;\n ngx_http_replace_script_len_code_pt lcode;\n ngx_http_replace_script_engine_t e;\n\n if (val->lengths == NULL) {\n *value = val->value;\n return NGX_OK;\n }\n\n ngx_memzero(&e, sizeof(ngx_http_replace_script_engine_t));\n\n e.request = r;\n e.ncaptures = (ncaps + 1) * 2;\n e.captures_data = captured;\n e.captures = cap;\n e.ip = val->lengths;\n\n len = 0;\n\n while (*(uintptr_t *) e.ip) {\n lcode = *(ngx_http_replace_script_len_code_pt *) e.ip;\n len += lcode(&e);\n }\n\n value->len = len;\n value->data = ngx_pnalloc(r->pool, len);\n if (value->data == NULL) {\n return NGX_ERROR;\n }\n\n e.ip = val->values;\n e.pos = value->data;\n\n while (*(uintptr_t *) e.ip) {\n code = *(ngx_http_replace_script_code_pt *) e.ip;\n code((ngx_http_replace_script_engine_t *) &e);\n }\n\n return NGX_OK;\n}\n\n\nstatic ngx_int_t\nngx_http_replace_script_compile(ngx_http_replace_script_compile_t *sc)\n{\n u_char ch;\n ngx_str_t name;\n ngx_uint_t i, bracket;\n unsigned num_var;\n ngx_uint_t n = 0;\n\n if (ngx_http_replace_script_init_arrays(sc) != NGX_OK) {\n return NGX_ERROR;\n }\n\n for (i = 0; i < sc->source->len; /* void */ ) {\n\n name.len = 0;\n\n if (sc->source->data[i] == '$') {\n\n if (++i == sc->source->len) {\n goto invalid_variable;\n }\n\n if (sc->source->data[i] == '$') {\n name.data = &sc->source->data[i];\n i++;\n name.len++;\n sc->size += name.len;\n\n if (ngx_http_replace_script_add_copy_code(sc, &name,\n (i == sc->source->len))\n != NGX_OK)\n {\n return NGX_ERROR;\n }\n\n continue;\n }\n\n if ((sc->source->data[i] >= '1' && sc->source->data[i] <= '9')\n || sc->source->data[i] == '&')\n {\n num_var = 1;\n n = 0;\n\n } else {\n num_var = 0;\n }\n\n if (sc->source->data[i] == '{') {\n bracket = 1;\n\n if (++i == sc->source->len) {\n goto invalid_variable;\n }\n\n if ((sc->source->data[i] >= '1' && sc->source->data[i] <= '9')\n || sc->source->data[i] == '&')\n {\n num_var = 1;\n n = 0;\n }\n\n name.data = &sc->source->data[i];\n\n } else {\n bracket = 0;\n name.data = &sc->source->data[i];\n }\n\n for ( /* void */ ; i < sc->source->len; i++, name.len++) {\n ch = sc->source->data[i];\n\n if (ch == '}' && bracket) {\n i++;\n bracket = 0;\n break;\n }\n\n if (num_var) {\n if (ch >= '0' && ch <= '9') {\n n = n * 10 + (ch - '0');\n continue;\n }\n\n if (ch == '&') {\n i++;\n name.len++;\n }\n\n break;\n }\n\n /* not a number variable like $1, $2, etc */\n\n if ((ch >= 'A' && ch <= 'Z')\n || (ch >= 'a' && ch <= 'z')\n || (ch >= '0' && ch <= '9')\n || ch == '_')\n {\n continue;\n }\n\n break;\n }\n\n if (bracket) {\n ngx_log_error(NGX_LOG_ERR, sc->cf->log, 0,\n \"the closing bracket in \\\"%V\\\" \"\n \"variable is missing\", &name);\n return NGX_ERROR;\n }\n\n if (name.len == 0) {\n goto invalid_variable;\n }\n\n if (num_var) {\n dd(\"found numbered capturing variable \\\"%.*s\\\"\",\n (int) name.len, name.data);\n\n sc->capture_variables++;\n\n if (ngx_http_replace_script_add_capture_code(sc, n) != NGX_OK) {\n return NGX_ERROR;\n }\n\n } else {\n sc->nginx_variables++;\n\n if (ngx_http_replace_script_add_var_code(sc, &name) != NGX_OK) {\n return NGX_ERROR;\n }\n }\n\n continue;\n }\n\n name.data = &sc->source->data[i];\n\n while (i < sc->source->len) {\n\n if (sc->source->data[i] == '$') {\n break;\n }\n\n i++;\n name.len++;\n }\n\n sc->size += name.len;\n\n if (ngx_http_replace_script_add_copy_code(sc, &name,\n (i == sc->source->len))\n != NGX_OK)\n {\n return NGX_ERROR;\n }\n }\n\n return ngx_http_replace_script_done(sc);\n\ninvalid_variable:\n\n ngx_log_error(NGX_LOG_ERR, sc->cf->log, 0,\n \"replace script: invalid capturing variable name found \"\n \"in \\\"%V\\\"\", sc->source);\n\n return NGX_ERROR;\n}\n\n\nstatic ngx_int_t\nngx_http_replace_script_add_copy_code(ngx_http_replace_script_compile_t *sc,\n ngx_str_t *value, ngx_uint_t last)\n{\n size_t size, len;\n ngx_http_replace_script_copy_code_t *code;\n\n len = value->len;\n\n code = ngx_http_replace_script_add_code(*sc->lengths,\n sizeof(ngx_http_replace_script_copy_code_t));\n if (code == NULL) {\n return NGX_ERROR;\n }\n\n code->code = (ngx_http_replace_script_code_pt)\n ngx_http_replace_script_copy_len_code;\n code->len = len;\n\n size = (sizeof(ngx_http_replace_script_copy_code_t) + len +\n sizeof(uintptr_t) - 1) & ~(sizeof(uintptr_t) - 1);\n\n code = ngx_http_replace_script_add_code(*sc->values, size);\n if (code == NULL) {\n return NGX_ERROR;\n }\n\n code->code = (ngx_http_replace_script_code_pt)\n ngx_http_replace_script_copy_code;\n code->len = len;\n\n ngx_memcpy((u_char *) code + sizeof(ngx_http_replace_script_copy_code_t),\n value->data, value->len);\n\n return NGX_OK;\n}\n\n\nstatic size_t\nngx_http_replace_script_copy_len_code(ngx_http_replace_script_engine_t *e)\n{\n ngx_http_replace_script_copy_code_t *code;\n\n code = (ngx_http_replace_script_copy_code_t *) e->ip;\n\n e->ip += sizeof(ngx_http_replace_script_copy_code_t);\n\n return code->len;\n}\n\n\nstatic size_t\nngx_http_replace_script_copy_code(ngx_http_replace_script_engine_t *e)\n{\n u_char *p;\n\n ngx_http_replace_script_copy_code_t *code;\n\n code = (ngx_http_replace_script_copy_code_t *) e->ip;\n\n p = e->pos;\n\n if (!e->skip) {\n e->pos = ngx_copy(p, e->ip\n + sizeof(ngx_http_replace_script_copy_code_t),\n code->len);\n }\n\n e->ip += sizeof(ngx_http_replace_script_copy_code_t)\n + ((code->len + sizeof(uintptr_t) - 1) & ~(sizeof(uintptr_t) - 1));\n\n ngx_log_debug2(NGX_LOG_DEBUG_HTTP, e->request->connection->log, 0,\n \"replace script copy: \\\"%*s\\\"\", e->pos - p, p);\n\n return 0;\n}\n\n\nstatic ngx_int_t\nngx_http_replace_script_add_capture_code(ngx_http_replace_script_compile_t *sc,\n ngx_uint_t n)\n{\n ngx_http_replace_script_capture_code_t *code;\n\n code = ngx_http_replace_script_add_code(*sc->lengths,\n sizeof(ngx_http_replace_script_capture_code_t));\n if (code == NULL) {\n return NGX_ERROR;\n }\n\n code->code = (ngx_http_replace_script_code_pt)\n ngx_http_replace_script_copy_capture_len_code;\n code->n = 2 * n;\n\n code = ngx_http_replace_script_add_code(*sc->values,\n sizeof(ngx_http_replace_script_capture_code_t));\n if (code == NULL) {\n return NGX_ERROR;\n }\n\n code->code = (ngx_http_replace_script_code_pt)\n ngx_http_replace_script_copy_capture_code;\n code->n = 2 * n;\n\n return NGX_OK;\n}\n\n\nstatic size_t\nngx_http_replace_script_copy_capture_len_code(\n ngx_http_replace_script_engine_t *e)\n{\n sre_int_t *cap;\n ngx_uint_t n;\n\n ngx_http_replace_script_capture_code_t *code;\n\n code = (ngx_http_replace_script_capture_code_t *) e->ip;\n\n e->ip += sizeof(ngx_http_replace_script_capture_code_t);\n\n n = code->n;\n\n dd(\"group index: %d, ncaptures: %d\", (int) n, (int) e->ncaptures);\n\n if (n + 1 < e->ncaptures) {\n cap = e->captures;\n return cap[n + 1] - cap[n];\n }\n\n return 0;\n}\n\n\nstatic size_t\nngx_http_replace_script_copy_capture_code(ngx_http_replace_script_engine_t *e)\n{\n sre_int_t *cap, from, to, len;\n u_char *p;\n#if (NGX_DEBUG)\n u_char *pos;\n#endif\n ngx_uint_t n;\n ngx_chain_t *cl;\n\n ngx_http_replace_script_capture_code_t *code;\n\n code = (ngx_http_replace_script_capture_code_t *) e->ip;\n\n e->ip += sizeof(ngx_http_replace_script_capture_code_t);\n\n n = code->n;\n\n#if (NGX_DEBUG)\n pos = e->pos;\n#endif\n\n if (n < e->ncaptures) {\n\n cap = e->captures;\n from = cap[n];\n to = cap[n + 1];\n\n dd(\"captures data: %p\", e->captures_data);\n\n for (cl = e->captures_data; cl; cl = cl->next) {\n\n if (from >= cl->buf->file_last) {\n continue;\n }\n\n /* from < cl->buf->file_last */\n\n if (to <= cl->buf->file_pos) {\n break;\n }\n\n p = cl->buf->pos + (from - cl->buf->file_pos);\n len = ngx_min(cl->buf->file_last, to) - from;\n e->pos = ngx_copy(e->pos, p, len);\n from += len;\n }\n }\n\n#if (NGX_DEBUG)\n ngx_log_debug2(NGX_LOG_DEBUG_HTTP, e->request->connection->log, 0,\n \"replace script capture: \\\"%*s\\\"\", e->pos - pos, pos);\n#endif\n\n return 0;\n}\n\n\nstatic ngx_int_t\nngx_http_replace_script_init_arrays(ngx_http_replace_script_compile_t *sc)\n{\n ngx_uint_t n;\n\n if (*sc->lengths == NULL) {\n n = sc->capture_variables\n * (2 * sizeof(ngx_http_replace_script_copy_code_t)\n + sizeof(ngx_http_replace_script_capture_code_t))\n + sc->nginx_variables\n * (2 * sizeof(ngx_http_replace_script_copy_code_t)\n + sizeof(ngx_http_replace_script_var_code_t))\n + sizeof(uintptr_t);\n\n *sc->lengths = ngx_array_create(sc->cf->pool, n, 1);\n if (*sc->lengths == NULL) {\n return NGX_ERROR;\n }\n }\n\n if (*sc->values == NULL) {\n n = sc->capture_variables\n * (2 * sizeof(ngx_http_replace_script_copy_code_t)\n + sizeof(ngx_http_replace_script_capture_code_t))\n + sc->nginx_variables\n * (2 * sizeof(ngx_http_replace_script_copy_code_t)\n + sizeof(ngx_http_replace_script_var_code_t))\n + sizeof(uintptr_t);\n\n *sc->values = ngx_array_create(sc->cf->pool, n, 1);\n if (*sc->values == NULL) {\n return NGX_ERROR;\n }\n }\n\n sc->nginx_variables = 0;\n sc->capture_variables = 0;\n\n return NGX_OK;\n}\n\n\nstatic ngx_int_t\nngx_http_replace_script_done(ngx_http_replace_script_compile_t *sc)\n{\n uintptr_t *code;\n\n code = ngx_http_replace_script_add_code(*sc->lengths,\n sizeof(uintptr_t));\n if (code == NULL) {\n return NGX_ERROR;\n }\n\n *code = (uintptr_t) NULL;\n\n code = ngx_http_replace_script_add_code(*sc->values, sizeof(uintptr_t));\n if (code == NULL) {\n return NGX_ERROR;\n }\n\n *code = (uintptr_t) NULL;\n\n return NGX_OK;\n}\n\n\nstatic void *\nngx_http_replace_script_add_code(ngx_array_t *codes, size_t size)\n{\n return ngx_array_push_n(codes, size);\n}\n\n\nstatic ngx_int_t\nngx_http_replace_script_add_var_code(ngx_http_replace_script_compile_t *sc,\n ngx_str_t *name)\n{\n ngx_int_t index;\n ngx_http_replace_script_var_code_t *code;\n\n index = ngx_http_get_variable_index(sc->cf, name);\n\n if (index == NGX_ERROR) {\n return NGX_ERROR;\n }\n\n code = ngx_http_replace_script_add_code(*sc->lengths,\n sizeof(ngx_http_replace_script_var_code_t));\n if (code == NULL) {\n return NGX_ERROR;\n }\n\n code->code = (ngx_http_replace_script_code_pt)\n ngx_http_replace_script_copy_var_len_code;\n\n code->index = (uintptr_t) index;\n\n code = ngx_http_replace_script_add_code(*sc->values,\n sizeof(ngx_http_replace_script_var_code_t));\n if (code == NULL) {\n return NGX_ERROR;\n }\n\n code->code = (ngx_http_replace_script_code_pt)\n ngx_http_replace_script_copy_var_code;\n code->index = (uintptr_t) index;\n\n return NGX_OK;\n}\n\n\nstatic size_t\nngx_http_replace_script_copy_var_len_code(ngx_http_replace_script_engine_t *e)\n{\n ngx_http_variable_value_t *value;\n ngx_http_replace_script_var_code_t *code;\n\n code = (ngx_http_replace_script_var_code_t *) e->ip;\n\n e->ip += sizeof(ngx_http_replace_script_var_code_t);\n\n value = ngx_http_get_indexed_variable(e->request, code->index);\n\n if (value && !value->not_found) {\n return value->len;\n }\n\n return 0;\n}\n\n\nstatic size_t\nngx_http_replace_script_copy_var_code(ngx_http_replace_script_engine_t *e)\n{\n u_char *p;\n ngx_http_variable_value_t *value;\n ngx_http_replace_script_var_code_t *code;\n\n code = (ngx_http_replace_script_var_code_t *) e->ip;\n\n e->ip += sizeof(ngx_http_replace_script_var_code_t);\n\n if (!e->skip) {\n\n value = ngx_http_get_indexed_variable(e->request, code->index);\n\n if (value && !value->not_found) {\n p = e->pos;\n e->pos = ngx_copy(p, value->data, value->len);\n\n ngx_log_debug2(NGX_LOG_DEBUG_HTTP,\n e->request->connection->log, 0,\n \"http replace script var: \\\"%*s\\\"\", e->pos - p, p);\n }\n }\n\n return 0;\n}\n\n\nstatic void\nngx_http_replace_count_variables(u_char *src, size_t len,\n ngx_uint_t *ngxvars, ngx_uint_t *capvars)\n{\n ngx_uint_t i;\n unsigned var = 0;\n u_char c;\n\n *ngxvars = 0;\n *capvars = 0;\n\n for (i = 0; i < len; i++) {\n c = src[i];\n\n if (c == '$') {\n if (var) {\n var = 0;\n\n } else {\n var = 1;\n }\n\n } else if (var) {\n if ((c >= '1' && c <= '9') || c == '&') {\n (*capvars)++;\n\n } else {\n (*ngxvars)++;\n }\n\n var = 0;\n }\n }\n}\n\n/* vi:set ft=c ts=4 sw=4 et fdm=marker: */\n" }, { "path": "src/ngx_http_replace_script.h", "content": "\n/*\n * Copyright (C) Yichun Zhang (agentzh)\n */\n\n\n#ifndef _NGX_HTTP_REPLACE_SCRIPT_H_INCLUDED_\n#define _NGX_HTTP_REPLACE_SCRIPT_H_INCLUDED_\n\n\n#include \n#include \n#include \n#include \n#include \n\n\ntypedef struct {\n ngx_conf_t *cf;\n ngx_str_t *source;\n\n ngx_array_t **lengths;\n ngx_array_t **values;\n\n ngx_uint_t capture_variables; /* captures $1, $2, etc */\n ngx_uint_t nginx_variables; /* nginx variables */\n ngx_uint_t size;\n} ngx_http_replace_script_compile_t;\n\n\ntypedef struct {\n ngx_str_t value;\n void *lengths;\n void *values;\n ngx_uint_t capture_variables;\n} ngx_http_replace_complex_value_t;\n\n\ntypedef struct {\n ngx_conf_t *cf;\n ngx_str_t *value;\n\n ngx_http_replace_complex_value_t *complex_value;\n} ngx_http_replace_compile_complex_value_t;\n\n\ntypedef struct {\n u_char *ip;\n u_char *pos;\n\n ngx_str_t buf;\n\n sre_int_t *captures;\n ngx_uint_t ncaptures;\n ngx_chain_t *captures_data;\n\n unsigned skip:1;\n\n ngx_http_request_t *request;\n} ngx_http_replace_script_engine_t;\n\n\ntypedef size_t (*ngx_http_replace_script_code_pt)\n (ngx_http_replace_script_engine_t *e);\n\ntypedef size_t (*ngx_http_replace_script_len_code_pt)\n (ngx_http_replace_script_engine_t *e);\n\n\ntypedef struct {\n ngx_http_replace_script_code_pt code;\n uintptr_t len;\n} ngx_http_replace_script_copy_code_t;\n\n\ntypedef struct {\n ngx_http_replace_script_code_pt code;\n uintptr_t n;\n} ngx_http_replace_script_capture_code_t;\n\n\ntypedef struct {\n ngx_http_replace_script_code_pt code;\n uintptr_t index;\n} ngx_http_replace_script_var_code_t;\n\n\nngx_int_t ngx_http_replace_compile_complex_value(\n ngx_http_replace_compile_complex_value_t *ccv);\nngx_int_t ngx_http_replace_complex_value(ngx_http_request_t *r,\n ngx_chain_t *captured, sre_uint_t ncaps, sre_int_t *cap,\n ngx_http_replace_complex_value_t *val, ngx_str_t *value);\n\n\n#endif /* _NGX_HTTP_REPLACE_SCRIPT_H_INCLUDED_ */\n\n/* vi:set ft=c ts=4 sw=4 et fdm=marker: */\n" }, { "path": "src/ngx_http_replace_util.c", "content": "\n/*\n * Copyright (C) Yichun Zhang (agentzh)\n */\n\n\n#ifndef DDEBUG\n#define DDEBUG 0\n#endif\n#include \"ddebug.h\"\n\n\n#include \"ngx_http_replace_util.h\"\n\n\nngx_chain_t *\nngx_http_replace_get_free_buf(ngx_pool_t *p, ngx_chain_t **free)\n{\n ngx_chain_t *cl;\n\n cl = ngx_chain_get_free_buf(p, free);\n if (cl == NULL) {\n return cl;\n }\n\n ngx_memzero(cl->buf, sizeof(ngx_buf_t));\n\n cl->buf->tag = (ngx_buf_tag_t) &ngx_http_replace_filter_module;\n\n return cl;\n}\n\n\nngx_int_t\nngx_http_replace_split_chain(ngx_http_request_t *r, ngx_http_replace_ctx_t *ctx,\n ngx_chain_t **pa, ngx_chain_t ***plast_a, sre_int_t split, ngx_chain_t **pb,\n ngx_chain_t ***plast_b, unsigned b_sane)\n{\n sre_int_t file_last;\n ngx_chain_t *cl, *newcl, **ll;\n\n#if 0\n b_sane = 0;\n#endif\n\n ll = pa;\n for (cl = *pa; cl; ll = &cl->next, cl = cl->next) {\n if (cl->buf->file_last > split) {\n /* found an overlap */\n\n if (cl->buf->file_pos < split) {\n\n dd(\"adjust cl buf (b_sane=%d): \\\"%.*s\\\"\", b_sane,\n (int) ngx_buf_size(cl->buf), cl->buf->pos);\n\n file_last = cl->buf->file_last;\n cl->buf->last -= file_last - split;\n cl->buf->file_last = split;\n\n dd(\"adjusted cl buf (next=%p): %.*s\",\n cl->next,\n (int) ngx_buf_size(cl->buf), cl->buf->pos);\n\n /* build the b chain */\n if (b_sane) {\n newcl = ngx_http_replace_get_free_buf(r->pool,\n &ctx->free);\n if (newcl == NULL) {\n return NGX_ERROR;\n }\n\n newcl->buf->memory = 1;\n newcl->buf->pos = cl->buf->last;\n newcl->buf->last = cl->buf->last + file_last - split;\n newcl->buf->file_pos = split;\n newcl->buf->file_last = file_last;\n\n newcl->next = cl->next;\n\n *pb = newcl;\n if (plast_b) {\n if (cl->next) {\n *plast_b = *plast_a;\n\n } else {\n *plast_b = &newcl->next;\n }\n }\n\n } else {\n *pb = cl->next;\n if (plast_b) {\n *plast_b = *plast_a;\n }\n }\n\n /* truncate the a chain */\n *plast_a = &cl->next;\n cl->next = NULL;\n\n return NGX_OK;\n }\n\n /* build the b chain */\n *pb = cl;\n if (plast_b) {\n *plast_b = *plast_a;\n }\n\n /* truncate the a chain */\n *plast_a = ll;\n *ll = NULL;\n\n return NGX_OK;\n }\n }\n\n /* missed */\n\n *pb = NULL;\n if (plast_b) {\n *plast_b = pb;\n }\n\n return NGX_OK;\n}\n\n\nngx_int_t\nngx_http_replace_new_pending_buf(ngx_http_request_t *r,\n ngx_http_replace_ctx_t *ctx, sre_int_t from, sre_int_t to,\n ngx_chain_t **out)\n{\n size_t len;\n ngx_buf_t *b;\n ngx_chain_t *cl;\n\n ngx_http_replace_loc_conf_t *rlcf;\n\n if (from < ctx->stream_pos) {\n from = ctx->stream_pos;\n }\n\n len = (size_t) (to - from);\n if (len == 0) {\n return NGX_ERROR;\n }\n\n ctx->total_buffered += len;\n\n rlcf = ngx_http_get_module_loc_conf(r, ngx_http_replace_filter_module);\n\n if (ctx->total_buffered > rlcf->max_buffered_size) {\n#if 1\n ngx_log_error(NGX_LOG_ALERT, r->connection->log, 0,\n \"replace filter: exceeding \"\n \"replace_filter_max_buffered_size (%uz): %uz\",\n rlcf->max_buffered_size, ctx->total_buffered);\n return NGX_BUSY;\n#endif\n }\n\n cl = ngx_http_replace_get_free_buf(r->pool, &ctx->free);\n if (cl == NULL) {\n return NGX_ERROR;\n }\n\n b = cl->buf;\n b->temporary = 1;\n\n /* abuse the file_pos and file_last fields here */\n b->file_pos = from;\n b->file_last = to;\n\n b->start = ngx_palloc(r->pool, len);\n if (b->start == NULL) {\n return NGX_ERROR;\n }\n b->end = b->start + len;\n\n b->pos = b->start;\n b->last = ngx_copy(b->pos, ctx->buf->pos + from - ctx->stream_pos, len);\n\n dd(\"buffered pending data: stream_pos=%ld (%ld, %ld): %.*s\",\n (long) ctx->stream_pos, (long) from, (long) to,\n (int) len, ctx->buf->pos + from - ctx->stream_pos);\n\n *out = cl;\n return NGX_OK;\n}\n\n\n#if (DDEBUG)\nvoid\nngx_http_replace_dump_chain(const char *prefix, ngx_chain_t **pcl,\n ngx_chain_t **last)\n{\n ngx_chain_t *cl;\n\n if (*pcl == NULL) {\n dd(\"%s buf empty\", prefix);\n if (last && last != pcl) {\n dd(\"BAD last %s\", prefix);\n assert(0);\n }\n }\n\n for (cl = *pcl; cl; cl = cl->next) {\n dd(\"%s buf: \\\"%.*s\\\"\", prefix, (int) ngx_buf_size(cl->buf),\n cl->buf->pos);\n\n if (cl->next == NULL) {\n if (last && last != &cl->next) {\n dd(\"BAD last %s\", prefix);\n assert(0);\n }\n }\n }\n}\n#endif /* DDEBUG */\n" }, { "path": "src/ngx_http_replace_util.h", "content": "#ifndef _NGX_HTTP_REPLACE_UTIL_H_INCLUDED_\n#define _NGX_HTTP_REPLACE_UTIL_H_INCLUDED_\n\n\n#include \"ngx_http_replace_filter_module.h\"\n\n\nngx_chain_t *ngx_http_replace_get_free_buf(ngx_pool_t *p,\n ngx_chain_t **free);\nngx_int_t ngx_http_replace_split_chain(ngx_http_request_t *r,\n ngx_http_replace_ctx_t *ctx, ngx_chain_t **pa, ngx_chain_t ***plast_a,\n sre_int_t split, ngx_chain_t **pb, ngx_chain_t ***plast_b, unsigned b_sane);\nngx_int_t ngx_http_replace_new_pending_buf(ngx_http_request_t *r,\n ngx_http_replace_ctx_t *ctx, sre_int_t from, sre_int_t to,\n ngx_chain_t **out);\n#if (DDEBUG)\nvoid ngx_http_replace_dump_chain(const char *prefix, ngx_chain_t **pcl,\n ngx_chain_t **last);\n#endif\n\n\n#endif /* _NGX_HTTP_REPLACE_UTIL_H_INCLUDED_ */\n" }, { "path": "t/01-sanity.t", "content": "# vim:set ft= ts=4 sw=4 et fdm=marker:\n\nuse lib 'lib';\nuse Test::Nginx::Socket;\n\n#worker_connections(1014);\n#master_on();\n#workers(2);\n#log_level('warn');\n\nrepeat_each(2);\n\n#no_shuffle();\n\nplan tests => repeat_each() * (blocks() * 4 + 1);\n\nour $StapOutputChains = <<'_EOC_';\nglobal active\n\nF(ngx_http_handler) {\n active = 1\n}\n\n/*\nF(ngx_http_write_filter) {\n if (active && pid() == target()) {\n printf(\"http writer filter: %s\\n\", ngx_chain_dump($in))\n }\n}\n*/\n\nF(ngx_http_chunked_body_filter) {\n if (active && pid() == target()) {\n printf(\"http chunked filter: %s\\n\", ngx_chain_dump($in))\n }\n}\n\nF(ngx_http_replace_output) {\n if (active && pid() == target()) {\n printf(\"http replace output: %s\\n\", ngx_chain_dump($ctx->out))\n }\n}\n\nprobe syscall.writev {\n if (active && pid() == target()) {\n printf(\"writev(%s)\", ngx_iovec_dump($vec, $vlen))\n /*\n for (i = 0; i < $vlen; i++) {\n printf(\" %p [%s]\", $vec[i]->iov_base, text_str(user_string_n($vec[i]->iov_base, $vec[i]->iov_len)))\n }\n */\n }\n}\n\nprobe syscall.writev.return {\n if (active && pid() == target()) {\n printf(\" = %s\\n\", retstr)\n }\n}\n\n_EOC_\n\n#no_diff();\n#no_long_string();\nrun_tests();\n\n__DATA__\n\n=== TEST 1: ambiguous pattern\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo abcabcabde;\n replace_filter abcabd X;\n }\n--- request\nGET /t\n\n--- stap\nF(ngx_http_replace_non_capturing_parse) {\n println(\"non capturing parse\")\n}\n\nF(ngx_http_replace_capturing_parse) {\n println(\"capturing parse\")\n}\n\nF(ngx_http_replace_complex_value) {\n println(\"complex value\")\n}\n\n--- stap_out_like chop\n^(non capturing parse\\n)+$\n\n--- response_body\nabcXe\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 2: ambiguous pattern\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo -n ababac;\n replace_filter abac X;\n }\n--- request\nGET /t\n--- response_body chop\nabX\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 3: alt\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo abc;\n replace_filter 'ab|abc' X;\n }\n--- request\nGET /t\n--- response_body\nXc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 4: caseless\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo abcabcaBde;\n replace_filter abCabd X i;\n }\n--- request\nGET /t\n--- response_body\nabcXe\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 5: case sensitive (no match)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo abcabcaBde;\n replace_filter abCabd X;\n }\n--- request\nGET /t\n--- response_body\nabcabcaBde\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 6: 1-byte chain bufs\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 3;\n\n location = /t {\n echo -n a;\n echo -n b;\n echo -n a;\n echo -n b;\n echo -n a;\n echo -n c;\n echo d;\n replace_filter abac X;\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- stap3\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1413\") {\n //printf(\"chain: %s\", ngx_chain_dump($ctx->busy))\n print_ubacktrace()\n}\n\n--- response_body\nabXd\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 7: 2-byte chain bufs\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 2;\n\n location = /t {\n echo -n ab;\n echo -n ab;\n echo -n ac;\n echo d;\n replace_filter abac X;\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- response_body\nabXd\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 8: 3-byte chain bufs\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 3;\n\n location = /t {\n echo -n aba;\n echo -n bac;\n echo d;\n replace_filter abac X;\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- response_body\nabXd\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 9: 3-byte chain bufs (more)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 4;\n\n location = /t {\n echo -n aba;\n echo -n bac;\n echo d;\n replace_filter abacd X;\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- response_body\nabX\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 10: once by default (1st char matched)\n--- config\n replace_filter_max_buffered_size 0;\n default_type text/html;\n location /t {\n echo abcabcabde;\n replace_filter a X;\n }\n--- request\nGET /t\n--- response_body\nXbcabcabde\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 11: once by default (2nd char matched)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo abcabcabde;\n replace_filter b X;\n }\n--- request\nGET /t\n--- response_body\naXcabcabde\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 12: global substitution\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo bbc;\n replace_filter b X g;\n }\n--- request\nGET /t\n--- response_body\nXXc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 13: global substitution\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo abcabcabde;\n replace_filter b X g;\n }\n--- request\nGET /t\n--- response_body\naXcaXcaXde\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 14: global substitution (empty captures)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo -n abcabcabde;\n replace_filter [0-9]* X g;\n }\n--- request\nGET /t\n--- response_body chop\nXaXbXcXaXbXcXaXbXdXeX\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 15: global substitution (empty captures, splitted)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo -n ab;\n echo -n cab;\n echo -n c;\n echo -n abde;\n replace_filter [0-9]* X g;\n }\n--- request\nGET /t\n--- response_body chop\nXaXbXcXaXbXcXaXbXdXeX\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 16: global substitution (\\d+)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo \"hello1234, 56 world\";\n replace_filter \\d+ X g;\n }\n--- request\nGET /t\n--- response_body\nhelloX, X world\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 17: replace_filter_types default to text/html\n--- config\n default_type text/plain;\n location /t {\n echo abc;\n replace_filter b X;\n }\n--- request\nGET /t\n--- response_body\nabc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 18: custom replace_filter_types\n--- config\n default_type text/plain;\n location /t {\n echo abc;\n replace_filter b X;\n replace_filter_types text/plain;\n }\n--- request\nGET /t\n--- response_body\naXc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 19: multiple replace_filter_types settings\n--- config\n default_type text/plain;\n location /t {\n echo abc;\n replace_filter b X;\n replace_filter_types text/css text/plain;\n }\n--- request\nGET /t\n--- response_body\naXc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 20: trim leading spaces\n--- config\n replace_filter_max_buffered_size 0;\n default_type text/html;\n location /a.html {\n replace_filter '^\\s+' '' g;\n }\n--- user_files\n>>> a.html\n hello, world \nblah yeah\nhello \n baby!\n \nabc\n--- request\nGET /a.html\n--- response_body\nhello, world \nblah yeah\nhello \nbaby!\nabc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 21: trim trailing spaces\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /a.html {\n replace_filter '\\s+$' '' g;\n }\n--- user_files\n>>> a.html\n hello, world \nblah yeah\nhello \n baby!\n \nabc\n--- request\nGET /a.html\n--- response_body chop\n hello, world\nblah yeah\nhello\n baby!\nabc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 22: trim both leading and trailing spaces\n--- config\n replace_filter_max_buffered_size 0;\n default_type text/html;\n location /a.html {\n replace_filter '^\\s+|\\s+$' '' g;\n }\n--- user_files\n>>> a.html\n hello, world \nblah yeah\nhello \n baby!\n \nabc\n--- request\nGET /a.html\n--- response_body chop\nhello, world\nblah yeah\nhello\nbaby!\nabc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 23: pure flush buf in the stream (no data)\n--- config\n replace_filter_max_buffered_size 0;\n default_type text/html;\n location = /t {\n echo_flush;\n replace_filter 'a' 'X' g;\n }\n--- request\nGET /t\n--- response_body chop\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 24: pure flush buf in the stream (with data)\n--- config\n replace_filter_max_buffered_size 0;\n default_type text/html;\n location = /t {\n echo a;\n echo_flush;\n replace_filter 'a' 'X' g;\n }\n--- request\nGET /t\n--- stap3 eval: $::StapOutputChains\n--- stap2\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:539\") {\n printf(\"chain: %s\", ngx_chain_dump($ctx->busy))\n //print_ubacktrace()\n}\n--- response_body\nX\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 25: trim both leading and trailing spaces (1 byte at a time)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 1;\n location = /t {\n echo -n 'a';\n echo ' ';\n echo \"b\";\n replace_filter '^\\s+|\\s+$' '' g;\n }\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body chop\na\nb\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 26: trim both leading and trailing spaces (1 byte at a time), no \\s for $\n--- config\n replace_filter_max_buffered_size 1;\n default_type text/html;\n location = /t {\n echo -n 'a';\n echo ' ';\n echo \"b\";\n replace_filter '^\\s+| +$' '' g;\n }\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\na\nb\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 27: trim both leading and trailing spaces (1 byte at a time)\n--- config\n replace_filter_max_buffered_size 4;\n default_type text/html;\n location /a.html {\n internal;\n }\n\n location = /t {\n content_by_lua '\n local res = ngx.location.capture(\"/a.html\")\n local txt = res.body\n for i = 1, string.len(txt) do\n ngx.print(string.sub(txt, i, i))\n ngx.flush(true)\n end\n ';\n replace_filter '^\\s+|\\s+$' '' g;\n }\n--- user_files\n>>> a.html\n hello, world \nblah yeah\nhello \n baby!\n \nabc\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1438\") {\n //printf(\"chain: %s\", ngx_chain_dump($ctx->busy))\n print_ubacktrace()\n exit()\n}\n\n--- request\nGET /t\n--- response_body chop\nhello, world\nblah yeah\nhello\nbaby!\nabc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 28: \\b at the border\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo -n a;\n echo b;\n replace_filter '\\bb|a' X g;\n }\n--- request\nGET /t\n--- response_body\nXb\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 29: \\B at the border\n--- config\n replace_filter_max_buffered_size 0;\n default_type text/html;\n location /t {\n echo -n a;\n echo ',';\n replace_filter '\\B,|a' X g;\n }\n--- request\nGET /t\n--- response_body\nX,\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 30: \\A at the border\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo -n a;\n echo 'b';\n replace_filter '\\Ab|a' X g;\n }\n--- request\nGET /t\n--- response_body\nXb\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 31: memory bufs with last_buf=1\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n return 200 \"abc\";\n replace_filter \\w+ X;\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- response_body chop\nX\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 32: trim both leading and trailing spaces (2 bytes at a time)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 4;\n location /a.html {\n internal;\n }\n\n location = /t {\n content_by_lua '\n local res = ngx.location.capture(\"/a.html\")\n local txt = res.body\n local len = string.len(txt)\n i = 1\n while i <= len do\n if i == len then\n ngx.print(string.sub(txt, i, i))\n i = i + 1\n else\n ngx.print(string.sub(txt, i, i + 1))\n i = i + 2\n end\n ngx.flush(true)\n end\n ';\n replace_filter '^\\s+|\\s+$' '' g;\n }\n--- user_files\n>>> a.html\n hello, world \nblah yeah\nhello \n baby!\n \nabc\n\n--- stap2 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body chop\nhello, world\nblah yeah\nhello\nbaby!\nabc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 33: trim both leading and trailing spaces (3 bytes at a time)\n--- config\n replace_filter_max_buffered_size 2;\n default_type text/html;\n location /a.html {\n internal;\n }\n\n location = /t {\n content_by_lua '\n local res = ngx.location.capture(\"/a.html\")\n local txt = res.body\n local len = string.len(txt)\n i = 1\n while i <= len do\n if i == len then\n ngx.print(string.sub(txt, i, i))\n i = i + 1\n elseif i == len - 1 then\n ngx.print(string.sub(txt, i, i + 1))\n i = i + 2\n else\n ngx.print(string.sub(txt, i, i + 2))\n i = i + 3\n end\n ngx.flush(true)\n end\n ';\n replace_filter '^\\s+|\\s+$' '' g;\n }\n--- user_files\n>>> a.html\n hello, world \nblah yeah\nhello \n baby!\n \nabc\n\n--- stap2 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body chop\nhello, world\nblah yeah\nhello\nbaby!\nabc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 34: github issue #2: error \"general look-ahead not supported\"\n--- config\n replace_filter_max_buffered_size 3;\n location /t {\n charset utf-8;\n default_type text/html;\n echo \"ABCabcABCabc\";\n #replace_filter_types text/plain;\n replace_filter \"a.+a\" \"X\" \"ig\";\n }\n--- request\nGET /t\n\n--- stap2\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1492\") {\n print_ubacktrace()\n}\n\n--- response_body\nXbc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 35: backtrack to the middle of a pending capture (pending: output|capture + rematch)\n--- config\n replace_filter_max_buffered_size 2;\n default_type text/html;\n location = /t {\n echo -n ab;\n echo -n c;\n echo d;\n replace_filter 'abce|b' 'X' g;\n }\n\n--- stap2\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1501\") {\n print_ubacktrace()\n}\n\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\naXcd\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 36: backtrack to the middle of a pending capture (pending: output + capture|rematch\n--- config\n replace_filter_max_buffered_size 2;\n default_type text/html;\n location = /t {\n echo -n a;\n echo -n bc;\n echo d;\n replace_filter 'abce|b' 'X' g;\n }\n\n--- stap2\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1501\") {\n print_ubacktrace()\n}\n\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\naXcd\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 37: backtrack to the middle of a pending capture (pending: output + capture + rematch\n--- config\n replace_filter_max_buffered_size 2;\n default_type text/html;\n location = /t {\n echo -n a;\n echo -n b;\n echo -n c;\n echo d;\n replace_filter 'abce|b' 'X' g;\n }\n\n--- stap2\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1522\") {\n print_ubacktrace()\n}\n\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\naXcd\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 38: backtrack to the middle of a pending capture (pending: output|capture|rematch\n--- config\n replace_filter_max_buffered_size 2;\n default_type text/html;\n location = /t {\n echo -n abc;\n echo d;\n replace_filter 'abce|b' 'X' g;\n }\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\naXcd\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 39: backtrack to the middle of a pending capture (pending: output|capture|rematch(2)\n--- config\n replace_filter_max_buffered_size 3;\n default_type text/html;\n location = /t {\n echo -n abcc;\n echo d;\n replace_filter 'abcce|b' 'X' g;\n }\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\naXccd\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 40: backtrack to the middle of a pending capture (pending: output|capture(2)|rematch\n--- config\n replace_filter_max_buffered_size 2;\n default_type text/html;\n location = /t {\n echo -n abbc;\n echo d;\n replace_filter 'abbce|bb' 'X' g;\n }\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\naXcd\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 41: backtrack to the middle of a pending capture (pending: output(2)|capture|rematch\n--- config\n replace_filter_max_buffered_size 3;\n default_type text/html;\n location = /t {\n echo -n aabc;\n echo d;\n replace_filter 'aabce|b' 'X' g;\n }\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\naaXcd\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 42: backtrack to the beginning of a pending capture (pending: output + capture|rematch(2)\n--- config\n replace_filter_max_buffered_size 3;\n default_type text/html;\n location = /t {\n echo -n a;\n echo -n bcc;\n echo d;\n replace_filter 'abcce|b' 'X' g;\n }\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\naXccd\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 43: backtrack to the beginning of a pending capture (pending: output + capture(2)|rematch\n--- config\n replace_filter_max_buffered_size 2;\n default_type text/html;\n location = /t {\n echo -n a;\n echo -n bbc;\n echo d;\n replace_filter 'abbce|bb' 'X' g;\n }\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\naXcd\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 44: backtrack to the middle of a pending capture (pending: output(2) + capture|rematch\n--- config\n replace_filter_max_buffered_size 3;\n default_type text/html;\n location = /t {\n echo -n aa;\n echo -n bc;\n echo d;\n replace_filter 'aabce|b' 'X' g;\n }\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\naaXcd\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 45: assertions across AGAIN\n--- config\n replace_filter_max_buffered_size 2;\n default_type text/html;\n location = /t {\n echo -n a;\n echo -n \"\\n\";\n echo b;\n replace_filter 'a\\n^b' 'X' g;\n }\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\nX\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 46: assertions when capture backtracking happens\n--- config\n replace_filter_max_buffered_size 3;\n default_type text/html;\n location = /t {\n echo -n a;\n echo -n b;\n echo -n c;\n echo -n d;\n echo f;\n #echo abcdf;\n replace_filter 'abcde|b|\\bc' 'X' g;\n }\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\naXcdf\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 47: assertions when capture backtracking happens (2 pending matches)\n--- config\n replace_filter_max_buffered_size 3;\n default_type text/html;\n location = /t {\n echo -n a;\n echo -n b;\n echo -n ' ';\n echo -n d;\n echo f;\n #echo ab df;\n replace_filter 'ab de|b|b |\\b ' 'X' g;\n }\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\naXXdf\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 48: github issue #2: error \"general look-ahead not supported\", no \"g\"\n--- config\n replace_filter_max_buffered_size 3;\n location /t {\n charset utf-8;\n default_type text/html;\n echo \"ABCabcABCabc\";\n #replace_filter_types text/plain;\n replace_filter \"a.+a\" \"X\" \"i\";\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- response_body\nXbc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 49: nested rematch bufs\n--- config\n replace_filter_max_buffered_size 4;\n location /t {\n default_type text/html;\n echo -n a;\n echo -n b;\n echo -n c;\n echo -n d;\n echo -n e;\n echo g;\n #echo abcdeg;\n replace_filter 'abcdef|b|cdf|c' X g;\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- response_body\naXXdeg\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 50: nested rematch bufs (splitting pending buf)\n--- config\n replace_filter_max_buffered_size 6;\n location /t {\n default_type text/html;\n echo -n a;\n echo -n b;\n echo -n cd;\n echo -n e;\n echo -n f;\n echo -n g;\n echo i;\n #echo abcdefh;\n replace_filter 'abcdefgh|b|cdeg|d' X g;\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- response_body\naXcXefgi\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 51: remove C/C++ comments (1 byte at a time)\n--- config\n replace_filter_max_buffered_size 42;\n default_type text/html;\n location /a.html {\n internal;\n }\n\n location = /t {\n content_by_lua '\n local res = ngx.location.capture(\"/a.html\")\n local txt = res.body\n for i = 1, string.len(txt) do\n ngx.print(string.sub(txt, i, i))\n ngx.flush(true)\n end\n ';\n replace_filter '/\\*.*?\\*/|//[^\\n]*' '' g;\n }\n--- user_files\n>>> a.html\n i don't know // hello // world /* */\nhello world /** abc * b/c /*\n hello ** // world\n *\n */\nblah /* hi */ */ b\n//\n///hi\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- request\nGET /t\n--- response_body eval\n\" i don't know \nhello world \nblah */ b\n\n\n\"\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 52: remove C/C++ comments (all at a time)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n\n location /a.html {\n replace_filter '/\\*.*?\\*/|//[^\\n]*' '' g;\n }\n\n--- user_files\n>>> a.html\n i don't know // hello // world /* */\nhello world /** abc * b/c /*\n hello ** // world\n *\n */\nblah /* hi */ */ b\n//\n///hi\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- request\nGET /a.html\n--- response_body eval\n\" i don't know \nhello world \nblah */ b\n\n\n\"\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 53: remove C/C++ comments (all at a time) - server-level config\n--- config\n replace_filter_max_buffered_size 0;\n default_type text/html;\n\n replace_filter '/\\*.*?\\*/|//[^\\n]*' '' g;\n\n--- user_files\n>>> a.html\n i don't know // hello // world /* */\nhello world /** abc * b/c /*\n hello ** // world\n *\n */\nblah /* hi */ */ b\n//\n///hi\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- request\nGET /a.html\n--- response_body eval\n\" i don't know \nhello world \nblah */ b\n\n\n\"\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 54: multiple replace_filter_types settings (server level)\n--- config\n replace_filter_max_buffered_size 0;\n default_type text/plain;\n replace_filter_types text/css text/plain;\n location /t {\n echo abc;\n replace_filter b X;\n }\n--- request\nGET /t\n--- response_body\naXc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 55: multiple replace_filter_types settings (server level, but overridding in location)\n--- config\n replace_filter_max_buffered_size 0;\n default_type text/plain;\n replace_filter_types text/css text/plain;\n location /t {\n echo abc;\n replace_filter_types text/javascript;\n replace_filter b X;\n }\n--- request\nGET /t\n--- response_body\nabc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 56: do not use replace_filter at all\n--- config\n replace_filter_max_buffered_size 0;\n default_type text/plain;\n replace_filter_types text/css text/plain;\n location /t {\n echo abc;\n replace_filter_types text/css;\n }\n--- request\nGET /t\n--- response_body\nabc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 57: bad regex\n--- config\n default_type text/html;\n location /t {\n echo abc;\n replace_filter '(a+b' '';\n }\n--- request\nGET /t\n--- response_body\nabc\n--- no_error_log\n[alert]\n[error]\n--- SKIP\n\n\n\n=== TEST 58: github issue #3: data lost in particular situation\n--- config\n replace_filter_max_buffered_size 4;\n default_type text/html;\n location /t {\n default_type text/html;\n echo \"ABCabcABC\";\n echo \"ABCabcABC\";\n #echo \"ABCabcABC\\nABCabcABC\";\n replace_filter \"(a.+?c){2}\" \"X\" \"ig\";\n }\n--- request\nGET /t\n--- response_body\nXXABC\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 59: variation\n--- config\n replace_filter_max_buffered_size 5;\n default_type text/html;\n location /t {\n default_type text/html;\n #echo \"ABCabcABC\";\n #echo \"ABCabcABC\";\n echo \"ACacAC ACacAC\";\n replace_filter \"(a.+?c){2}\" \"X\" \"ig\";\n }\n--- request\nGET /t\n--- response_body\nXacAC\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 60: nested pending matched\n--- config\n replace_filter_max_buffered_size 4;\n default_type text/html;\n location /t {\n default_type text/html;\n echo -n a;\n echo -n b;\n echo -n c;\n echo -n def;\n echo -n gh;\n echo -n i;\n echo k;\n #echo abcdefig;\n replace_filter \"abcdefghij|bcdefg|cd\" \"X\" \"ig\";\n }\n--- request\nGET /t\n--- response_body\naXhik\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 61: test split chain with b_sane=1, next=NULL\n--- config\n replace_filter_max_buffered_size 4;\n default_type text/html;\n\n location = /t {\n echo -n aba;\n echo -n ba;\n echo -n bac;\n echo d;\n #echo abababacd;\n replace_filter abacd X;\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- stap3\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1217\") {\n print_ubacktrace()\n}\n--- response_body\nababX\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 62: test split chain with b_sane=1, next not NULL\n--- config\n replace_filter_max_buffered_size 6;\n default_type text/html;\n\n location = /t {\n echo -n aba;\n echo -n ba;\n echo -n ba;\n echo -n bac;\n echo d;\n #echo abababacd;\n replace_filter ababacd X;\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- stap3\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1217\") {\n print_ubacktrace()\n}\n--- response_body\nababX\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 63: trim leading spaces (1 byte at a time)\n--- config\n replace_filter_max_buffered_size 0;\n default_type text/html;\n location /a.html {\n }\n\n location = /t {\n content_by_lua '\n local res = ngx.location.capture(\"/a.html\")\n local txt = res.body\n for i = 1, string.len(txt) do\n ngx.print(string.sub(txt, i, i))\n ngx.flush(true)\n end\n ';\n replace_filter '^\\s+' '' g;\n }\n\n--- user_files\n>>> a.html\n hello, world \nblah yeah\nhello \n baby!\n \nabc\n--- request\nGET /t\n--- response_body\nhello, world \nblah yeah\nhello \nbaby!\nabc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 64: split ctx->pending into ctx->pending and ctx->free\n--- config\n replace_filter_max_buffered_size 3;\n default_type text/html;\n\n location = /t {\n #echo \"abc\\nd\";\n echo -n a;\n echo -n b;\n echo -n c;\n echo -n \"\\n\";\n echo d;\n replace_filter \"abcd|bc\\ne|c$\" X;\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- stap3\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1482\") {\n print_ubacktrace()\n}\n--- response_body\nabX\nd\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 65: trim both leading and trailing spaces (1 byte at a time)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 2;\n location /t {\n echo -n 'a';\n echo_sleep 0.001;\n echo ' ';\n echo_sleep 0.001;\n echo '';\n echo_sleep 0.001;\n echo ' ';\n echo_sleep 0.001;\n echo \"b\";\n echo_sleep 0.001;\n echo \" \";\n replace_filter '^\\s+|\\s+$' '' g;\n }\n\n location = /main {\n echo_location_async /t1;\n echo_location_async /t2;\n echo_location_async /t3;\n echo_location_async /t4;\n echo_location_async /t5;\n echo_location_async /t6;\n }\n\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /main\n--- response_body\na\nb\na\nb\na\nb\na\nb\na\nb\na\nb\n\n--- no_error_log\n[alert]\n[error]\n\n" }, { "path": "t/02-max-buffered.t", "content": "# vim:set ft= ts=4 sw=4 et fdm=marker:\n\nuse lib 'lib';\nuse Test::Nginx::Socket;\n\n#worker_connections(1014);\n#master_on();\n#workers(2);\n#log_level('warn');\n\nrepeat_each(2);\n\n#no_shuffle();\n\nplan tests => repeat_each() * (blocks() * 4);\n\nour $StapOutputChains = <<'_EOC_';\nglobal active\n\nF(ngx_http_handler) {\n active = 1\n}\n\n/*\nF(ngx_http_write_filter) {\n if (active && pid() == target()) {\n printf(\"http writer filter: %s\\n\", ngx_chain_dump($in))\n }\n}\n*/\n\nF(ngx_http_chunked_body_filter) {\n if (active && pid() == target()) {\n printf(\"http chunked filter: %s\\n\", ngx_chain_dump($in))\n }\n}\n\nF(ngx_http_replace_output) {\n if (active && pid() == target()) {\n printf(\"http replace output: %s\\n\", ngx_chain_dump($ctx->out))\n }\n}\n\nprobe syscall.writev {\n if (active && pid() == target()) {\n printf(\"writev(%s)\", ngx_iovec_dump($vec, $vlen))\n /*\n for (i = 0; i < $vlen; i++) {\n printf(\" %p [%s]\", $vec[i]->iov_base, text_str(user_string_n($vec[i]->iov_base, $vec[i]->iov_len)))\n }\n */\n }\n}\n\nprobe syscall.writev.return {\n if (active && pid() == target()) {\n printf(\" = %s\\n\", retstr)\n }\n}\n\n_EOC_\n\n#no_diff();\nno_long_string();\nrun_tests();\n\n__DATA__\n\n=== TEST 1: 1-byte chain bufs (0)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n\n location = /t {\n echo -n a;\n echo -n b;\n echo -n a;\n echo -n b;\n echo -n a;\n echo -n c;\n echo d;\n replace_filter abac X;\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- stap3\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1413\") {\n //printf(\"chain: %s\", ngx_chain_dump($ctx->busy))\n print_ubacktrace()\n}\n\n--- response_body\nababacd\n--- error_log\nreplace filter: exceeding replace_filter_max_buffered_size (0): 1\n--- no_error_log\n[error]\n\n\n\n=== TEST 2: 1-byte chain bufs (1)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 1;\n\n location = /t {\n echo -n a;\n echo -n b;\n echo -n a;\n echo -n b;\n echo -n a;\n echo -n c;\n echo d;\n replace_filter abac X;\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- stap3\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1439\") {\n //printf(\"chain: %s\", ngx_chain_dump($ctx->busy))\n print_ubacktrace()\n exit()\n}\n\n--- response_body\nababacd\n--- error_log\nreplace filter: exceeding replace_filter_max_buffered_size (1): 2\n--- no_error_log\n[error]\n\n\n\n=== TEST 3: trim both leading and trailing spaces (1 byte at a time) (2)\n--- config\n replace_filter_max_buffered_size 2;\n default_type text/html;\n location /a.html {\n internal;\n }\n\n location = /t {\n content_by_lua '\n local res = ngx.location.capture(\"/a.html\")\n local txt = res.body\n for i = 1, string.len(txt) do\n ngx.print(string.sub(txt, i, i))\n ngx.flush(true)\n end\n ';\n replace_filter '^\\s+|\\s+$' '' g;\n }\n--- user_files\n>>> a.html\n hello, world \nblah yeah\nhello \n baby!\n \nabc\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1438\") {\n //printf(\"chain: %s\", ngx_chain_dump($ctx->busy))\n print_ubacktrace()\n exit()\n}\n\n--- request\nGET /t\n--- response_body\nhello, world\nblah yeah\nhello\n baby!\n \nabc\n\n--- error_log\nreplace filter: exceeding replace_filter_max_buffered_size (2): 3\n--- no_error_log\n[error]\n\n\n\n=== TEST 4: github issue #2: error \"general look-ahead not supported\"\n--- config\n replace_filter_max_buffered_size 0;\n location /t {\n charset utf-8;\n default_type text/html;\n echo \"ABCabcABCabc\";\n #replace_filter_types text/plain;\n replace_filter \"a.+a\" \"X\" \"ig\";\n }\n--- request\nGET /t\n\n--- stap3\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1481\") {\n print_ubacktrace()\n}\n\n--- response_body\nABCabcABCabc\n--- error_log\nreplace filter: exceeding replace_filter_max_buffered_size (0): 2\n--- no_error_log\n[error]\n\n\n\n=== TEST 5: backtrack to the middle of a pending capture (pending: output|capture + rematch) (0)\n--- config\n replace_filter_max_buffered_size 0;\n default_type text/html;\n location = /t {\n echo -n ab;\n echo -n c;\n echo d;\n replace_filter 'abce|b' 'X' g;\n }\n\n--- stap2\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1492\") {\n print_ubacktrace()\n}\n\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\nabcd\n\n--- error_log\nreplace filter: exceeding replace_filter_max_buffered_size (0): 1\n--- no_error_log\n[error]\n\n\n\n=== TEST 6: backtrack to the middle of a pending capture (pending: output|capture + rematch) (1)\n--- config\n replace_filter_max_buffered_size 1;\n default_type text/html;\n location = /t {\n echo -n ab;\n echo -n c;\n echo d;\n replace_filter 'abce|b' 'X' g;\n }\n\n--- stap2\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1501\") {\n print_ubacktrace()\n}\n\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\naXcd\n\n--- error_log\nreplace filter: exceeding replace_filter_max_buffered_size (1): 2\n--- no_error_log\n[error]\n\n" }, { "path": "t/03-var.t", "content": "# vim:set ft= ts=4 sw=4 et fdm=marker:\n\nuse lib 'lib';\nuse Test::Nginx::Socket;\n\n#worker_connections(1014);\n#master_on();\n#workers(2);\n#log_level('warn');\n\nrepeat_each(2);\n\n#no_shuffle();\n\nplan tests => repeat_each() * (blocks() * 4 + 3);\n\nour $StapOutputChains = <<'_EOC_';\nglobal active\n\nF(ngx_http_handler) {\n active = 1\n}\n\n/*\nF(ngx_http_write_filter) {\n if (active && pid() == target()) {\n printf(\"http writer filter: %s\\n\", ngx_chain_dump($in))\n }\n}\n*/\n\nF(ngx_http_chunked_body_filter) {\n if (active && pid() == target()) {\n printf(\"http chunked filter: %s\\n\", ngx_chain_dump($in))\n }\n}\n\nF(ngx_http_replace_output) {\n if (active && pid() == target()) {\n printf(\"http replace output: %s\\n\", ngx_chain_dump($ctx->out))\n }\n}\n\nprobe syscall.writev {\n if (active && pid() == target()) {\n printf(\"writev(%s)\", ngx_iovec_dump($vec, $vlen))\n /*\n for (i = 0; i < $vlen; i++) {\n printf(\" %p [%s]\", $vec[i]->iov_base, text_str(user_string_n($vec[i]->iov_base, $vec[i]->iov_len)))\n }\n */\n }\n}\n\nprobe syscall.writev.return {\n if (active && pid() == target()) {\n printf(\" = %s\\n\", retstr)\n }\n}\n\n_EOC_\n\n#no_diff();\n#no_long_string();\nrun_tests();\n\n__DATA__\n\n=== TEST 1: nginx vars (global)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n set $foo X;\n echo abc;\n replace_filter . $foo g;\n }\n--- request\nGET /t\n\n--- stap\nF(ngx_http_replace_non_capturing_parse) {\n println(\"non capturing parse\")\n}\n\nF(ngx_http_replace_capturing_parse) {\n println(\"capturing parse\")\n}\n\n--- stap_out_like chop\n^(non capturing parse\\n)+$\n\n--- response_body chop\nXXXX\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 2: nginx vars (non-global)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n set $foo X;\n echo abc;\n replace_filter . $foo;\n }\n--- request\nGET /t\n\n--- stap\nF(ngx_http_replace_non_capturing_parse) {\n println(\"non capturing parse\")\n}\n\nF(ngx_http_replace_capturing_parse) {\n println(\"capturing parse\")\n}\n\n--- stap_out_like chop\n^(non capturing parse\\n)+$\n\n--- response_body\nXbc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 3: undefined nginx vars\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo abc;\n replace_filter . $foo;\n }\n--- request\nGET /t\n--- response_body\nXbc\n--- no_error_log\n[alert]\n[error]\n--- SKIP\n\n\n\n=== TEST 4: use of capturing variables\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo abc;\n replace_filter . $1;\n }\n--- request\nGET /t\n--- response_body\nXbc\n--- no_error_log\n[alert]\n[error]\n--- SKIP\n\n\n\n=== TEST 5: more contexts\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n set $foo X;\n echo abc;\n replace_filter . \"[$foo]\";\n }\n--- request\nGET /t\n--- response_body\n[X]bc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 6: more nginx vars\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n set $foo X;\n set $bar Y;\n echo abc;\n replace_filter . \"[$foo,$bar]\";\n }\n--- request\nGET /t\n--- response_body\n[X,Y]bc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 7: various lengths of nginx var values\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n set $foo XYZ;\n set $bar \"\";\n echo abc;\n replace_filter . \"[$foo,$bar]\";\n }\n--- request\nGET /t\n--- response_body\n[XYZ,]bc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 8: escaping the dollar sign\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n set $foo X;\n set $bar Y;\n echo abc;\n replace_filter . \"[$foo,$$bar]\";\n }\n--- request\nGET /t\n--- response_body\n[X,$bar]bc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 9: \\ is not an escaping sequence\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n set $foo X;\n set $bar Y;\n echo abc;\n replace_filter . \"[\\$foo,\\$bar]\";\n }\n--- request\nGET /t\n--- response_body\n[\\X,\\Y]bc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 10: cached subs values\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n set $foo X;\n echo abc;\n replace_filter . \"$foo\" g;\n }\n--- request\nGET /t\n--- response_body chop\nXXXX\n\n--- stap\nF(ngx_http_replace_complex_value) {\n println(\"complex value\")\n}\n\n--- stap_out\ncomplex value\n\n--- no_error_log\n[alert]\n[error]\n\n" }, { "path": "t/04-capturing.t", "content": "# vim:set ft= ts=4 sw=4 et fdm=marker:\n\nuse lib 'lib';\nuse Test::Nginx::Socket;\n\n#worker_connections(1014);\n#master_on();\n#workers(2);\n#log_level('warn');\n\nrepeat_each(2);\n\n#no_shuffle();\n\nplan tests => repeat_each() * (blocks() * 4 + 1);\n\nour $StapOutputChains = <<'_EOC_';\nglobal active\n\nF(ngx_http_handler) {\n active = 1\n}\n\n/*\nF(ngx_http_write_filter) {\n if (active && pid() == target()) {\n printf(\"http writer filter: %s\\n\", ngx_chain_dump($in))\n }\n}\n*/\n\nF(ngx_http_chunked_body_filter) {\n if (active && pid() == target()) {\n printf(\"http chunked filter: %s\\n\", ngx_chain_dump($in))\n }\n}\n\nF(ngx_http_replace_output) {\n if (active && pid() == target()) {\n printf(\"http replace output: %s\\n\", ngx_chain_dump($ctx->out))\n }\n}\n\nprobe syscall.writev {\n if (active && pid() == target()) {\n printf(\"writev(%s)\", ngx_iovec_dump($vec, $vlen))\n /*\n for (i = 0; i < $vlen; i++) {\n printf(\" %p [%s]\", $vec[i]->iov_base, text_str(user_string_n($vec[i]->iov_base, $vec[i]->iov_len)))\n }\n */\n }\n}\n\nprobe syscall.writev.return {\n if (active && pid() == target()) {\n printf(\" = %s\\n\", retstr)\n }\n}\n\n_EOC_\n\n#no_diff();\n#no_long_string();\nrun_tests();\n\n__DATA__\n\n=== TEST 1: ambiguous pattern\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo abcabcabde;\n replace_filter abcabd \"[$&]\";\n }\n--- request\nGET /t\n--- response_body\nabc[abcabd]e\n\n--- stap2 eval: $::StapOutputChains\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 2: ambiguous pattern\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo -n ababac;\n replace_filter abac \"[$&]\";\n }\n--- request\nGET /t\n--- response_body chop\nab[abac]\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 3: alt\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo abc;\n replace_filter 'ab|abc' [$&];\n }\n--- request\nGET /t\n--- response_body\n[ab]c\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 4: caseless\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo abcabcaBde;\n replace_filter abCabd [$&] i;\n }\n--- request\nGET /t\n--- response_body\nabc[abcaBd]e\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 5: case sensitive (no match)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo abcabcaBde;\n replace_filter abCabd [$&];\n }\n--- request\nGET /t\n--- response_body\nabcabcaBde\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 6: 1-byte chain bufs\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 3;\n\n location = /t {\n echo -n a;\n echo -n b;\n echo -n a;\n echo -n b;\n echo -n a;\n echo -n c;\n echo d;\n replace_filter abac [$&];\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- stap3\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1413\") {\n //printf(\"chain: %s\", ngx_chain_dump($ctx->busy))\n print_ubacktrace()\n}\n\n--- response_body\nab[abac]d\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 7: 2-byte chain bufs\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 2;\n\n location = /t {\n echo -n ab;\n echo -n ab;\n echo -n ac;\n echo d;\n replace_filter abac [$&];\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- response_body\nab[abac]d\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 8: 3-byte chain bufs\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 3;\n\n location = /t {\n echo -n aba;\n echo -n bac;\n echo d;\n replace_filter abac [$&];\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- response_body\nab[abac]d\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 9: 3-byte chain bufs (more)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 4;\n\n location = /t {\n echo -n aba;\n echo -n bac;\n echo d;\n replace_filter abacd [$&];\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- response_body\nab[abacd]\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 10: once by default (1st char matched)\n--- config\n replace_filter_max_buffered_size 0;\n default_type text/html;\n location /t {\n echo abcabcabde;\n replace_filter a [$&];\n }\n--- request\nGET /t\n--- response_body\n[a]bcabcabde\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 11: once by default (2nd char matched)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo abcabcabde;\n replace_filter b [$&];\n }\n--- request\nGET /t\n--- response_body\na[b]cabcabde\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 12: global substitution\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo bbc;\n replace_filter b [$&] g;\n }\n--- request\nGET /t\n--- response_body\n[b][b]c\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 13: global substitution\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo abcabcabde;\n replace_filter b [$&] g;\n }\n--- request\nGET /t\n--- response_body\na[b]ca[b]ca[b]de\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 14: global substitution (empty captures)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo -n abcabcabde;\n replace_filter [0-9]* [$&] g;\n }\n--- request\nGET /t\n--- response_body chop\n[]a[]b[]c[]a[]b[]c[]a[]b[]d[]e[]\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 15: global substitution (empty captures, splitted)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo -n ab;\n echo -n cab;\n echo -n c;\n echo -n abde;\n replace_filter [0-9]* [$&] g;\n }\n--- request\nGET /t\n--- response_body chop\n[]a[]b[]c[]a[]b[]c[]a[]b[]d[]e[]\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 16: global substitution (\\d+)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo \"hello1234, 56 world\";\n replace_filter \\d+ [$&] g;\n }\n--- request\nGET /t\n--- response_body\nhello[1234], [56] world\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 17: replace_filter_types default to text/html\n--- config\n default_type text/plain;\n location /t {\n echo abc;\n replace_filter b [$&];\n }\n--- request\nGET /t\n--- response_body\nabc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 18: custom replace_filter_types\n--- config\n default_type text/plain;\n location /t {\n echo abc;\n replace_filter b [$&];\n replace_filter_types text/plain;\n }\n--- request\nGET /t\n--- response_body\na[b]c\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 19: multiple replace_filter_types settings\n--- config\n default_type text/plain;\n location /t {\n echo abc;\n replace_filter b [$&];\n replace_filter_types text/css text/plain;\n }\n--- request\nGET /t\n--- response_body\na[b]c\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 20: trim leading spaces\n--- config\n replace_filter_max_buffered_size 0;\n default_type text/html;\n location /a.html {\n replace_filter '^\\s+' '[$&]' g;\n }\n--- user_files\n>>> a.html\n hello, world \nblah yeah\nhello \n baby!\n \nabc\n--- request\nGET /a.html\n--- response_body\n[ ]hello, world \nblah yeah\nhello \n[ ]baby!\n[ \n]abc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 21: trim trailing spaces\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /a.html {\n replace_filter '\\s+$' '[$&]' g;\n }\n--- user_files\n>>> a.html\n hello, world \nblah yeah\nhello \n baby!\n \nabc\n--- request\nGET /a.html\n--- response_body chop\n hello, world[ ]\nblah yeah\nhello[ ]\n baby![\n ]\nabc[\n]\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 22: trim both leading and trailing spaces\n--- config\n replace_filter_max_buffered_size 0;\n default_type text/html;\n location /a.html {\n replace_filter '^\\s+|\\s+$' '[$&]' g;\n }\n--- user_files\n>>> a.html\n hello, world \nblah yeah\nhello \n baby!\n \nabc\n--- request\nGET /a.html\n--- response_body chop\n[ ]hello, world[ ]\nblah yeah\nhello[ ]\n[ ]baby!\n[ \n]abc[\n]\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 23: pure flush buf in the stream (no data)\n--- config\n replace_filter_max_buffered_size 0;\n default_type text/html;\n location = /t {\n echo_flush;\n replace_filter 'a' '[$&]' g;\n }\n--- request\nGET /t\n--- response_body chop\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 24: pure flush buf in the stream (with data)\n--- config\n replace_filter_max_buffered_size 0;\n default_type text/html;\n location = /t {\n echo a;\n echo_flush;\n replace_filter 'a' '[$&]' g;\n }\n--- request\nGET /t\n--- stap3 eval: $::StapOutputChains\n--- stap2\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:539\") {\n printf(\"chain: %s\", ngx_chain_dump($ctx->busy))\n //print_ubacktrace()\n}\n--- response_body\n[a]\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 25: trim both leading and trailing spaces (1 byte at a time)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 2;\n location = /t {\n echo -n 'a';\n echo ' ';\n echo \"b\";\n replace_filter '^\\s+|\\s+$' '[$&]' g;\n }\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body chop\na[ ]\nb[\n]\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 26: trim both leading and trailing spaces (1 byte at a time), no \\s for $\n--- config\n replace_filter_max_buffered_size 1;\n default_type text/html;\n location = /t {\n echo -n 'a';\n echo ' ';\n echo \"b\";\n replace_filter '^\\s+| +$' '[$&]' g;\n }\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\na[ ]\nb\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 27: trim both leading and trailing spaces (1 byte at a time)\n--- config\n replace_filter_max_buffered_size 7;\n default_type text/html;\n location /a.html {\n internal;\n }\n\n location = /t {\n content_by_lua '\n local res = ngx.location.capture(\"/a.html\")\n local txt = res.body\n for i = 1, string.len(txt) do\n ngx.print(string.sub(txt, i, i))\n ngx.flush(true)\n end\n ';\n replace_filter '^\\s+|\\s+$' '[$&]' g;\n }\n--- user_files\n>>> a.html\n hello, world \nblah yeah\nhello \n baby!\n \nabc\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1438\") {\n //printf(\"chain: %s\", ngx_chain_dump($ctx->busy))\n print_ubacktrace()\n exit()\n}\n\n--- request\nGET /t\n--- response_body chop\n[ ]hello, world[ ]\nblah yeah\nhello[ ]\n[ ]baby!\n[ \n]abc[\n]\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 28: \\b at the border\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo -n a;\n echo b;\n replace_filter '\\bb|a' [$&] g;\n }\n--- request\nGET /t\n--- response_body\n[a]b\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 29: \\B at the border\n--- config\n replace_filter_max_buffered_size 0;\n default_type text/html;\n location /t {\n echo -n a;\n echo ',';\n replace_filter '\\B,|a' [$&] g;\n }\n--- request\nGET /t\n--- response_body\n[a],\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 30: \\A at the border\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo -n a;\n echo 'b';\n replace_filter '\\Ab|a' [$&] g;\n }\n--- request\nGET /t\n--- response_body\n[a]b\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 31: memory bufs with last_buf=1\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n return 200 \"abc\";\n replace_filter \\w+ [$&];\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- response_body chop\n[abc]\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 32: trim both leading and trailing spaces (2 bytes at a time)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 7;\n location /a.html {\n internal;\n }\n\n location = /t {\n content_by_lua '\n local res = ngx.location.capture(\"/a.html\")\n local txt = res.body\n local len = string.len(txt)\n i = 1\n while i <= len do\n if i == len then\n ngx.print(string.sub(txt, i, i))\n i = i + 1\n else\n ngx.print(string.sub(txt, i, i + 1))\n i = i + 2\n end\n ngx.flush(true)\n end\n ';\n replace_filter '^\\s+|\\s+$' '[$&]' g;\n }\n--- user_files\n>>> a.html\n hello, world \nblah yeah\nhello \n baby!\n \nabc\n\n--- stap2 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body chop\n[ ]hello, world[ ]\nblah yeah\nhello[ ]\n[ ]baby!\n[ \n]abc[\n]\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 33: trim both leading and trailing spaces (3 bytes at a time)\n--- config\n replace_filter_max_buffered_size 5;\n default_type text/html;\n location /a.html {\n internal;\n }\n\n location = /t {\n content_by_lua '\n local res = ngx.location.capture(\"/a.html\")\n local txt = res.body\n local len = string.len(txt)\n i = 1\n while i <= len do\n if i == len then\n ngx.print(string.sub(txt, i, i))\n i = i + 1\n elseif i == len - 1 then\n ngx.print(string.sub(txt, i, i + 1))\n i = i + 2\n else\n ngx.print(string.sub(txt, i, i + 2))\n i = i + 3\n end\n ngx.flush(true)\n end\n ';\n replace_filter '^\\s+|\\s+$' '[$&]' g;\n }\n--- user_files\n>>> a.html\n hello, world \nblah yeah\nhello \n baby!\n \nabc\n\n--- stap2 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body chop\n[ ]hello, world[ ]\nblah yeah\nhello[ ]\n[ ]baby!\n[ \n]abc[\n]\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 34: github issue #2: error \"general look-ahead not supported\"\n--- config\n replace_filter_max_buffered_size 13;\n location /t {\n charset utf-8;\n default_type text/html;\n echo \"ABCabcABCabc\";\n #replace_filter_types text/plain;\n replace_filter \"a.+a\" \"[$&]\" \"ig\";\n }\n--- request\nGET /t\n\n--- stap2\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1492\") {\n print_ubacktrace()\n}\n\n--- response_body\n[ABCabcABCa]bc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 35: backtrack to the middle of a pending capture (pending: output|capture + rematch)\n--- config\n replace_filter_max_buffered_size 3;\n default_type text/html;\n location = /t {\n echo -n ab;\n echo -n c;\n echo d;\n replace_filter 'abce|b' '[$&]' g;\n }\n\n--- stap2\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1501\") {\n print_ubacktrace()\n}\n\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\na[b]cd\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 36: backtrack to the middle of a pending capture (pending: output + capture|rematch\n--- config\n replace_filter_max_buffered_size 3;\n default_type text/html;\n location = /t {\n echo -n a;\n echo -n bc;\n echo d;\n replace_filter 'abce|b' '[$&]' g;\n }\n\n--- stap2\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1501\") {\n print_ubacktrace()\n}\n\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\na[b]cd\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 37: backtrack to the middle of a pending capture (pending: output + capture + rematch\n--- config\n replace_filter_max_buffered_size 3;\n default_type text/html;\n location = /t {\n echo -n a;\n echo -n b;\n echo -n c;\n echo d;\n replace_filter 'abce|b' '[$&]' g;\n }\n\n--- stap2\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1522\") {\n print_ubacktrace()\n}\n\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\na[b]cd\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 38: backtrack to the middle of a pending capture (pending: output|capture|rematch\n--- config\n replace_filter_max_buffered_size 3;\n default_type text/html;\n location = /t {\n echo -n abc;\n echo d;\n replace_filter 'abce|b' '[$&]' g;\n }\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\na[b]cd\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 39: backtrack to the middle of a pending capture (pending: output|capture|rematch(2)\n--- config\n replace_filter_max_buffered_size 4;\n default_type text/html;\n location = /t {\n echo -n abcc;\n echo d;\n replace_filter 'abcce|b' '[$&]' g;\n }\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\na[b]ccd\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 40: backtrack to the middle of a pending capture (pending: output|capture(2)|rematch\n--- config\n replace_filter_max_buffered_size 4;\n default_type text/html;\n location = /t {\n echo -n abbc;\n echo d;\n replace_filter 'abbce|bb' '[$&]' g;\n }\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\na[bb]cd\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 41: backtrack to the middle of a pending capture (pending: output(2)|capture|rematch\n--- config\n replace_filter_max_buffered_size 4;\n default_type text/html;\n location = /t {\n echo -n aabc;\n echo d;\n replace_filter 'aabce|b' '[$&]' g;\n }\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\naa[b]cd\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 42: backtrack to the beginning of a pending capture (pending: output + capture|rematch(2)\n--- config\n replace_filter_max_buffered_size 4;\n default_type text/html;\n location = /t {\n echo -n a;\n echo -n bcc;\n echo d;\n replace_filter 'abcce|b' '[$&]' g;\n }\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\na[b]ccd\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 43: backtrack to the beginning of a pending capture (pending: output + capture(2)|rematch\n--- config\n replace_filter_max_buffered_size 4;\n default_type text/html;\n location = /t {\n echo -n a;\n echo -n bbc;\n echo d;\n replace_filter 'abbce|bb' '[$&]' g;\n }\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\na[bb]cd\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 44: backtrack to the middle of a pending capture (pending: output(2) + capture|rematch\n--- config\n replace_filter_max_buffered_size 4;\n default_type text/html;\n location = /t {\n echo -n aa;\n echo -n bc;\n echo d;\n replace_filter 'aabce|b' '[$&]' g;\n }\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\naa[b]cd\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 45: assertions across AGAIN\n--- config\n replace_filter_max_buffered_size 2;\n default_type text/html;\n location = /t {\n echo -n a;\n echo -n \"\\n\";\n echo b;\n replace_filter 'a\\n^b' '[$&]' g;\n }\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\n[a\nb]\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 46: assertions when capture backtracking happens\n--- config\n replace_filter_max_buffered_size 4;\n default_type text/html;\n location = /t {\n echo -n a;\n echo -n b;\n echo -n c;\n echo -n d;\n echo f;\n #echo abcdf;\n replace_filter 'abcde|b|\\bc' '[$&]' g;\n }\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\na[b]cdf\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 47: assertions when capture backtracking happens (2 pending matches)\n--- config\n replace_filter_max_buffered_size 4;\n default_type text/html;\n location = /t {\n echo -n a;\n echo -n b;\n echo -n ' ';\n echo -n d;\n echo f;\n #echo ab df;\n replace_filter 'ab de|b|b |\\b ' '[$&]' g;\n }\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\na[b][ ]df\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 48: github issue #2: error \"general look-ahead not supported\", no \"g\"\n--- config\n replace_filter_max_buffered_size 13;\n location /t {\n charset utf-8;\n default_type text/html;\n echo \"ABCabcABCabc\";\n #replace_filter_types text/plain;\n replace_filter \"a.+a\" \"[$&]\" \"i\";\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- response_body\n[ABCabcABCa]bc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 49: nested rematch bufs\n--- config\n replace_filter_max_buffered_size 5;\n location /t {\n default_type text/html;\n echo -n a;\n echo -n b;\n echo -n c;\n echo -n d;\n echo -n e;\n echo g;\n #echo abcdeg;\n replace_filter 'abcdef|b|cdf|c' [$&] g;\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- response_body\na[b][c]deg\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 50: nested rematch bufs (splitting pending buf)\n--- config\n replace_filter_max_buffered_size 7;\n location /t {\n default_type text/html;\n echo -n a;\n echo -n b;\n echo -n cd;\n echo -n e;\n echo -n f;\n echo -n g;\n echo i;\n #echo abcdefh;\n replace_filter 'abcdefgh|b|cdeg|d' [$&] g;\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- response_body\na[b]c[d]efgi\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 51: remove C/C++ comments (1 byte at a time)\n--- config\n replace_filter_max_buffered_size 50;\n default_type text/html;\n location /a.html {\n internal;\n }\n\n location = /t {\n content_by_lua '\n local res = ngx.location.capture(\"/a.html\")\n local txt = res.body\n for i = 1, string.len(txt) do\n ngx.print(string.sub(txt, i, i))\n ngx.flush(true)\n end\n ';\n replace_filter '/\\*.*?\\*/|//[^\\n]*' '[$&]' g;\n }\n--- user_files\n>>> a.html\n i don't know // hello // world /* */\nhello world /** abc * b/c /*\n hello ** // world\n *\n */\nblah /* hi */ */ b\n//\n///hi\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- request\nGET /t\n--- response_body eval\n\" i don't know [// hello // world /* */]\nhello world [/** abc * b/c /*\n hello ** // world\n *\n */]\nblah [/* hi */] */ b\n[//]\n[///hi]\n\"\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 52: remove C/C++ comments (all at a time)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n\n location /a.html {\n replace_filter '/\\*.*?\\*/|//[^\\n]*' '[$&]' g;\n }\n\n--- user_files\n>>> a.html\n i don't know // hello // world /* */\nhello world /** abc * b/c /*\n hello ** // world\n *\n */\nblah /* hi */ */ b\n//\n///hi\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- request\nGET /a.html\n--- response_body eval\n\" i don't know [// hello // world /* */]\nhello world [/** abc * b/c /*\n hello ** // world\n *\n */]\nblah [/* hi */] */ b\n[//]\n[///hi]\n\"\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 53: remove C/C++ comments (all at a time) - server-level config\n--- config\n replace_filter_max_buffered_size 0;\n default_type text/html;\n\n replace_filter '/\\*.*?\\*/|//[^\\n]*' '[$&]' g;\n\n--- user_files\n>>> a.html\n i don't know // hello // world /* */\nhello world /** abc * b/c /*\n hello ** // world\n *\n */\nblah /* hi */ */ b\n//\n///hi\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- request\nGET /a.html\n--- response_body eval\n\" i don't know [// hello // world /* */]\nhello world [/** abc * b/c /*\n hello ** // world\n *\n */]\nblah [/* hi */] */ b\n[//]\n[///hi]\n\"\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 54: multiple replace_filter_types settings (server level)\n--- config\n replace_filter_max_buffered_size 0;\n default_type text/plain;\n replace_filter_types text/css text/plain;\n location /t {\n echo abc;\n replace_filter b [$&];\n }\n--- request\nGET /t\n--- response_body\na[b]c\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 55: multiple replace_filter_types settings (server level, but overridding in location)\n--- config\n replace_filter_max_buffered_size 0;\n default_type text/plain;\n replace_filter_types text/css text/plain;\n location /t {\n echo abc;\n replace_filter_types text/javascript;\n replace_filter b [$&];\n }\n--- request\nGET /t\n--- response_body\nabc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 56: github issue #3: data lost in particular situation\n--- config\n replace_filter_max_buffered_size 4;\n default_type text/html;\n location /t {\n default_type text/html;\n echo \"ABCabcABC\";\n echo \"ABCabcABC\";\n #echo \"ABCabcABC\\nABCabcABC\";\n replace_filter \"(a.+?c){2}\" \"[$&]\" \"ig\";\n }\n--- request\nGET /t\n--- response_body\n[ABCabc][ABC\nABCabc]ABC\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 57: variation\n--- config\n replace_filter_max_buffered_size 5;\n default_type text/html;\n location /t {\n default_type text/html;\n #echo \"ABCabcABC\";\n #echo \"ABCabcABC\";\n echo \"ACacAC ACacAC\";\n replace_filter \"(a.+?c){2}\" \"[$&]\" \"ig\";\n }\n--- request\nGET /t\n--- response_body\n[ACacAC AC]acAC\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 58: nested pending matched\n--- config\n replace_filter_max_buffered_size 9;\n default_type text/html;\n location /t {\n default_type text/html;\n echo -n a;\n echo -n b;\n echo -n c;\n echo -n def;\n echo -n gh;\n echo -n i;\n echo k;\n #echo abcdefig;\n replace_filter \"abcdefghij|bcdefg|cd\" \"[$&]\" \"ig\";\n }\n--- request\nGET /t\n--- response_body\na[bcdefg]hik\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 59: test split chain with b_sane=1, next=NULL\n--- config\n replace_filter_max_buffered_size 4;\n default_type text/html;\n\n location = /t {\n echo -n aba;\n echo -n ba;\n echo -n bac;\n echo d;\n #echo abababacd;\n replace_filter abacd [$&];\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- stap3\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1217\") {\n print_ubacktrace()\n}\n--- response_body\nabab[abacd]\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 60: test split chain with b_sane=1, next not NULL\n--- config\n replace_filter_max_buffered_size 6;\n default_type text/html;\n\n location = /t {\n echo -n aba;\n echo -n ba;\n echo -n ba;\n echo -n bac;\n echo d;\n #echo abababacd;\n replace_filter ababacd [$&];\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- stap3\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1217\") {\n print_ubacktrace()\n}\n--- response_body\nabab[ababacd]\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 61: trim leading spaces (1 byte at a time)\n--- config\n replace_filter_max_buffered_size 6;\n default_type text/html;\n location /a.html {\n }\n\n location = /t {\n content_by_lua '\n local res = ngx.location.capture(\"/a.html\")\n local txt = res.body\n for i = 1, string.len(txt) do\n ngx.print(string.sub(txt, i, i))\n ngx.flush(true)\n end\n ';\n replace_filter '^\\s+' '[$&]' g;\n }\n\n--- user_files\n>>> a.html\n hello, world \nblah yeah\nhello \n baby!\n \nabc\n--- request\nGET /t\n--- response_body\n[ ]hello, world \nblah yeah\nhello \n[ ]baby!\n[ \n]abc\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 62: split ctx->pending into ctx->pending and ctx->free\n--- config\n replace_filter_max_buffered_size 3;\n default_type text/html;\n\n location = /t {\n #echo \"abc\\nd\";\n echo -n a;\n echo -n b;\n echo -n c;\n echo -n \"\\n\";\n echo d;\n replace_filter \"abcd|bc\\ne|c$\" [$&];\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- stap3\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1482\") {\n print_ubacktrace()\n}\n--- response_body\nab[c]\nd\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 63: trim both leading and trailing spaces (1 byte at a time)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 5;\n location /t {\n echo -n 'a';\n echo_sleep 0.001;\n echo ' ';\n echo_sleep 0.001;\n echo '';\n echo_sleep 0.001;\n echo ' ';\n echo_sleep 0.001;\n echo \"b\";\n echo_sleep 0.001;\n echo \" \";\n replace_filter '^\\s+|\\s+$' '[$&]' g;\n }\n\n location = /main {\n echo_location_async /t1;\n echo_location_async /t2;\n echo_location_async /t3;\n echo_location_async /t4;\n echo_location_async /t5;\n echo_location_async /t6;\n }\n\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /main\n--- response_body chop\na[ \n\n ]\nb\n[ \n]a[ \n\n ]\nb\n[ \n]a[ \n\n ]\nb\n[ \n]a[ \n\n ]\nb\n[ \n]a[ \n\n ]\nb\n[ \n]a[ \n\n ]\nb\n[ \n]\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 64: global substitution + nginx vars\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n set $foo X;\n echo bbcd;\n replace_filter ([bc])|(d) [$1-$2-$foo] g;\n }\n--- request\nGET /t\n--- response_body\n[b--X][b--X][c--X][-d-X]\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 65: global substitution + nginx vars (splitted bufs)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n set $foo X;\n echo -n b;\n echo -n b;\n echo -n c;\n echo d;\n replace_filter ([bc])|(d) [$1-$2-$foo] g;\n }\n--- request\nGET /t\n--- response_body\n[b--X][b--X][c--X][-d-X]\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 66: global substitution + nginx vars (out of captures)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n set $foo X;\n echo -n b;\n echo -n b;\n echo -n c;\n echo d;\n replace_filter [bc]|d [$1-$2-$foo] g;\n }\n--- request\nGET /t\n\n--- stap\nF(ngx_http_replace_complex_value) {\n println(\"complex value\")\n}\n\n--- stap_out\ncomplex value\ncomplex value\ncomplex value\ncomplex value\n\n--- response_body\n[--X][--X][--X][--X]\n\n--- no_error_log\n[alert]\n[error]\n\n" }, { "path": "t/05-capturing-max-buffered.t", "content": "# vim:set ft= ts=4 sw=4 et fdm=marker:\n\nuse lib 'lib';\nuse Test::Nginx::Socket;\n\n#worker_connections(1014);\n#master_on();\n#workers(2);\n#log_level('warn');\n\nrepeat_each(2);\n\n#no_shuffle();\n\nplan tests => repeat_each() * (blocks() * 4);\n\nour $StapOutputChains = <<'_EOC_';\nglobal active\n\nF(ngx_http_handler) {\n active = 1\n}\n\n/*\nF(ngx_http_write_filter) {\n if (active && pid() == target()) {\n printf(\"http writer filter: %s\\n\", ngx_chain_dump($in))\n }\n}\n*/\n\nF(ngx_http_chunked_body_filter) {\n if (active && pid() == target()) {\n printf(\"http chunked filter: %s\\n\", ngx_chain_dump($in))\n }\n}\n\nF(ngx_http_replace_output) {\n if (active && pid() == target()) {\n printf(\"http replace output: %s\\n\", ngx_chain_dump($ctx->out))\n }\n}\n\nprobe syscall.writev {\n if (active && pid() == target()) {\n printf(\"writev(%s)\", ngx_iovec_dump($vec, $vlen))\n /*\n for (i = 0; i < $vlen; i++) {\n printf(\" %p [%s]\", $vec[i]->iov_base, text_str(user_string_n($vec[i]->iov_base, $vec[i]->iov_len)))\n }\n */\n }\n}\n\nprobe syscall.writev.return {\n if (active && pid() == target()) {\n printf(\" = %s\\n\", retstr)\n }\n}\n\n_EOC_\n\n#no_diff();\nno_long_string();\nrun_tests();\n\n__DATA__\n\n=== TEST 1: 1-byte chain bufs (0)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n\n location = /t {\n echo -n a;\n echo -n b;\n echo -n a;\n echo -n b;\n echo -n a;\n echo -n c;\n echo d;\n replace_filter abac [$&];\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- stap3\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1413\") {\n //printf(\"chain: %s\", ngx_chain_dump($ctx->busy))\n print_ubacktrace()\n}\n\n--- response_body\nababacd\n--- error_log\nreplace filter: exceeding replace_filter_max_buffered_size (0): 1\n--- no_error_log\n[error]\n\n\n\n=== TEST 2: 1-byte chain bufs (1)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 1;\n\n location = /t {\n echo -n a;\n echo -n b;\n echo -n a;\n echo -n b;\n echo -n a;\n echo -n c;\n echo d;\n replace_filter abac [$&];\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- stap3\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1439\") {\n //printf(\"chain: %s\", ngx_chain_dump($ctx->busy))\n print_ubacktrace()\n exit()\n}\n\n--- response_body\nababacd\n--- error_log\nreplace filter: exceeding replace_filter_max_buffered_size (1): 2\n--- no_error_log\n[error]\n\n\n\n=== TEST 3: 1-byte chain bufs (2)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 2;\n\n location = /t {\n echo -n a;\n echo -n b;\n echo -n a;\n echo -n b;\n echo -n a;\n echo -n c;\n echo d;\n replace_filter abac [$&];\n }\n--- request\nGET /t\n--- stap2 eval: $::StapOutputChains\n--- stap3\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1439\") {\n //printf(\"chain: %s\", ngx_chain_dump($ctx->busy))\n print_ubacktrace()\n exit()\n}\n\n--- response_body\nababacd\n--- error_log\nreplace filter: exceeding replace_filter_max_buffered_size (2): 3\n--- no_error_log\n[error]\n\n\n\n=== TEST 4: trim both leading and trailing spaces (1 byte at a time) (6)\n--- config\n replace_filter_max_buffered_size 6;\n default_type text/html;\n location /a.html {\n internal;\n }\n\n location = /t {\n content_by_lua '\n local res = ngx.location.capture(\"/a.html\")\n local txt = res.body\n for i = 1, string.len(txt) do\n ngx.print(string.sub(txt, i, i))\n ngx.flush(true)\n end\n ';\n replace_filter '^\\s+|\\s+$' '[$&]' g;\n }\n--- user_files\n>>> a.html\n hello, world \nblah yeah\nhello \n baby!\n \nabc\n\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- stap3\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1438\") {\n //printf(\"chain: %s\", ngx_chain_dump($ctx->busy))\n print_ubacktrace()\n exit()\n}\n\n--- request\nGET /t\n--- response_body\n[ ]hello, world[ ]\nblah yeah\nhello[ ]\n[ ]baby!\n \nabc\n\n--- error_log\nreplace filter: exceeding replace_filter_max_buffered_size (6): 7\n--- no_error_log\n[error]\n\n\n\n=== TEST 5: github issue #2: error \"general look-ahead not supported\"\n--- config\n replace_filter_max_buffered_size 0;\n location /t {\n charset utf-8;\n default_type text/html;\n echo \"ABCabcABCabc\";\n #replace_filter_types text/plain;\n replace_filter \"a.+a\" \"[$&]\" \"ig\";\n }\n--- request\nGET /t\n\n--- stap3\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1481\") {\n print_ubacktrace()\n}\n\n--- response_body\nABCabcABCabc\n--- error_log\nreplace filter: exceeding replace_filter_max_buffered_size (0): 12\n--- no_error_log\n[error]\n\n\n\n=== TEST 6: backtrack to the middle of a pending capture (pending: output|capture + rematch) (0)\n--- config\n replace_filter_max_buffered_size 0;\n default_type text/html;\n location = /t {\n echo -n ab;\n echo -n c;\n echo d;\n replace_filter 'abce|b' '[$&]' g;\n }\n\n--- stap2\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1492\") {\n print_ubacktrace()\n}\n\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\nabcd\n\n--- error_log\nreplace filter: exceeding replace_filter_max_buffered_size (0): 2\n--- no_error_log\n[error]\n\n\n\n=== TEST 7: backtrack to the middle of a pending capture (pending: output|capture + rematch) (1)\n--- config\n replace_filter_max_buffered_size 1;\n default_type text/html;\n location = /t {\n echo -n ab;\n echo -n c;\n echo d;\n replace_filter 'abce|b' '[$&]' g;\n }\n\n--- stap2\nprobe process(\"nginx\").statement(\"*@ngx_http_replace_filter_module.c:1501\") {\n print_ubacktrace()\n}\n\n--- stap3 eval: $::StapOutputChains\n--- request\nGET /t\n--- response_body\nabcd\n\n--- error_log\nreplace filter: exceeding replace_filter_max_buffered_size (1): 2\n--- no_error_log\n[error]\n\n" }, { "path": "t/06-if.t", "content": "# vim:set ft= ts=4 sw=4 et fdm=marker:\n\nuse lib 'lib';\nuse Test::Nginx::Socket;\n\n#worker_connections(1014);\n#master_on();\n#workers(2);\n#log_level('warn');\n\nrepeat_each(2);\n\n#no_shuffle();\n\nplan tests => repeat_each() * (blocks() * 4);\n\n#no_diff();\n#no_long_string();\nrun_tests();\n\n__DATA__\n\n=== TEST 1: local if hit\n--- config\n location /t {\n default_type text/plain;\n echo abcabcabde;\n\n if ($arg_disable = \"\") {\n replace_filter_types text/plain;\n replace_filter_max_buffered_size 0;\n replace_filter abcabd X;\n }\n }\n--- request\nGET /t\n--- response_body\nabcXe\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 2: local if miss\n--- config\n replace_filter_max_buffered_size 0;\n location /t {\n default_type text/plain;\n echo abcabcabde;\n\n if ($arg_disable = \"\") {\n replace_filter_types text/plain;\n replace_filter_max_buffered_size 0;\n replace_filter abcabd X;\n }\n }\n--- request\nGET /t?disable=1\n--- response_body\nabcabcabde\n--- no_error_log\n[alert]\n[error]\n\n" }, { "path": "t/07-multi.t", "content": "# vim:set ft= ts=4 sw=4 et fdm=marker:\n\nuse lib 'lib';\nuse Test::Nginx::Socket;\n\n#worker_connections(1014);\n#master_on();\n#workers(2);\n#log_level('warn');\n\nrepeat_each(2);\n\nno_shuffle();\n\nplan tests => repeat_each() * (blocks() * 4 + 5);\n\nour $StapOutputChains = <<'_EOC_';\nglobal active\n\nF(ngx_http_handler) {\n active = 1\n}\n\n/*\nF(ngx_http_write_filter) {\n if (active && pid() == target()) {\n printf(\"http writer filter: %s\\n\", ngx_chain_dump($in))\n }\n}\n*/\n\nF(ngx_http_chunked_body_filter) {\n if (active && pid() == target()) {\n printf(\"http chunked filter: %s\\n\", ngx_chain_dump($in))\n }\n}\n\nF(ngx_http_replace_output) {\n if (active && pid() == target()) {\n printf(\"http replace output: %s\\n\", ngx_chain_dump($ctx->out))\n }\n}\n\nprobe syscall.writev {\n if (active && pid() == target()) {\n printf(\"writev(%s)\", ngx_iovec_dump($vec, $vlen))\n /*\n for (i = 0; i < $vlen; i++) {\n printf(\" %p [%s]\", $vec[i]->iov_base, text_str(user_string_n($vec[i]->iov_base, $vec[i]->iov_len)))\n }\n */\n }\n}\n\nprobe syscall.writev.return {\n if (active && pid() == target()) {\n printf(\" = %s\\n\", retstr)\n }\n}\n\n_EOC_\n\n#no_diff();\n#no_long_string();\nrun_tests();\n\n__DATA__\n\n=== TEST 1: once patterns\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo 'hello world world hello';\n replace_filter world \"<$&>\";\n replace_filter hello \"[$&]\";\n }\n--- request\nGET /t\n--- response_body\n[hello] world hello\n\n--- stap\nF(ngx_http_replace_non_capturing_parse) {\n println(\"non capturing parse\")\n}\n\nF(ngx_http_replace_capturing_parse) {\n println(\"capturing parse\")\n}\n\n--- stap_out_like chop\n^(capturing parse\\n)+$\n\n--- stap2 eval: $::StapOutputChains\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 2: once patterns\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo 'Hello world Hello world';\n replace_filter world \"<$&>\";\n replace_filter hello \"[$&]\";\n }\n--- request\nGET /t\n--- response_body\nHello Hello world\n\n--- stap2 eval: $::StapOutputChains\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 3: case-insensitive patterns\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo 'Hello world WORLD HELLO';\n replace_filter world \"<$&>\";\n replace_filter hello \"[$&]\" i;\n }\n--- request\nGET /t\n--- response_body\n[Hello] WORLD HELLO\n\n--- stap2 eval: $::StapOutputChains\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 4: global subs\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo 'hello world world hello';\n replace_filter world \"<$&>\" g;\n replace_filter hello \"[$&]\" g;\n }\n--- request\nGET /t\n--- response_body\n[hello] [hello]\n\n--- stap2 eval: $::StapOutputChains\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 5: global subs (case sensitive)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo 'Hello World worlD hellO';\n replace_filter world \"<$&>\" g;\n replace_filter hello \"[$&]\" g;\n }\n--- request\nGET /t\n--- response_body\nHello World worlD hellO\n\n--- stap2 eval: $::StapOutputChains\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 6: global subs (case insensitive)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo 'Hello World worlD hellO';\n replace_filter world \"<$&>\" ig;\n replace_filter hello \"[$&]\" g;\n }\n--- request\nGET /t\n--- response_body\nHello hellO\n\n--- stap2 eval: $::StapOutputChains\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 7: global subs (case insensitive) (2)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo 'Hello World worlD hellO';\n replace_filter world \"<$&>\" g;\n replace_filter hello \"[$&]\" ig;\n }\n--- request\nGET /t\n--- response_body\n[Hello] World worlD [hellO]\n\n--- stap2 eval: $::StapOutputChains\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 8: global subs (case insensitive) (3)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo 'Hello World worlD hellO';\n replace_filter world \"<$&>\" gi;\n replace_filter hello \"[$&]\" ig;\n }\n--- request\nGET /t\n--- response_body\n[Hello] [hellO]\n\n--- stap\nF(ngx_http_replace_non_capturing_parse) {\n println(\"non capturing parse\")\n}\n\nF(ngx_http_replace_capturing_parse) {\n println(\"capturing parse\")\n}\n\n--- stap_out_like chop\n^(capturing parse\\n)+$\n\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 9: global subs (case insensitive) - non-capturing\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo 'Hello World';\n replace_filter world \"<>\" gi;\n replace_filter hello \"[]\" ig;\n }\n--- request\nGET /t\n--- response_body\n[] <>\n\n--- stap\nF(ngx_http_replace_non_capturing_parse) {\n println(\"non capturing parse\")\n}\n\nF(ngx_http_replace_capturing_parse) {\n println(\"capturing parse\")\n}\n\n--- stap_out_like chop\n^(non capturing parse\\n)+$\n\n--- stap2 eval: $::StapOutputChains\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 10: working as a tokenizer\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo -n a;\n echo b;\n replace_filter a \"[$&]\" g;\n replace_filter ab \"<$&>\" g;\n }\n--- request\nGET /t\n--- response_body\n[a]b\n\n--- stap2 eval: $::StapOutputChains\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 11: working as a tokenizer (2)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 1;\n location /t {\n echo -n a;\n echo b;\n replace_filter ab \"<$&>\" g;\n replace_filter a \"[$&]\" g;\n }\n--- request\nGET /t\n--- response_body\n\n\n--- stap2 eval: $::StapOutputChains\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 12: on server level\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 1;\n replace_filter ab \"<$&>\" g;\n replace_filter a \"[$&]\" g;\n\n location /t {\n echo -n a;\n echo b;\n }\n--- request\nGET /t\n--- response_body\n\n\n--- stap2 eval: $::StapOutputChains\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 13: mixing once and global patterns\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 1;\n location /t {\n echo hello world hiya hiya world hello;\n replace_filter hello \"<$&>\";\n replace_filter hiya \"{$&}\";\n replace_filter world \"[$&]\" g;\n }\n--- request\nGET /t\n--- response_body\n [world] {hiya} hiya [world] hello\n\n--- stap2 eval: $::StapOutputChains\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 14: all once\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 1;\n location /t {\n echo hello world hiya hiya world hello;\n replace_filter hello \"<$&>\";\n replace_filter hiya \"{$&}\";\n replace_filter world \"[$&]\";\n }\n--- request\nGET /t\n--- response_body\n [world] {hiya} hiya world hello\n\n--- stap\nF(ngx_http_replace_complex_value) {\n println(\"complex value\")\n}\n\n--- stap_out\ncomplex value\ncomplex value\ncomplex value\n\n--- stap2 eval: $::StapOutputChains\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 15: all once (server level)\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 1;\n replace_filter hello \"<$&>\";\n replace_filter hiya \"{$&}\";\n replace_filter world \"[$&]\";\n\n location /t {\n echo hello world hiya hiya world hello;\n }\n--- request\nGET /t\n--- response_body\n [world] {hiya} hiya world hello\n\n--- stap\nF(ngx_http_replace_complex_value) {\n println(\"complex value\")\n}\n\n--- stap_out\ncomplex value\ncomplex value\ncomplex value\n\n--- stap2 eval: $::StapOutputChains\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 16: remove C/C++ comments (1 byte at a time)\n--- config\n replace_filter_max_buffered_size 50;\n default_type text/html;\n location /a.html {\n internal;\n }\n\n location = /t {\n content_by_lua '\n local res = ngx.location.capture(\"/a.html\")\n local txt = res.body\n for i = 1, string.len(txt) do\n ngx.print(string.sub(txt, i, i))\n ngx.flush(true)\n end\n ';\n replace_filter \"'(?:\\\\\\\\[^\\n]|[^'\\n])*'\" $& g;\n replace_filter '\"(?:\\\\\\\\[^\\n]|[^\"\\n])*\"' $& g;\n replace_filter '/\\*.*?\\*/|//[^\\n]*' '' g;\n }\n--- user_files\n>>> a.html\nb = '\"'; /* blah */ c = '\"'\na = \"h\\\"/* */\";\n i don't know // hello // world /* */\nhello world /** abc * b/c /*\n hello ** // world\n *\n */\nblah /* hi */ */ b\n//\n///hi\n--- stap2\nF(ngx_palloc) {\n if ($size < 0) {\n print_ubacktrace()\n exit()\n }\n}\n--- request\nGET /t\n--- response_body eval\nqq{b = '\"'; c = '\"'\na = \"h\\\\\"/* */\";\n i don't know \nhello world \nblah */ b\n\n\n}\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 17: more patterns\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 1;\n location /t {\n #echo hello world hiya hiya world hello;\n replace_filter a A;\n replace_filter b B;\n replace_filter c C;\n replace_filter d D;\n replace_filter e E;\n replace_filter f F;\n replace_filter g G;\n replace_filter h H;\n replace_filter i I;\n replace_filter j J;\n replace_filter k K;\n replace_filter l L;\n replace_filter m M;\n replace_filter n N;\n replace_filter o O;\n replace_filter p P;\n replace_filter q Q;\n replace_filter r R;\n replace_filter s S;\n replace_filter t T;\n replace_filter u U;\n replace_filter v V;\n replace_filter w W;\n replace_filter x X;\n replace_filter y Y;\n replace_filter z Z;\n }\n--- request\nGET /t\n--- user_files\n>>> t\nIt'll be officially possible when the timer_by_lua directive is\nimplemented in ngx_lua :)\n\nFor now, people have been using some tricks to do something like that,\ni.e., using detached long-running requests (by calling ngx.eof early).\nSee the related documentation for details:\n\n--- response_body\nIT'Ll BE OFfICiAllY PoSsible WHeN the tiMeR_by_lUa DirectiVe is\nimplemented in nGX_lua :)\n\nFor now, people have been using some tricKs to do something like that,\ni.e., using detached long-running reQuests (by calling ngx.eof early).\nSee the related documentation for details:\n\n--- stap2 eval: $::StapOutputChains\n--- no_error_log\n[alert]\n[error]\n\n" }, { "path": "t/08-gzip.t", "content": "# vim:set ft= ts=4 sw=4 et fdm=marker:\n\nuse lib 'lib';\nuse Test::Nginx::Socket;\n\n#worker_connections(1014);\n#master_on();\n#workers(2);\n#log_level('warn');\n\nrepeat_each(2);\n\nno_shuffle();\n\nplan tests => repeat_each() * (blocks() * 3);\n\nrun_tests();\n\n__DATA__\n\n=== TEST 1: once patterns\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n content_by_lua '\n ngx.header.content_encoding = \"gzip\"\n ngx.say(\"hello world world hello\");\n ';\n replace_filter hello \"[$&]\";\n replace_filter world \"<$&>\";\n }\n--- request\nGET /t\n--- response_body\nhello world world hello\n--- no_error_log\n[error]\n\n" }, { "path": "t/09-unused.t", "content": "# vim:set ft= ts=4 sw=4 et fdm=marker:\n\nuse lib 'lib';\nuse Test::Nginx::Socket;\n\n#worker_connections(1014);\n#master_on();\n#workers(2);\n#log_level('warn');\n\nrepeat_each(2);\n\n#no_shuffle();\n\nplan tests => repeat_each() * (blocks() * 5);\n\nrun_tests();\n\n__DATA__\n\n=== TEST 1: used\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo abcabcabde;\n replace_filter abcabd X;\n }\n--- request\nGET /t\n\n--- stap\nF(ngx_http_replace_header_filter) {\n println(\"replace header filter\")\n}\n\nF(ngx_http_replace_body_filter) {\n println(\"replace body filter\")\n}\n\n--- stap_out\nreplace header filter\nreplace body filter\nreplace body filter\n\n--- response_body\nabcXe\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 2: unused\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo abcabcabde;\n #replace_filter abcabd X;\n }\n--- request\nGET /t\n\n--- stap\nF(ngx_http_replace_header_filter) {\n println(\"replace header filter\")\n}\n\nF(ngx_http_replace_body_filter) {\n println(\"replace body filter\")\n}\n\n--- stap_out\n\n--- response_body\nabcabcabde\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 3: used (multi http {} blocks)\nThis test case won't run with nginx 1.9.3+ since duplicate http {} blocks\nhave been prohibited since then.\n--- SKIP\n--- config\n default_type text/html;\n replace_filter_max_buffered_size 0;\n location /t {\n echo abcabcabde;\n replace_filter abcabd X;\n }\n--- post_main_config\n http {\n }\n\n--- request\nGET /t\n\n--- stap\nF(ngx_http_replace_header_filter) {\n println(\"replace header filter\")\n}\n\nF(ngx_http_replace_body_filter) {\n println(\"replace body filter\")\n}\n\n--- stap_out\nreplace header filter\nreplace body filter\nreplace body filter\n\n--- response_body\nabcXe\n--- no_error_log\n[alert]\n[error]\n\n" }, { "path": "t/10-last-modified.t", "content": "# vim:set ft= ts=4 sw=4 et fdm=marker:\n\nuse lib 'lib';\nuse Test::Nginx::Socket;\n\n#worker_connections(1014);\n#master_on();\n#workers(2);\n#log_level('warn');\n\nrepeat_each(2);\n\n#no_shuffle();\n\nplan tests => repeat_each() * (blocks() * 5);\n\nrun_tests();\n\n__DATA__\n\n=== TEST 1: replace_filter_last_modified clear\n--- config\n default_type text/html;\n replace_filter_last_modified clear;\n location /t {\n content_by_lua '\n ngx.header[\"Last-Modified\"] = \"Wed, 20 Nov 2013 05:30:35 GMT\"\n ngx.say(\"ok\")\n ';\n replace_filter abcabd X;\n }\n--- request\nGET /t\n\n--- response_body\nok\n--- response_headers\n!Last-Modified\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 2: replace_filter_last_modified keep\n--- config\n default_type text/html;\n replace_filter_last_modified keep;\n location /t {\n content_by_lua '\n ngx.header[\"Last-Modified\"] = \"Wed, 20 Nov 2013 05:30:35 GMT\"\n ngx.say(\"ok\")\n ';\n replace_filter abcabd X;\n }\n--- request\nGET /t\n\n--- response_body\nok\n--- response_headers\nLast-Modified: Wed, 20 Nov 2013 05:30:35 GMT\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 3: replace_filter_last_modified default to clear\n--- config\n default_type text/html;\n #replace_filter_last_modified clear;\n location /t {\n content_by_lua '\n ngx.header[\"Last-Modified\"] = \"Wed, 20 Nov 2013 05:30:35 GMT\"\n ngx.say(\"ok\")\n ';\n replace_filter abcabd X;\n }\n--- request\nGET /t\n\n--- response_body\nok\n--- response_headers\n!Last-Modified\n--- no_error_log\n[alert]\n[error]\n\n" }, { "path": "t/11-skip.t", "content": "# vim:set ft= ts=4 sw=4 et fdm=marker:\n\nuse lib 'lib';\nuse Test::Nginx::Socket;\n\n#worker_connections(1014);\n#master_on();\n#workers(2);\n#log_level('warn');\n\nrepeat_each(2);\n\n#no_shuffle();\n\nplan tests => repeat_each() * (blocks() * 4);\n\nrun_tests();\n\n__DATA__\n\n=== TEST 1: skip true (constant)\n--- config\n default_type text/html;\n replace_filter_skip 1;\n location /t {\n content_by_lua '\n ngx.say(\"abcabd\")\n ';\n replace_filter abcabd X;\n }\n--- request\nGET /t\n\n--- response_body\nabcabd\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 2: skip false (constant 0)\n--- config\n default_type text/html;\n replace_filter_skip 0;\n location /t {\n content_by_lua '\n ngx.say(\"abcabd\")\n ';\n replace_filter abcabd X;\n }\n--- request\nGET /t\n\n--- response_body\nX\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 3: skip false (constant \"\")\n--- config\n default_type text/html;\n replace_filter_skip \"\";\n location /t {\n content_by_lua '\n ngx.say(\"abcabd\")\n ';\n replace_filter abcabd X;\n }\n--- request\nGET /t\n\n--- response_body\nX\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 4: skip true (constant, random strings)\n--- config\n default_type text/html;\n replace_filter_skip ab;\n location /t {\n content_by_lua '\n ngx.say(\"abcabd\")\n ';\n replace_filter abcabd X;\n }\n--- request\nGET /t\n\n--- response_body\nabcabd\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 5: skip variable (1)\n--- config\n default_type text/html;\n set $skip '';\n replace_filter_skip $skip;\n location /t {\n content_by_lua '\n ngx.var.skip = 1\n ngx.say(\"abcabd\")\n ';\n replace_filter abcabd X;\n }\n--- request\nGET /t\n\n--- response_body\nabcabd\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 6: skip variable (0)\n--- config\n default_type text/html;\n set $skip '';\n replace_filter_skip $skip;\n location /t {\n content_by_lua '\n ngx.var.skip = 0\n ngx.say(\"abcabd\")\n ';\n replace_filter abcabd X;\n }\n--- request\nGET /t\n\n--- response_body\nX\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 7: skip variable (\"\")\n--- config\n default_type text/html;\n set $skip '';\n replace_filter_skip $skip;\n location /t {\n content_by_lua '\n ngx.var.skip = \"\"\n ngx.say(\"abcabd\")\n ';\n replace_filter abcabd X;\n }\n--- request\nGET /t\n\n--- response_body\nX\n--- no_error_log\n[alert]\n[error]\n\n\n\n=== TEST 8: skip variable (nil)\n--- config\n default_type text/html;\n set $skip '';\n replace_filter_skip $skip;\n location /t {\n content_by_lua '\n ngx.var.skip = nil\n ngx.say(\"abcabd\")\n ';\n replace_filter abcabd X;\n }\n--- request\nGET /t\n\n--- response_body\nX\n--- no_error_log\n[alert]\n[error]\n\n" }, { "path": "util/build.sh", "content": "#!/bin/bash\n\n# this file is mostly meant to be used by the author himself.\n\nroot=`pwd`\nhome=~\nversion=$1\nforce=$2\n\nngx-build $force $version \\\n $NGX_EXTRA_OPT \\\n --with-cc-opt=\"-I$PCRE_INC -I$OPENSSL_INC\" \\\n --with-ld-opt=\"-L$PCRE_LIB -L$OPENSSL_LIB -Wl,-rpath,$SREGEX_LIB:$PCRE_LIB:$LIBDRIZZLE_LIB:$OPENSSL_LIB\" \\\n --with-http_ssl_module \\\n --without-mail_pop3_module \\\n --without-mail_imap_module \\\n --without-mail_smtp_module \\\n --without-http_upstream_ip_hash_module \\\n --without-http_empty_gif_module \\\n --without-http_memcached_module \\\n --without-http_referer_module \\\n --without-http_autoindex_module \\\n --without-http_auth_basic_module \\\n --without-http_userid_module \\\n --add-module=$root $opts \\\n --add-module=$root/../ndk-nginx-module \\\n --add-module=$root/../lua-nginx-module \\\n --add-module=$root/../echo-nginx-module \\\n --with-debug\n #--with-cc-opt=\"-g3 -O0\"\n #--without-http_ssi_module # we cannot disable ssi because echo_location_async depends on it (i dunno why?!)\n\n" }, { "path": "valgrind.suppress", "content": "{\n \n Memcheck:Addr1\n fun:ngx_init_cycle\n fun:ngx_master_process_cycle\n fun:main\n}\n{\n \n Memcheck:Addr4\n fun:ngx_init_cycle\n fun:ngx_master_process_cycle\n fun:main\n}\n{\n \n Memcheck:Cond\n fun:ngx_vslprintf\n fun:ngx_snprintf\n fun:ngx_sock_ntop\n fun:ngx_event_accept\n fun:ngx_epoll_process_events\n fun:ngx_process_events_and_timers\n}\n{\n \n Memcheck:Cond\n fun:ngx_vslprintf\n fun:ngx_snprintf\n fun:ngx_sock_ntop\n fun:ngx_event_accept\n fun:ngx_epoll_process_events\n fun:ngx_process_events_and_timers\n}\n{\n \n Memcheck:Addr1\n fun:ngx_vslprintf\n fun:ngx_snprintf\n fun:ngx_sock_ntop\n fun:ngx_event_accept\n}\n{\n \n Memcheck:Leak\n fun:malloc\n fun:ngx_alloc\n obj:*\n}\n{\n \n exp-sgcheck:SorG\n fun:ngx_http_lua_ndk_set_var_get\n}\n{\n \n exp-sgcheck:SorG\n fun:ngx_http_variables_init_vars\n fun:ngx_http_block\n}\n{\n \n exp-sgcheck:SorG\n fun:ngx_conf_parse\n}\n{\n \n exp-sgcheck:SorG\n fun:ngx_vslprintf\n fun:ngx_log_error_core\n}\n{\n \n Memcheck:Leak\n fun:malloc\n fun:ngx_alloc\n fun:ngx_calloc\n fun:ngx_event_process_init\n}\n{\n \n Memcheck:Param\n epoll_ctl(event)\n fun:epoll_ctl\n}\n{\n \n Memcheck:Leak\n fun:malloc\n fun:ngx_alloc\n fun:ngx_event_process_init\n}\n{\n \n Memcheck:Cond\n fun:ngx_conf_flush_files\n fun:ngx_single_process_cycle\n}\n{\n \n Memcheck:Cond\n fun:memcpy\n fun:ngx_vslprintf\n fun:ngx_log_error_core\n fun:ngx_http_charset_header_filter\n}\n{\n \n Memcheck:Param\n socketcall.setsockopt(optval)\n fun:setsockopt\n fun:drizzle_state_connect\n}\n{\n \n Memcheck:Leak\n fun:malloc\n fun:ngx_alloc\n fun:ngx_pool_cleanup_add\n}\n{\n \n Memcheck:Cond\n fun:ngx_conf_flush_files\n fun:ngx_single_process_cycle\n fun:main\n}\n{\n \n Memcheck:Cond\n fun:index\n fun:expand_dynamic_string_token\n fun:_dl_map_object\n fun:map_doit\n fun:_dl_catch_error\n fun:do_preload\n fun:dl_main\n}\n{\n \n Memcheck:Leak\n match-leak-kinds: definite\n fun:malloc\n fun:ngx_alloc\n fun:ngx_set_environment\n fun:ngx_single_process_cycle\n}\n{\n \n Memcheck:Leak\n match-leak-kinds: definite\n fun:malloc\n fun:ngx_alloc\n fun:ngx_set_environment\n fun:ngx_worker_process_init\n fun:ngx_worker_process_cycle\n}\n" } ]