Full Code of ncr/rack-proxy for AI

master 57973871f4f8 cached
20 files
31.7 KB
8.8k tokens
64 symbols
1 requests
Download .txt
Repository: ncr/rack-proxy
Branch: master
Commit: 57973871f4f8
Files: 20
Total size: 31.7 KB

Directory structure:
gitextract_la8_nwgm/

├── .github/
│   └── FUNDING.yml
├── .gitignore
├── .travis.yml
├── Gemfile
├── LICENSE
├── README.md
├── Rakefile
├── lib/
│   ├── net_http_hacked.rb
│   ├── rack/
│   │   ├── http_streaming_response.rb
│   │   └── proxy.rb
│   ├── rack-proxy.rb
│   └── rack_proxy_examples/
│       ├── example_service_proxy.rb
│       ├── forward_host.rb
│       ├── rack_php_proxy.rb
│       └── trusting_proxy.rb
├── rack-proxy.gemspec
└── test/
    ├── http_streaming_response_test.rb
    ├── net_http_hacked_test.rb
    ├── rack_proxy_test.rb
    └── test_helper.rb

================================================
FILE CONTENTS
================================================

================================================
FILE: .github/FUNDING.yml
================================================
# These are supported funding model platforms

github: [ncr]


================================================
FILE: .gitignore
================================================
pkg/*
*.gem
.bundle


================================================
FILE: .travis.yml
================================================
cache: bundler
language: ruby
before_install:
  - "echo 'gem: --no-ri --no-rdoc' > ~/.gemrc"
  - gem install bundler
  - gem update bundler
script: bundle exec rake test
rvm:
  - 2.0.0
  - 2.1.5
  - 2.2.2
  - 2.2.3
  - 2.3.0
  - 2.3.1
env:
 - RAILS_ENV=test RACK_ENV=test
notifications:
  email: false


================================================
FILE: Gemfile
================================================
source "https://rubygems.org"

gem 'rake'

# Specify your gem's dependencies in rack-proxy.gemspec
gemspec


================================================
FILE: LICENSE
================================================
The MIT License (MIT)

Copyright (c) 2013 Jacek Becela jacek.becela@gmail.com

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.


================================================
FILE: README.md
================================================
A request/response rewriting HTTP proxy. A Rack app. Subclass `Rack::Proxy` and provide your `rewrite_env` and `rewrite_response` methods.

Installation
----

Add the following to your `Gemfile`:

```
gem 'rack-proxy', '~> 0.7.7'
```

Or install:

```
gem install rack-proxy
```

Use Cases
----

Below are some examples of real world use cases for Rack-Proxy. If you have done something interesting, add it to the list below and send a PR.

* Allowing one app to act as central trust authority
  * handle accepting self-sign certificates for internal apps
  * authentication / authorization prior to proxying requests to a blindly trusting backend
  * avoiding CORs complications by proxying from same domain to another backend
* subdomain based pass-through to multiple apps
* Complex redirect rules
   * redirect pages with different extensions (ex: `.php`) to another app
   * useful for handling awkward redirection rules for moved pages
* fan Parallel Requests: turning a single API request to [multiple concurrent backend requests](https://github.com/typhoeus/typhoeus#making-parallel-requests) & merging results.
* inserting or stripping headers required or problematic for certain clients

Options
----

Options can be set when initializing the middleware or overriding a method.


* `:streaming` - defaults to `true`, but does not work on all Ruby versions, recommend to set to `false`
* `:ssl_verify_none` - tell `Net::HTTP` to not validate certs
* `:ssl_version` - tell `Net::HTTP` to set a specific `ssl_version`
* `:backend` - the URI parseable format of host and port of the target proxy backend. If not set it will assume the backend target is the same as the source.
* `:read_timeout` - set proxy timeout it defaults to 60 seconds

To pass in options, when you configure your middleware you can pass them in as an optional hash.

```ruby
Rails.application.config.middleware.use ExampleServiceProxy, backend: 'http://guides.rubyonrails.org', streaming: false
```

Examples
----

See and run the examples below from `lib/rack_proxy_examples/`. To mount any example into an existing Rails app:

1. create `config/initializers/proxy.rb`
2. modify the file to require the example file
```ruby
require 'rack_proxy_examples/forward_host'
```

### Forward request to Host and Insert Header

Test with `require 'rack_proxy_examples/forward_host'`

```ruby
class ForwardHost < Rack::Proxy

  def rewrite_env(env)
    env["HTTP_HOST"] = "example.com"
    env
  end

  def rewrite_response(triplet)
    status, headers, body = triplet

    # example of inserting an additional header
    headers["X-Foo"] = "Bar"

    # if you rewrite env, it appears that content-length isn't calculated correctly
    # resulting in only partial responses being sent to users
    # you can remove it or recalculate it here
    headers["content-length"] = nil

    triplet
  end

end
```

### Disable SSL session verification when proxying a server with e.g. self-signed SSL certs

Test with `require 'rack_proxy_examples/trusting_proxy'`

```ruby
class TrustingProxy < Rack::Proxy

  def rewrite_env(env)
    env["HTTP_HOST"] = "self-signed.badssl.com"

    # We are going to trust the self-signed SSL
    env["rack.ssl_verify_none"] = true
    env
  end

  def rewrite_response(triplet)
    status, headers, body = triplet

    # if you rewrite env, it appears that content-length isn't calculated correctly
    # resulting in only partial responses being sent to users
    # you can remove it or recalculate it here
    headers["content-length"] = nil

    triplet
  end

end
```

The same can be achieved for *all* requests going through the `Rack::Proxy` instance by using

```ruby
Rack::Proxy.new(ssl_verify_none: true)
```

### Rails middleware example

Test with `require 'rack_proxy_examples/example_service_proxy'`

```ruby
###
# This is an example of how to use Rack-Proxy in a Rails application.
#
# Setup:
# 1. rails new test_app
# 2. cd test_app
# 3. install Rack-Proxy in `Gemfile`
#    a. `gem 'rack-proxy', '~> 0.7.7'`
# 4. install gem: `bundle install`
# 5. create `config/initializers/proxy.rb` adding this line `require 'rack_proxy_examples/example_service_proxy'`
# 6. run: `SERVICE_URL=http://guides.rubyonrails.org rails server`
# 7. open in browser: `http://localhost:3000/example_service`
#
###
ENV['SERVICE_URL'] ||= 'http://guides.rubyonrails.org'

class ExampleServiceProxy < Rack::Proxy
  def perform_request(env)
    request = Rack::Request.new(env)

    # use rack proxy for anything hitting our host app at /example_service
    if request.path =~ %r{^/example_service}
        backend = URI(ENV['SERVICE_URL'])
        # most backends required host set properly, but rack-proxy doesn't set this for you automatically
        # even when a backend host is passed in via the options
        env["HTTP_HOST"] = backend.host

        # This is the only path that needs to be set currently on Rails 5 & greater
        env['PATH_INFO'] = ENV['SERVICE_PATH'] || '/configuring.html'

        # don't send your sites cookies to target service, unless it is a trusted internal service that can parse all your cookies
        env['HTTP_COOKIE'] = ''
        super(env)
    else
      @app.call(env)
    end
  end
end
```

### Using as middleware to forward only some extensions to another Application

Test with `require 'rack_proxy_examples/rack_php_proxy'`

Example: Proxying only requests that end with ".php" could be done like this:

```ruby
###
# Open http://localhost:3000/test.php to trigger proxy
###
class RackPhpProxy < Rack::Proxy

  def perform_request(env)
    request = Rack::Request.new(env)
    if request.path =~ %r{\.php}
      env["HTTP_HOST"] = ENV["HTTP_HOST"] ? URI(ENV["HTTP_HOST"]).host : "localhost"
      ENV["PHP_PATH"] ||= '/manual/en/tutorial.firstpage.php'

      # Rails 3 & 4
      env["REQUEST_PATH"] = ENV["PHP_PATH"] || "/php/#{request.fullpath}"
      # Rails 5 and above
      env['PATH_INFO'] = ENV["PHP_PATH"] || "/php/#{request.fullpath}"

      env['content-length'] = nil

      super(env)
    else
      @app.call(env)
    end
  end

  def rewrite_response(triplet)
    status, headers, body = triplet

    # if you proxy depending on the backend, it appears that content-length isn't calculated correctly
    # resulting in only partial responses being sent to users
    # you can remove it or recalculate it here
    headers["content-length"] = nil

    triplet
  end
end
```

To use the middleware, please consider the following:

1) For Rails we could add a configuration in `config/application.rb`

```ruby
  config.middleware.use RackPhpProxy, {ssl_verify_none: true}
```

2) For Sinatra or any Rack-based application:

```ruby
class MyAwesomeSinatra < Sinatra::Base
   use  RackPhpProxy, {ssl_verify_none: true}
end
```

This will allow to run the other requests through the application and only proxy the requests that match the condition from the middleware.

See tests for more examples.

### SSL proxy for SpringBoot applications debugging

Whenever you need to debug communication with external services with HTTPS protocol (like OAuth based) you have to be able to access to your local web app through HTTPS protocol too. Typical way is to use nginx or Apache httpd as a reverse proxy but it might be inconvinuent for development environment. Simple proxy server is a better way in this case. The only what we need is to unpack incoming SSL queries and proxy them to a backend. We can prepare minimal set of files to create autonomous proxy server.

Create `config.ru` file:
```ruby
#
# config.ru
#
require 'rack'
require 'rack-proxy'

class ForwardHost < Rack::Proxy
  def rewrite_env(env)
    env['HTTP_X_FORWARDED_HOST'] = env['SERVER_NAME']
    env['HTTP_X_FORWARDED_PROTO'] = env['rack.url_scheme']
    env
  end
end

run ForwardHost.new(backend: 'http://localhost:8080')
```

Create `Gemfile` file:
```ruby
source "https://rubygems.org"

gem 'thin'
gem 'rake'
gem 'rack-proxy'
```

Create `config.yml` file with configuration of web server `thin`:
```yml
---
ssl: true
ssl-key-file: keys/domain.key
ssl-cert-file: keys/domain.crt
ssl-disable-verify: false
```

Create 'keys' directory and generate SSL key and certificates files `domain.key` and `domain.crt`

Run `bundle exec thin start` for running it with `thin`'s default port.

Or use `sudo -E thin start -C config.yml -p 443` for running with default for `https://` port.

Don't forget to enable processing of `X-Forwarded-...` headers on your application side. Just add following strings to your `resources/application.yml` file.
```yml
---
server:
  tomcat:
    remote-ip-header: x-forwarded-for
    protocol-header:  x-forwarded-proto
  use-forward-headers:  true
```

Add some domain name like `debug.your_app.com` into your local `/etc/hosts` file like
```
127.0.0.1	debug.your_app.com
```

Next start the proxy and your app. And now you can access to your Spring application through SSL connection via `https://debug.your_app.com` URI in a browser.

### Using SSL/TLS certificates with HTTP connection
This may be helpful, when third-party API has authentication by client TLS certificates and you need to proxy your requests and sign them with certificate.

Just specify Rack::Proxy SSL options and your request will use TLS HTTP connection:
```ruby
# config.ru
. . .

cert_raw = File.read('./certs/rootCA.crt')
key_raw = File.read('./certs/key.pem')

cert = OpenSSL::X509::Certificate.new(cert_raw)
key = OpenSSL::PKey.read(key_raw)

use TLSProxy, cert: cert, key: key, use_ssl: true, verify_mode: OpenSSL::SSL::VERIFY_PEER, ssl_version: 'TLSv1_2'
```

And rewrite host for example:
```ruby
# tls_proxy.rb
class TLSProxy < Rack::Proxy
  attr_accessor :original_request, :query_params

  def rewrite_env(env)
    env["HTTP_HOST"] = "client-tls-auth-api.com:443"
    env
  end
end
```

WARNING
----

Doesn't work with `fakeweb`/`webmock`. Both libraries monkey-patch net/http code.

Todos
----

* Make the docs up to date with the current use case for this code: everything except streaming which involved a rather ugly monkey patch and only worked in 1.8, but does not work now.
* Improve and validate requirements for Host and Path rewrite rules
* Ability to inject logger and set log level


================================================
FILE: Rakefile
================================================
require 'rubygems'
require 'bundler'
Bundler::GemHelper.install_tasks

require "rake/testtask"
task :test do
  Rake::TestTask.new do |t|
    t.libs << "test"
    t.test_files = FileList['test/*_test.rb']
    t.verbose = true
  end
end

task :default => :test


================================================
FILE: lib/net_http_hacked.rb
================================================
# We are hacking net/http to change semantics of streaming handling
# from "block" semantics to regular "return" semantics.
# We need it to construct a streamable rack triplet:
#
# [status, headers, streamable_body]
#
# See http://github.com/zerowidth/rack-streaming-proxy
# for alternative that uses additional process.
#
# BTW I don't like monkey patching either
# but this is not real monkey patching.
# I just added some methods and named them very uniquely
# to avoid eventual conflicts. You're safe. Trust me.
#
# Also, in Ruby 1.9.2 you could use Fibers to avoid hacking net/http.

require 'net/https'

class Net::HTTP
  # Original #request with block semantics.
  #
  # def request(req, body = nil, &block)
  #   unless started?
  #     start {
  #       req['connection'] ||= 'close'
  #       return request(req, body, &block)
  #     }
  #   end
  #   if proxy_user()
  #     unless use_ssl?
  #       req.proxy_basic_auth proxy_user(), proxy_pass()
  #     end
  #   end
  #
  #   req.set_body_internal body
  #   begin_transport req
  #     req.exec @socket, @curr_http_version, edit_path(req.path)
  #     begin
  #       res = HTTPResponse.read_new(@socket)
  #     end while res.kind_of?(HTTPContinue)
  #     res.reading_body(@socket, req.response_body_permitted?) {
  #       yield res if block_given?
  #     }
  #   end_transport req, res
  #
  #   res
  # end

  def begin_request_hacked(req)
    begin_transport req
    req.exec @socket, @curr_http_version, edit_path(req.path)
    begin
      res = Net::HTTPResponse.read_new(@socket)
    end while res.kind_of?(Net::HTTPContinue)
    res.begin_reading_body_hacked(@socket, req.response_body_permitted?)
    @req_hacked, @res_hacked = req, res
    @res_hacked
  end

  def end_request_hacked
    @res_hacked.end_reading_body_hacked
    end_transport @req_hacked, @res_hacked
    @res_hacked
  end
end

class Net::HTTPResponse
  # Original #reading_body with block semantics
  #
  # def reading_body(sock, reqmethodallowbody)  #:nodoc: internal use only
  #   @socket = sock
  #   @body_exist = reqmethodallowbody && self.class.body_permitted?
  #   begin
  #     yield
  #     self.body   # ensure to read body
  #   ensure
  #     @socket = nil
  #   end
  # end

  def begin_reading_body_hacked(sock, reqmethodallowbody)
    @socket = sock
    @body_exist = reqmethodallowbody && self.class.body_permitted?
  end

  def end_reading_body_hacked
    self.body
    @socket = nil
  end
end


================================================
FILE: lib/rack/http_streaming_response.rb
================================================
require "net_http_hacked"
require "stringio"

module Rack
  # Wraps the hacked net/http in a Rack way.
  class HttpStreamingResponse
    STATUSES_WITH_NO_ENTITY_BODY = {
      204 => true,
      205 => true,
      304 => true
    }.freeze

    attr_accessor :use_ssl, :verify_mode, :read_timeout, :ssl_version, :cert, :key

    def initialize(request, host, port = nil)
      @request, @host, @port = request, host, port
    end

    def body
      self
    end

    def code
      response.code.to_i.tap do |response_code|
        STATUSES_WITH_NO_ENTITY_BODY[response_code] && close_connection
      end
    end
    # #status is deprecated
    alias_method :status, :code

    def headers
      Rack::Proxy.build_header_hash(response.to_hash)
    end

    # Can be called only once!
    def each(&block)
      return if connection_closed

      response.read_body(&block)
    ensure
      close_connection
    end

    def to_s
      @to_s ||= StringIO.new.tap { |io| each { |line| io << line } }.string
    end

    protected

    # Net::HTTPResponse
    def response
      @response ||= session.begin_request_hacked(request)
    end

    # Net::HTTP
    def session
      @session ||= Net::HTTP.new(host, port).tap do |http|
        http.use_ssl = use_ssl
        http.verify_mode = verify_mode
        http.read_timeout = read_timeout
        http.ssl_version = ssl_version if ssl_version
        http.cert = cert if cert
        http.key = key if key
        http.start
      end
    end

    private

    attr_reader :request, :host, :port

    attr_accessor :connection_closed

    def close_connection
      return if connection_closed

      session.end_request_hacked
      session.finish
      self.connection_closed = true
    end
  end
end


================================================
FILE: lib/rack/proxy.rb
================================================
require "net_http_hacked"
require "rack/http_streaming_response"

module Rack

  # Subclass and bring your own #rewrite_request and #rewrite_response
  class Proxy
    VERSION = "0.7.7".freeze

    HOP_BY_HOP_HEADERS = {
      'connection' => true,
      'keep-alive' => true,
      'proxy-authenticate' => true,
      'proxy-authorization' => true,
      'te' => true,
      'trailer' => true,
      'transfer-encoding' => true,
      'upgrade' => true
    }.freeze

    class << self
      def extract_http_request_headers(env)
        headers = env.reject do |k, v|
          !(/^HTTP_[A-Z0-9_\.]+$/ === k) || v.nil?
        end.map do |k, v|
          [reconstruct_header_name(k), v]
        end.then { |pairs| build_header_hash(pairs) }

        x_forwarded_for = (headers['X-Forwarded-For'].to_s.split(/, +/) << env['REMOTE_ADDR']).join(', ')

        headers.merge!('X-Forwarded-For' => x_forwarded_for)
      end

      def normalize_headers(headers)
        mapped = headers.map do |k, v|
          [titleize(k), if v.is_a? Array then v.join("\n") else v end]
        end
        build_header_hash Hash[mapped]
      end

      def build_header_hash(pairs)
        if Rack.const_defined?(:Headers)
          # Rack::Headers is only available from Rack 3 onward
          Headers.new.tap { |headers| pairs.each { |k, v| headers[k] = v } }
        else
          # Rack::Utils::HeaderHash is deprecated from Rack 3 onward and is to be removed in 3.1
          Utils::HeaderHash.new(pairs)
        end
      end

      protected

      def reconstruct_header_name(name)
        titleize(name.sub(/^HTTP_/, "").gsub("_", "-"))
      end

      def titleize(str)
        str.split("-").map(&:capitalize).join("-")
      end
    end

    # @option opts [String, URI::HTTP] :backend Backend host to proxy requests to
    def initialize(app = nil, opts= {})
      if app.is_a?(Hash)
        opts = app
        @app = nil
      else
        @app = app
      end

      @streaming = opts.fetch(:streaming, true)
      @ssl_verify_none = opts.fetch(:ssl_verify_none, false)
      @backend = opts[:backend] ? URI(opts[:backend]) : nil
      @read_timeout = opts.fetch(:read_timeout, 60)
      @ssl_version = opts[:ssl_version]
      @cert = opts[:cert]
      @key = opts[:key]
      @verify_mode = opts[:verify_mode]

      @username = opts[:username]
      @password = opts[:password]

      @opts = opts
    end

    def call(env)
      rewrite_response(perform_request(rewrite_env(env)))
    end

    # Return modified env
    def rewrite_env(env)
      env
    end

    # Return a rack triplet [status, headers, body]
    def rewrite_response(triplet)
      triplet
    end

    protected

    def perform_request(env)
      source_request = Rack::Request.new(env)

      # Initialize request
      if source_request.fullpath == ""
        full_path = URI.parse(env['REQUEST_URI']).request_uri
      else
        full_path = source_request.fullpath
      end

      target_request = Net::HTTP.const_get(source_request.request_method.capitalize, false).new(full_path)

      # Setup headers
      target_request.initialize_http_header(self.class.extract_http_request_headers(source_request.env))

      # Setup body
      if target_request.request_body_permitted? && source_request.body
        target_request.body_stream    = source_request.body
        target_request.content_length = source_request.content_length.to_i
        target_request.content_type   = source_request.content_type if source_request.content_type
        target_request.body_stream.rewind
      end

      # Use basic auth if we have to
      target_request.basic_auth(@username, @password) if @username && @password

      backend = env.delete('rack.backend') || @backend || source_request
      use_ssl = backend.scheme == "https" || @cert
      read_timeout = env.delete('http.read_timeout') || @read_timeout

      # Create the response
      if @streaming
        # streaming response (the actual network communication is deferred, a.k.a. streamed)
        target_response = HttpStreamingResponse.new(target_request, backend.host, backend.port)
        target_response.use_ssl = use_ssl
        target_response.read_timeout = read_timeout
        target_response.ssl_version = @ssl_version if @ssl_version
        target_response.verify_mode = (@verify_mode || OpenSSL::SSL::VERIFY_NONE) if use_ssl
        target_response.cert = @cert if @cert
        target_response.key = @key if @key
      else
        http = Net::HTTP.new(backend.host, backend.port)
        http.use_ssl = use_ssl if use_ssl
        http.read_timeout = read_timeout
        http.ssl_version = @ssl_version if @ssl_version
        http.verify_mode = (@verify_mode || OpenSSL::SSL::VERIFY_NONE if use_ssl) if use_ssl
        http.cert = @cert if @cert
        http.key = @key if @key

        target_response = http.start do
          http.request(target_request)
        end
      end

      code    = target_response.code
      headers = self.class.normalize_headers(target_response.respond_to?(:headers) ? target_response.headers : target_response.to_hash)
      body    = target_response.body || [""]
      body    = [body] unless body.respond_to?(:each)

      # According to https://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-14#section-7.1.3.1Acc
      # should remove hop-by-hop header fields
      headers.reject! { |k| HOP_BY_HOP_HEADERS[k.downcase] }

      [code, headers, body]
    end
  end
end


================================================
FILE: lib/rack-proxy.rb
================================================
require "rack/proxy"

================================================
FILE: lib/rack_proxy_examples/example_service_proxy.rb
================================================
###
# This is an example of how to use Rack-Proxy in a Rails application.
#
# Setup:
# 1. rails new test_app
# 2. cd test_app
# 3. install Rack-Proxy in `Gemfile`
#    a. `gem 'rack-proxy', '~> 0.7.7'`
# 4. install gem: `bundle install`
# 5. create `config/initializers/proxy.rb` adding this line `require 'rack_proxy_examples/example_service_proxy'`
# 6. run: `SERVICE_URL=http://guides.rubyonrails.org rails server`
# 7. open in browser: `http://localhost:3000/example_service`
#
###
ENV['SERVICE_URL'] ||= 'http://guides.rubyonrails.org'

class ExampleServiceProxy < Rack::Proxy
  def perform_request(env)
    request = Rack::Request.new(env)

    # use rack proxy for anything hitting our host app at /example_service
    if request.path =~ %r{^/example_service}
        backend = URI(ENV['SERVICE_URL'])
        # most backends required host set properly, but rack-proxy doesn't set this for you automatically
        # even when a backend host is passed in via the options
        env["HTTP_HOST"] = backend.host

        # This is the only path that needs to be set currently on Rails 5 & greater
        env['PATH_INFO'] = ENV['SERVICE_PATH'] || '/configuring.html'

        # don't send your sites cookies to target service, unless it is a trusted internal service that can parse all your cookies
        env['HTTP_COOKIE'] = ''
        super(env)
    else
      @app.call(env)
    end
  end
end

Rails.application.config.middleware.use ExampleServiceProxy, backend: ENV['SERVICE_URL'], streaming: false


================================================
FILE: lib/rack_proxy_examples/forward_host.rb
================================================
class ForwardHost < Rack::Proxy

  def rewrite_env(env)
    env["HTTP_HOST"] = "example.com"
    env
  end

  def rewrite_response(triplet)
    status, headers, body = triplet

    # example of inserting an additional header
    headers["X-Foo"] = "Bar"
    
    # if you rewrite env, it appears that content-length isn't calculated correctly
    # resulting in only partial responses being sent to users
    # you can remove it or recalculate it here
    headers["content-length"] = nil

    triplet
  end

end

Rails.application.config.middleware.use ForwardHost, backend: 'http://example.com', streaming: false


================================================
FILE: lib/rack_proxy_examples/rack_php_proxy.rb
================================================
###
# Open http://localhost:3000/test.php to trigger proxy
###
class RackPhpProxy < Rack::Proxy

  def perform_request(env)
    request = Rack::Request.new(env)
    if request.path =~ %r{\.php}
      env["HTTP_HOST"] = ENV["HTTP_HOST"] ? URI(ENV["HTTP_HOST"]).host : "localhost"
      ENV["PHP_PATH"] ||= '/manual/en/tutorial.firstpage.php'
       
      # Rails 3 & 4
      env["REQUEST_PATH"] = ENV["PHP_PATH"] || "/php/#{request.fullpath}"
      # Rails 5 and above
      env['PATH_INFO'] = ENV["PHP_PATH"] || "/php/#{request.fullpath}"

      env['content-length'] = nil
      
      super(env)
    else
      @app.call(env)
    end
  end

  def rewrite_response(triplet)
    status, headers, body = triplet
    
    # if you proxy depending on the backend, it appears that content-length isn't calculated correctly
    # resulting in only partial responses being sent to users
    # you can remove it or recalculate it here
    headers["content-length"] = nil

    triplet
  end
end

Rails.application.config.middleware.use RackPhpProxy, backend: ENV["HTTP_HOST"]='http://php.net', streaming: false


================================================
FILE: lib/rack_proxy_examples/trusting_proxy.rb
================================================
class TrustingProxy < Rack::Proxy

  def rewrite_env(env)
    env["HTTP_HOST"] = "self-signed.badssl.com"

    # We are going to trust the self-signed SSL 
    env["rack.ssl_verify_none"] = true
    env
  end

  def rewrite_response(triplet)
    status, headers, body = triplet
    
    # if you rewrite env, it appears that content-length isn't calculated correctly
    # resulting in only partial responses being sent to users
    # you can remove it or recalculate it here
    headers["content-length"] = nil

    triplet
  end

end

Rails.application.config.middleware.use TrustingProxy, backend: 'https://self-signed.badssl.com', streaming: false


================================================
FILE: rack-proxy.gemspec
================================================
# -*- encoding: utf-8 -*-
$:.push File.expand_path("../lib", __FILE__)
require "rack-proxy"

Gem::Specification.new do |s|
  s.name        = "rack-proxy"
  s.version     = Rack::Proxy::VERSION
  s.platform    = Gem::Platform::RUBY
  s.license     = 'MIT'
  s.authors     = ["Jacek Becela"]
  s.email       = ["jacek.becela@gmail.com"]
  s.homepage    = "https://github.com/ncr/rack-proxy"
  s.summary     = %q{A request/response rewriting HTTP proxy. A Rack app.}
  s.description = %q{A Rack app that provides request/response rewriting proxy capabilities with streaming.}
  s.required_ruby_version = '>= 2.6'

  s.files         = `git ls-files`.split("\n")
  s.test_files    = `git ls-files -- {test,spec,features}/*`.split("\n")
  s.executables   = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
  s.require_paths = ["lib"]

  s.add_dependency("rack")
  s.add_development_dependency("rack-test")
  s.add_development_dependency("test-unit")
end


================================================
FILE: test/http_streaming_response_test.rb
================================================
require "test_helper"
require "rack/http_streaming_response"

class HttpStreamingResponseTest < Test::Unit::TestCase

  def setup
    host, req = "example.com", Net::HTTP::Get.new("/")
    @response = Rack::HttpStreamingResponse.new(req, host, 443)
    @response.use_ssl = true
  end

  def test_streaming
    # Response status
    assert_equal 200, @response.status
    assert_equal 200, @response.status

    # Headers
    headers = @response.headers

    assert headers.size.positive?

    assert_match %r{text/html; ?charset=utf-8}, headers["content-type"].first.downcase
    assert_equal headers["content-type"], headers["CoNtEnT-TyPe"]
    assert headers["content-length"].first.to_i.positive?

    # Body
    chunks = []
    @response.body.each do |chunk|
      chunks << chunk
    end

    assert chunks.size.positive?
    chunks.each do |chunk|
      assert chunk.is_a?(String)
    end

  end

  def test_to_s
    assert_equal @response.headers["Content-Length"].first.to_i, @response.body.to_s.bytesize
  end

  def test_to_s_called_twice
    body = @response.body
    assert_equal body.to_s, body.to_s
  end

end


================================================
FILE: test/net_http_hacked_test.rb
================================================
require "test_helper"
require "net_http_hacked"

class NetHttpHackedTest < Test::Unit::TestCase
  
  def test_net_http_hacked
    req = Net::HTTP::Get.new("/")
    http = Net::HTTP.start("www.iana.org", "80")

    # Response code
    res = http.begin_request_hacked(req)
    assert res.code == "200"

    # Headers
    headers = {}
    res.each_header { |k, v| headers[k] = v }

    assert headers.size > 0
    assert headers["content-type"] == "text/html; charset=UTF-8"
    assert !headers["date"].nil?

    # Body
    chunks = []
    res.read_body do |chunk|
      chunks << chunk
    end

    assert chunks.size > 0
    chunks.each do |chunk|
      assert chunk.is_a?(String)
    end

    http.end_request_hacked
  end
  
end


================================================
FILE: test/rack_proxy_test.rb
================================================
require "test_helper"
require "rack/proxy"

class RackProxyTest < Test::Unit::TestCase
  class HostProxy < Rack::Proxy
    attr_accessor :host

    def rewrite_env(env)
      env["HTTP_HOST"] = self.host || 'example.com'
      env
    end
  end

  def app(opts = {})
    return @app ||= HostProxy.new(opts)
  end

  def test_http_streaming
    get "/"
    assert last_response.ok?

    assert_match(/Example Domain/, last_response.body)
  end

  def test_http_full_request
    app(:streaming => false)
    get "/"
    assert last_response.ok?
    assert_match(/Example Domain/, last_response.body)
  end

  def test_http_full_request_headers
    app(:streaming => false)
    app.host = 'www.google.com'
    get "/"
    assert !Array(last_response['Set-Cookie']).empty?, 'Google always sets a cookie, yo. Where my cookies at?!'
  end

  def test_https_streaming
    app.host = 'www.apple.com'
    get 'https://example.com'
    assert last_response.ok?
    assert_match(/(itunes|iphone|ipod|mac|ipad)/, last_response.body)
  end

  def test_https_streaming_tls
    app(:ssl_version => :TLSv1).host = 'www.apple.com'
    get 'https://example.com'
    assert last_response.ok?
    assert_match(/(itunes|iphone|ipod|mac|ipad)/, last_response.body)
  end

  def test_https_full_request
    app(:streaming => false).host = 'www.apple.com'
    get 'https://example.com'
    assert last_response.ok?
    assert_match(/(itunes|iphone|ipod|mac|ipad)/, last_response.body)
  end

  def test_https_full_request_tls
    app({:streaming => false, :ssl_version => :TLSv1}).host = 'www.apple.com'
    get 'https://example.com'
    assert last_response.ok?
    assert_match(/(itunes|iphone|ipod|mac|ipad)/, last_response.body)
  end

  def test_normalize_headers
    proxy_class = Rack::Proxy
    headers = { 'header_array' => ['first_entry'], 'header_non_array' => :entry }

    normalized_headers = proxy_class.send(:normalize_headers, headers)
    assert normalized_headers.instance_of?(Rack::Utils::HeaderHash)
    assert normalized_headers['header_array'] == 'first_entry'
    assert normalized_headers['header_non_array'] == :entry
  end

  def test_header_reconstruction
    proxy_class = Rack::Proxy

    header = proxy_class.send(:reconstruct_header_name, "HTTP_ABC")
    assert header == "Abc"

    header = proxy_class.send(:reconstruct_header_name, "HTTP_ABC_D")
    assert header == "Abc-D"
  end

  def test_extract_http_request_headers
    proxy_class = Rack::Proxy
    env = {
      'NOT-HTTP-HEADER' => 'test-value',
      'HTTP_ACCEPT' => 'text/html',
      'HTTP_CONNECTION' => nil,
      'HTTP_CONTENT_MD5' => 'deadbeef',
      'HTTP_HEADER.WITH.PERIODS' => 'stillmooing'
    }

    headers = proxy_class.extract_http_request_headers(env)
    assert headers.key?('ACCEPT')
    assert headers.key?('CONTENT-MD5')
    assert headers.key?('HEADER.WITH.PERIODS')
    assert !headers.key?('CONNECTION')
    assert !headers.key?('NOT-HTTP-HEADER')
  end

  def test_duplicate_headers
    proxy_class = Rack::Proxy
    env = { 'Set-Cookie' => ["cookie1=foo", "cookie2=bar"] }

    headers = proxy_class.normalize_headers(env)
    assert headers['Set-Cookie'].include?('cookie1=foo'), "Include the first value"
    assert headers['Set-Cookie'].include?("\n"), "Join multiple cookies with newlines"
    assert headers['Set-Cookie'].include?('cookie2=bar'), "Include the second value"
  end


  def test_handles_missing_content_length
    assert_nothing_thrown do
      post "/", nil, "CONTENT_LENGTH" => nil
    end
  end

  def test_response_header_included_Hop_by_hop
    app({:streaming => true}).host = 'mockapi.io'
    get 'https://example.com/oauth2/token/info?access_token=123'
    assert !last_response.headers.key?('transfer-encoding')
  end
end


================================================
FILE: test/test_helper.rb
================================================
require "rubygems"
require 'bundler/setup'
require 'bundler/gem_tasks'
require "test/unit"

require "rack"
require "rack/test"

Test::Unit::TestCase.class_eval do
  include Rack::Test::Methods
end
Download .txt
gitextract_la8_nwgm/

├── .github/
│   └── FUNDING.yml
├── .gitignore
├── .travis.yml
├── Gemfile
├── LICENSE
├── README.md
├── Rakefile
├── lib/
│   ├── net_http_hacked.rb
│   ├── rack/
│   │   ├── http_streaming_response.rb
│   │   └── proxy.rb
│   ├── rack-proxy.rb
│   └── rack_proxy_examples/
│       ├── example_service_proxy.rb
│       ├── forward_host.rb
│       ├── rack_php_proxy.rb
│       └── trusting_proxy.rb
├── rack-proxy.gemspec
└── test/
    ├── http_streaming_response_test.rb
    ├── net_http_hacked_test.rb
    ├── rack_proxy_test.rb
    └── test_helper.rb
Download .txt
SYMBOL INDEX (64 symbols across 10 files)

FILE: lib/net_http_hacked.rb
  class Net::HTTP (line 19) | class Net::HTTP
    method begin_request_hacked (line 49) | def begin_request_hacked(req)
    method end_request_hacked (line 60) | def end_request_hacked
  class Net::HTTPResponse (line 67) | class Net::HTTPResponse
    method begin_reading_body_hacked (line 81) | def begin_reading_body_hacked(sock, reqmethodallowbody)
    method end_reading_body_hacked (line 86) | def end_reading_body_hacked

FILE: lib/rack/http_streaming_response.rb
  type Rack (line 4) | module Rack
    class HttpStreamingResponse (line 6) | class HttpStreamingResponse
      method initialize (line 15) | def initialize(request, host, port = nil)
      method body (line 19) | def body
      method code (line 23) | def code
      method headers (line 31) | def headers
      method each (line 36) | def each(&block)
      method to_s (line 44) | def to_s
      method response (line 51) | def response
      method session (line 56) | def session
      method close_connection (line 74) | def close_connection

FILE: lib/rack/proxy.rb
  type Rack (line 4) | module Rack
    class Proxy (line 7) | class Proxy
      method extract_http_request_headers (line 22) | def extract_http_request_headers(env)
      method normalize_headers (line 34) | def normalize_headers(headers)
      method build_header_hash (line 41) | def build_header_hash(pairs)
      method reconstruct_header_name (line 53) | def reconstruct_header_name(name)
      method titleize (line 57) | def titleize(str)
      method initialize (line 63) | def initialize(app = nil, opts= {})
      method call (line 86) | def call(env)
      method rewrite_env (line 91) | def rewrite_env(env)
      method rewrite_response (line 96) | def rewrite_response(triplet)
      method perform_request (line 102) | def perform_request(env)

FILE: lib/rack_proxy_examples/example_service_proxy.rb
  class ExampleServiceProxy (line 17) | class ExampleServiceProxy < Rack::Proxy
    method perform_request (line 18) | def perform_request(env)

FILE: lib/rack_proxy_examples/forward_host.rb
  class ForwardHost (line 1) | class ForwardHost < Rack::Proxy
    method rewrite_env (line 3) | def rewrite_env(env)
    method rewrite_response (line 8) | def rewrite_response(triplet)

FILE: lib/rack_proxy_examples/rack_php_proxy.rb
  class RackPhpProxy (line 4) | class RackPhpProxy < Rack::Proxy
    method perform_request (line 6) | def perform_request(env)
    method rewrite_response (line 25) | def rewrite_response(triplet)

FILE: lib/rack_proxy_examples/trusting_proxy.rb
  class TrustingProxy (line 1) | class TrustingProxy < Rack::Proxy
    method rewrite_env (line 3) | def rewrite_env(env)
    method rewrite_response (line 11) | def rewrite_response(triplet)

FILE: test/http_streaming_response_test.rb
  class HttpStreamingResponseTest (line 4) | class HttpStreamingResponseTest < Test::Unit::TestCase
    method setup (line 6) | def setup
    method test_streaming (line 12) | def test_streaming
    method test_to_s (line 39) | def test_to_s
    method test_to_s_called_twice (line 43) | def test_to_s_called_twice

FILE: test/net_http_hacked_test.rb
  class NetHttpHackedTest (line 4) | class NetHttpHackedTest < Test::Unit::TestCase
    method test_net_http_hacked (line 6) | def test_net_http_hacked

FILE: test/rack_proxy_test.rb
  class RackProxyTest (line 4) | class RackProxyTest < Test::Unit::TestCase
    class HostProxy (line 5) | class HostProxy < Rack::Proxy
      method rewrite_env (line 8) | def rewrite_env(env)
    method app (line 14) | def app(opts = {})
    method test_http_streaming (line 18) | def test_http_streaming
    method test_http_full_request (line 25) | def test_http_full_request
    method test_http_full_request_headers (line 32) | def test_http_full_request_headers
    method test_https_streaming (line 39) | def test_https_streaming
    method test_https_streaming_tls (line 46) | def test_https_streaming_tls
    method test_https_full_request (line 53) | def test_https_full_request
    method test_https_full_request_tls (line 60) | def test_https_full_request_tls
    method test_normalize_headers (line 67) | def test_normalize_headers
    method test_header_reconstruction (line 77) | def test_header_reconstruction
    method test_extract_http_request_headers (line 87) | def test_extract_http_request_headers
    method test_duplicate_headers (line 105) | def test_duplicate_headers
    method test_handles_missing_content_length (line 116) | def test_handles_missing_content_length
    method test_response_header_included_Hop_by_hop (line 122) | def test_response_header_included_Hop_by_hop
Condensed preview — 20 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (35K chars).
[
  {
    "path": ".github/FUNDING.yml",
    "chars": 61,
    "preview": "# These are supported funding model platforms\n\ngithub: [ncr]\n"
  },
  {
    "path": ".gitignore",
    "chars": 20,
    "preview": "pkg/*\n*.gem\n.bundle\n"
  },
  {
    "path": ".travis.yml",
    "chars": 302,
    "preview": "cache: bundler\nlanguage: ruby\nbefore_install:\n  - \"echo 'gem: --no-ri --no-rdoc' > ~/.gemrc\"\n  - gem install bundler\n  -"
  },
  {
    "path": "Gemfile",
    "chars": 107,
    "preview": "source \"https://rubygems.org\"\n\ngem 'rake'\n\n# Specify your gem's dependencies in rack-proxy.gemspec\ngemspec\n"
  },
  {
    "path": "LICENSE",
    "chars": 1102,
    "preview": "The MIT License (MIT)\n\nCopyright (c) 2013 Jacek Becela jacek.becela@gmail.com\n\nPermission is hereby granted, free of cha"
  },
  {
    "path": "README.md",
    "chars": 10227,
    "preview": "A request/response rewriting HTTP proxy. A Rack app. Subclass `Rack::Proxy` and provide your `rewrite_env` and `rewrite_"
  },
  {
    "path": "Rakefile",
    "chars": 259,
    "preview": "require 'rubygems'\nrequire 'bundler'\nBundler::GemHelper.install_tasks\n\nrequire \"rake/testtask\"\ntask :test do\n  Rake::Tes"
  },
  {
    "path": "lib/net_http_hacked.rb",
    "chars": 2461,
    "preview": "# We are hacking net/http to change semantics of streaming handling\n# from \"block\" semantics to regular \"return\" semanti"
  },
  {
    "path": "lib/rack/http_streaming_response.rb",
    "chars": 1754,
    "preview": "require \"net_http_hacked\"\nrequire \"stringio\"\n\nmodule Rack\n  # Wraps the hacked net/http in a Rack way.\n  class HttpStrea"
  },
  {
    "path": "lib/rack/proxy.rb",
    "chars": 5468,
    "preview": "require \"net_http_hacked\"\nrequire \"rack/http_streaming_response\"\n\nmodule Rack\n\n  # Subclass and bring your own #rewrite_"
  },
  {
    "path": "lib/rack-proxy.rb",
    "chars": 20,
    "preview": "require \"rack/proxy\""
  },
  {
    "path": "lib/rack_proxy_examples/example_service_proxy.rb",
    "chars": 1513,
    "preview": "###\n# This is an example of how to use Rack-Proxy in a Rails application.\n#\n# Setup:\n# 1. rails new test_app\n# 2. cd tes"
  },
  {
    "path": "lib/rack_proxy_examples/forward_host.rb",
    "chars": 614,
    "preview": "class ForwardHost < Rack::Proxy\n\n  def rewrite_env(env)\n    env[\"HTTP_HOST\"] = \"example.com\"\n    env\n  end\n\n  def rewrit"
  },
  {
    "path": "lib/rack_proxy_examples/rack_php_proxy.rb",
    "chars": 1104,
    "preview": "###\n# Open http://localhost:3000/test.php to trigger proxy\n###\nclass RackPhpProxy < Rack::Proxy\n\n  def perform_request(e"
  },
  {
    "path": "lib/rack_proxy_examples/trusting_proxy.rb",
    "chars": 652,
    "preview": "class TrustingProxy < Rack::Proxy\n\n  def rewrite_env(env)\n    env[\"HTTP_HOST\"] = \"self-signed.badssl.com\"\n\n    # We are "
  },
  {
    "path": "rack-proxy.gemspec",
    "chars": 963,
    "preview": "# -*- encoding: utf-8 -*-\n$:.push File.expand_path(\"../lib\", __FILE__)\nrequire \"rack-proxy\"\n\nGem::Specification.new do |"
  },
  {
    "path": "test/http_streaming_response_test.rb",
    "chars": 1124,
    "preview": "require \"test_helper\"\nrequire \"rack/http_streaming_response\"\n\nclass HttpStreamingResponseTest < Test::Unit::TestCase\n\n  "
  },
  {
    "path": "test/net_http_hacked_test.rb",
    "chars": 730,
    "preview": "require \"test_helper\"\nrequire \"net_http_hacked\"\n\nclass NetHttpHackedTest < Test::Unit::TestCase\n  \n  def test_net_http_h"
  },
  {
    "path": "test/rack_proxy_test.rb",
    "chars": 3748,
    "preview": "require \"test_helper\"\nrequire \"rack/proxy\"\n\nclass RackProxyTest < Test::Unit::TestCase\n  class HostProxy < Rack::Proxy\n "
  },
  {
    "path": "test/test_helper.rb",
    "chars": 197,
    "preview": "require \"rubygems\"\nrequire 'bundler/setup'\nrequire 'bundler/gem_tasks'\nrequire \"test/unit\"\n\nrequire \"rack\"\nrequire \"rack"
  }
]

About this extraction

This page contains the full source code of the ncr/rack-proxy GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 20 files (31.7 KB), approximately 8.8k tokens, and a symbol index with 64 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.

Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.

Copied to clipboard!