Repository: gliderlabs/connectable
Branch: master
Commit: 8ee985fbfac4
Files: 17
Total size: 17.5 KB

Directory structure:
gitextract_64rlitdy/

├── .dockerignore
├── .gitignore
├── Dockerfile
├── Dockerfile.dev
├── LICENSE
├── Makefile
├── README.md
├── SPONSORS
├── VERSION
├── connectable.go
├── pkg/
│   └── lookup/
│       ├── cache.go
│       ├── consulkv/
│       │   └── consulkv.go
│       ├── dns/
│       │   └── dns.go
│       ├── etcd/
│       │   └── etcd.go
│       ├── lookup.go
│       └── ports.go
└── run

================================================
FILE CONTENTS
================================================

================================================
FILE: .dockerignore
================================================
.git


================================================
FILE: .gitignore
================================================
release


================================================
FILE: Dockerfile
================================================
FROM alpine:3.2
ENTRYPOINT ["/bin/connectable"]
COPY . /go/src/github.com/gliderlabs/connectable
RUN apk add --update go git mercurial iptables \
  && cd /go/src/github.com/gliderlabs/connectable \
  && export GOPATH=/go \
  && go get \
  && go build -ldflags "-X main.Version $(cat VERSION)" -o /bin/connectable \
  && apk del go git mercurial \
  && rm -rf /go /var/cache/apk/*


================================================
FILE: Dockerfile.dev
================================================
FROM gliderlabs/alpine:3.2
RUN apk-install go git mercurial iptables


================================================
FILE: LICENSE
================================================
The MIT License (MIT)

Copyright (c) 2015 Glider Labs

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.


================================================
FILE: Makefile
================================================
NAME=connectable
REPO=gliderlabs
VERSION=$(shell cat VERSION)

dev:
	@docker history $(NAME):dev &> /dev/null \
		|| docker build -f Dockerfile.dev -t $(NAME):dev .
	@docker run --rm --name $(NAME)-dev \
		-v /var/run/docker.sock:/var/run/docker.sock \
		-v $(PWD):/go/src/github.com/$(REPO)/$(NAME) \
		$(NAME):dev


================================================
FILE: README.md
================================================
# Connectable

[![Docker Hub](https://img.shields.io/badge/docker-ready-blue.svg)](https://registry.hub.docker.com/u/gliderlabs/connectable/)
[![IRC Channel](https://img.shields.io/badge/irc-%23gliderlabs-blue.svg)](https://kiwiirc.com/client/irc.freenode.net/#gliderlabs)

A smart Docker proxy that lets your containers connect to other containers via service
discovery *without being service discovery aware*.

## Getting Connectable

You can get the Connectable micro container from the Docker Hub.

	$ docker pull gliderlabs/connectable

## Using Connectable

Basic overview is:

 1. Run a service registry like Consul, perhaps with Registrator
 1. Start a Connectable container on each host
 1. Expose Connectable to your containers, using links or Resolvable (experimental)
 1. Run containers with labels defining what they need to connect to with what ports
 1. Have software in those containers connect via those ports on localhost

#### Starting Connectable

Once you have a service registry set up, you point Connectable to it when you launch it.
You also need to mount the Docker socket. Here is an example using the local Consul agent, assuming you're running Resolvable:

	$ docker run -d --name connectable \
			-v /var/run/docker.sock:/var/run/docker.sock \
			gliderlabs/connectable:latest

With Resolvable running, it will have access to Consul DNS. It will be able to resolve any connections using DNS names.

#### Start containers that use Connectable

All you have to do is specify a port to use and what you'd like to connect to as a label. For example:

	connect.6000=redis.service.consul

With this label set, you can connect to Redis on localhost:6000. You can also specify multiple services:

	$ docker run -d --name myservice \
			-l connect.6000=redis.service.consul \
			-l connect.3306=master.mysql.service.consul \
			example/myservice

## Load Balancing

Connectable acts as a load balancer across instance of services it finds. It shuffles them randomly on new connections. Although this seems less predictable, it ensures even balancing cluster-wide.

Connectable is a reverse proxy and balancer, but it is not recommended to be used as your public facing balancer. Instead, use a more configurable balancer like haproxy or Nginx. Use Connectable for internal service-to-service connections. For example, you could use Connectable *with* Nginx to simplify your Nginx container setup.

## Health Checking

Currently Connectable does not have native health checking integration. For now, Connectable defers to the registry to return healthy services. For example, this is how Consul DNS works. Otherwise, when Connectable tries to connect to an endpoint and is unable to connect, it will try the next one transparently until all services have been tried. This covers some but not all "unhealthy" service cases.

Future modules may add support for integration with health checking mechanisms.

## Overhead

Like all proxies, you incur overhead to your connections. Connectable is roughly comparable but slightly slower than Nginx. Not by much. Here is some data collected using HTTP requests via ApacheBench using `-n 200 -c 20`:
```
nginx:

    Requests per second:    754.53 [#/sec] (mean)
    Time per request:       26.507 [ms] (mean)
    Time per request:       1.325 [ms] (mean, across all concurrent requests)

connectable:

    Requests per second:    606.32 [#/sec] (mean)
    Time per request:       32.986 [ms] (mean)
    Time per request:       1.649 [ms] (mean, across all concurrent requests)
```
Memory overhead is also roughly comparable per connection. Added network latency is near zero since it's running on the same host as clients. Keep in mind, Connectable is designed to run on each host for best performance and to avoid SPOF.

Although Connectable is Good Enough for most cases, if the overhead is a deal breaker for a particular case, don't use it in that particular case. Alternatives include working with service registries directly, just using DNS discovery with known ports, setting up a full SDN, etc.

## Modules

Todo

## Why not just DNS?

If you're using Consul DNS, SkyDNS, et al, you may wonder why Connectable is necessary. The answer is ports. Most software is not designed for dynamic ports. Most software can only resolve hostnames to IPs. You have to hard configure the port used.

If you are able to run all containers publishing exposed ports on known ports (`-p 80:80`), you might not need Connectable. If you have a fancy SDN solution that makes private container IPs publicly addressable and they use known ports, you don't need Connectable.

However, if you run containers with non-conventional ports, or don't have control over published ports, or just want to not care and wish it were magically taken care of ... that's what Connectable is for.

Connectable when combined with Registrator lets you run containers with `-P` and not care about what port they publish as.

Also, DNS may not randomize results, effectively balancing services. Connectable ensures internal load balancing.

## Notes

https://github.com/docker/docker/issues/7468
https://github.com/docker/docker/issues/7467

## Sponsor and Thanks

Connectable is sponsored by [Weave](http://weave.works). The original ambassadord proof of concept was made possible thanks to [DigitalOcean](http://digitalocean.com). Also thanks to [Jérôme Petazzoni](https://github.com/jpetazzo) for helping with the iptables bits that make this magical.

## License

MIT
<img src="https://ga-beacon.appspot.com/UA-58928488-2/connectable/readme?pixel" />


================================================
FILE: SPONSORS
================================================
DigitalOcean 	http://digitalocean.com
Weave         http://weave.works


================================================
FILE: VERSION
================================================
0.1.0


================================================
FILE: connectable.go
================================================
package main

import (
	"fmt"
	"io"
	"log"
	"net"
	"os"
	"regexp"
	"strconv"
	"strings"
	"syscall"

	env "github.com/MattAitchison/envconfig"
	"github.com/fsouza/go-dockerclient"
	"github.com/gliderlabs/connectable/pkg/lookup"

	_ "github.com/gliderlabs/connectable/pkg/lookup/dns"
)

var Version string

var (
	endpoint = env.String("docker_host", "unix:///var/run/docker.sock", "docker endpoint")
	port     = env.String("port", "10000", "primary listen port")

	self *docker.Container
)

func assert(err error) {
	if err != nil {
		log.Fatal(err)
	}
}

func runNetCmd(container, image string, cmd string) error {
	client, err := docker.NewClient(endpoint)
	if err != nil {
		return err
	}
	c, err := client.CreateContainer(docker.CreateContainerOptions{
		Config: &docker.Config{
			Image:      image,
			Cmd:        []string{cmd},
			Entrypoint: []string{"/bin/sh", "-c"},
		},
		HostConfig: &docker.HostConfig{
			Privileged:  true,
			NetworkMode: fmt.Sprintf("container:%s", container),
		},
	})
	if err != nil {
		return err
	}
	if err := client.StartContainer(c.ID, nil); err != nil {
		return err
	}
	status, err := client.WaitContainer(c.ID)
	if err != nil {
		return err
	}
	if status != 0 {
		return fmt.Errorf("netcmd non-zero exit: %v", status)
	}
	return client.RemoveContainer(docker.RemoveContainerOptions{
		ID:    c.ID,
		Force: true,
	})
}

func originalDestinationPort(conn net.Conn) (string, error) {
	f, err := conn.(*net.TCPConn).File()
	if err != nil {
		return "", err
	}
	defer f.Close()
	addr, err := syscall.GetsockoptIPv6Mreq(
		int(f.Fd()), syscall.IPPROTO_IP, 80) // 80 = SO_ORIGINAL_DST
	if err != nil {
		return "", err
	}
	port := uint16(addr.Multiaddr[2])<<8 + uint16(addr.Multiaddr[3])
	return strconv.Itoa(int(port)), nil
}

func inspectBackend(sourceIP, destPort string) (string, error) {
	client, err := docker.NewClient(endpoint)
	if err != nil {
		return "", err
	}
	label := fmt.Sprintf("connect.%s", destPort)

	// todo: cache, invalidate with container destroy events
	containers, err := client.ListContainers(docker.ListContainersOptions{})
	if err != nil {
		return "", err
	}
	for _, listing := range containers {
		container, err := client.InspectContainer(listing.ID)
		if err != nil {
			return "", err
		}
		if container.NetworkSettings.IPAddress == sourceIP {
			backend, ok := container.Config.Labels[label]
			if !ok {
				return "", fmt.Errorf("connect label '%s' not found: %v", label, container.Config.Labels)
			}
			return backend, nil
		}
	}
	return "", fmt.Errorf("unable to find container with source IP")
}

func lookupBackend(conn net.Conn) string {
	sourceIP, _, _ := net.SplitHostPort(conn.RemoteAddr().String())
	destPort, err := originalDestinationPort(conn)
	if err != nil {
		log.Println("unable to determine destination port")
		return ""
	}

	backend, err := inspectBackend(sourceIP, destPort)
	if err != nil {
		log.Println(err)
		return ""
	}
	return backend
}

func proxyConn(conn net.Conn, addr string) {
	backend, err := net.Dial("tcp", addr)
	defer conn.Close()
	if err != nil {
		log.Println("proxy", err.Error())
		return
	}
	defer backend.Close()

	done := make(chan struct{})
	go func() {
		io.Copy(backend, conn)
		backend.(*net.TCPConn).CloseWrite()
		close(done)
	}()
	io.Copy(conn, backend)
	conn.(*net.TCPConn).CloseWrite()
	<-done
}

func setupContainer(id string) error {
	re := regexp.MustCompile("connect\\.(\\d+)")
	client, err := docker.NewClient(endpoint)
	if err != nil {
		log.Println(err)
		return err
	}
	container, err := client.InspectContainer(id)
	if err != nil {
		log.Println(err)
		return err
	}
	if container.HostConfig.NetworkMode == "bridge" || container.HostConfig.NetworkMode == "default" {
		hasBackends := false
		cmds := []string{
			"/sbin/sysctl -w net.ipv4.conf.all.route_localnet=1",
			"iptables -t nat -I POSTROUTING 1 -m addrtype --src-type LOCAL --dst-type UNICAST -j MASQUERADE",
		}
		for k, _ := range container.Config.Labels {
			results := re.FindStringSubmatch(k)
			if len(results) > 1 {
				hasBackends = true
				cmds = append(cmds, fmt.Sprintf(
					"iptables -t nat -I OUTPUT 1 -m addrtype --src-type LOCAL --dst-type LOCAL -p tcp --dport %s -j DNAT --to-destination %s:%s",
					results[1], self.NetworkSettings.IPAddress, results[1]))
			}
		}
		if hasBackends {
			log.Printf("setting iptables on %s \n", container.ID[:12])
			shellCmd := strings.Join(cmds, " && ")
			err := runNetCmd(container.ID, self.Image, shellCmd)
			if err != nil {
				log.Printf("error setting iptables on %s: %s \n", container.ID[:12], err)
				return err
			}
		}
	}
	return nil

}

func monitorContainers() {
	client, err := docker.NewClient(endpoint)
	assert(err)
	events := make(chan *docker.APIEvents)
	assert(client.AddEventListener(events))
	list, _ := client.ListContainers(docker.ListContainersOptions{})
	for _, listing := range list {
		go setupContainer(listing.ID)
	}
	for msg := range events {
		switch msg.Status {
		case "start":
			go setupContainer(msg.ID)
		}
	}
}

func main() {
	listener, err := net.Listen("tcp", ":"+port)
	assert(err)

	fmt.Printf("# Connectable %s listening on %s ...\n", Version, port)

	client, err := docker.NewClient(endpoint)
	assert(err)

	selfImageRe := regexp.MustCompile("(?:^|/)connectable(?:$|:)")

	list, err := client.ListContainers(docker.ListContainersOptions{})
	assert(err)
	for _, listing := range list {
		c, err := client.InspectContainer(listing.ID)
		assert(err)
		if c.Config.Hostname == os.Getenv("HOSTNAME") && selfImageRe.FindString(c.Config.Image) != "" {
			self = c
			if c.HostConfig.NetworkMode == "bridge" || c.HostConfig.NetworkMode == "default" {
				fmt.Printf("# Setting iptables on connectable... ")
				shellCmd := fmt.Sprintf("iptables -t nat -A PREROUTING -p tcp -j REDIRECT --to-ports %s", port)
				assert(runNetCmd(c.ID, c.Image, shellCmd))
				fmt.Printf("done.\n")
			}
		}
	}

	if self == nil {
		fmt.Println("# unable to find self")
		os.Exit(1)
	}

	go monitorContainers()

	for {
		conn, err := listener.Accept()
		if err != nil {
			log.Fatal(err)
		}

		backend := lookupBackend(conn)
		if backend == "" {
			conn.Close()
			continue
		}

		backendAddrs, err := lookup.Resolve(backend)
		if err != nil {
			log.Println(err)
			conn.Close()
			continue
		}
		if len(backendAddrs) == 0 {
			log.Println(conn.RemoteAddr(), backend, "no backends")
			conn.Close()
			continue
		}

		log.Println(conn.RemoteAddr(), backend, "->", backendAddrs[0])
		go proxyConn(conn, backendAddrs[0])
	}
}


================================================
FILE: pkg/lookup/cache.go
================================================
package lookup

import (
	"time"

	"github.com/youtube/vitess/go/cache"
)

const (
	cacheCapacity = 1024 * 1024 // 1MB
	cacheTTL      = 1           // 1 second
)

var (
	resolveCache = cache.NewLRUCache(cacheCapacity)
)

type cacheValue struct {
	Value     []string
	CreatedAt int64
}

func (cv *cacheValue) Size() int {
	var size int
	for _, s := range cv.Value {
		size += len(s)
	}
	return size
}

func (cv *cacheValue) Expired() bool {
	return (time.Now().Unix() - cv.CreatedAt) > cacheTTL
}


================================================
FILE: pkg/lookup/consulkv/consulkv.go
================================================
package consulkv

// TODO


================================================
FILE: pkg/lookup/dns/dns.go
================================================
package dns

import (
	"log"
	"net"
	"strconv"

	"github.com/gliderlabs/connectable/pkg/lookup"
	"github.com/miekg/dns"
)

var (
	config *dns.ClientConfig
)

func init() {
	lookup.Register("dns", new(dnsResolver))
	var err error
	config, err = dns.ClientConfigFromFile("/etc/resolv.conf")
	if err != nil {
		log.Fatal(err)
	}
}

type dnsResolver struct{}

func (r *dnsResolver) Lookup(addr string) ([]string, error) {
	query := new(dns.Msg)
	query.SetQuestion(dns.Fqdn(addr), dns.TypeSRV)
	query.RecursionDesired = false
	client := new(dns.Client)
	resp, _, err := client.Exchange(query, net.JoinHostPort(config.Servers[0], config.Port))
	if err != nil {
		return nil, err
	}
	if len(resp.Answer) == 0 {
		return []string{}, nil
	}
	var addrs []string
	for i, record := range resp.Answer {
		port := strconv.Itoa(int(record.(*dns.SRV).Port))
		ip := record.(*dns.SRV).Target
		if len(resp.Extra) >= i+1 {
			ip = resp.Extra[i].(*dns.A).A.String()
		}
		addrs = append(addrs, net.JoinHostPort(ip, port))
	}
	return addrs, nil
}


================================================
FILE: pkg/lookup/etcd/etcd.go
================================================
package etcd

// TODO


================================================
FILE: pkg/lookup/lookup.go
================================================
package lookup

import (
	"fmt"
	"log"
	"time"

	env "github.com/MattAitchison/envconfig"
)

var (
	resolverName = env.String("lookup_resolver", "dns", "resolver to use for lookups")
	debugMode    = env.Bool("lookup_debug", false, "enable debug output")
	resolvers    = make(map[string]Resolver)
)

func debug(v ...interface{}) {
	if debugMode {
		log.Println(v...)
	}
}

type Resolver interface {
	Lookup(addr string) ([]string, error)
}

func Register(name string, resolver Resolver) {
	resolvers[name] = resolver
}

func Resolve(addr string) ([]string, error) {
	cached, ok := resolveCache.Get(addr)
	if ok && !cached.(*cacheValue).Expired() {
		debug("lookup: resolving [cache]:", addr, cached.(*cacheValue).Value)
		return cached.(*cacheValue).Value, nil
	}
	resolver, ok := resolvers[resolverName]
	if !ok {
		debug("lookup: resolver not found:", resolverName)
		return []string{}, fmt.Errorf("resolver not found: %s", resolverName)
	}
	value, err := resolver.Lookup(addr)
	if err != nil {
		return nil, err
	}
	resolveCache.Set(addr, &cacheValue{value, time.Now().Unix()})
	debug("lookup: resolving:", addr, value)
	return value, nil
}


================================================
FILE: pkg/lookup/ports.go
================================================
package lookup

var namedPorts = map[string]string{
	"syslog":      "514",
	"http":        "80",
	"https":       "443",
	"ssh":         "22",
	"consul":      "8500",
	"consul-http": "8500",
	"etcd":        "2379",
	"dns":         "53",
}


================================================
FILE: run
================================================
#!/bin/sh
set -e

: "${DEV_IMAGE_TAG:=connectable:dev-env}"
: "${DEV_CONTAINER_NAME:=connectable-dev}"

exists() { # type name
  docker inspect --type "$1" -f '{{.Id}}' "$2" >/dev/null 2>&1
}

if ! exists image "$DEV_IMAGE_TAG"; then
  printf '\n==> %s\n\n' 'Building base image...'
  docker build -t "$DEV_IMAGE_TAG" -f Dockerfile.dev .
fi

if ! exists container "$DEV_CONTAINER_NAME"; then
  docker create >/dev/null -it --name "$DEV_CONTAINER_NAME" \
    -h dev \
    -e GOPATH=/go \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v "$(pwd)":/go/src/github.com/gliderlabs/connectable \
    -w /go/src/github.com/gliderlabs/connectable \
    "$DEV_IMAGE_TAG" sh -c '
      run() { cmd=$1; shift; printf "\\n==> %s\\n\\n" "$cmd $*"; "$cmd" "$@"; }
      set -e
      run go get
      run go build -ldflags "-X main.Version dev" -o /bin/connectable
      run exec /bin/connectable'
fi

exec docker start -ia "$DEV_CONTAINER_NAME"