[ { "path": ".github/FUNDING.yml", "content": "# These are supported funding model platforms\n\ngithub: progrium\n" }, { "path": ".gitignore", "content": "TODO\ntypescript/dist \ndemos/groktunnel/groktunnel" }, { "path": ".vscode/extensions.json", "content": "{\n \"recommendations\": [\n \"denoland.vscode-deno\",\n \"golang.go\"\n ]\n}\n" }, { "path": ".vscode/settings.json", "content": "{\n \"deno.enable\": true,\n \"deno.lint\": true,\n \"deno.unstable\": true\n}" }, { "path": "LICENSE", "content": "MIT License\n\nCopyright (c) 2021 Jeff Lindsay\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n" }, { "path": "README.md", "content": "# qmux\n\nqmux is a wire protocol for multiplexing connections or streams into a single connection. It is based on the [SSH Connection Protocol](https://tools.ietf.org/html/rfc4254#page-5), which is the simplest, longest running, most widely deployed TCP multiplexing protocol with flow control. \n\nIt is meant as a drop-in layer for any stream capable transport (TCP, WebSocket, stdio, etc) to provide basic multiplexing. This brings any connection API to rough semantic parity with [QUIC](https://en.wikipedia.org/wiki/QUIC) multiplexing, so it can act as a stopgap or fallback when QUIC is not available. You can then design higher level protocols based on multiplexing semantics that sit on top of QUIC or any other streaming transport with qmux.\n\n## Spec\n\nThe specification is [here](https://github.com/progrium/qmux/blob/main/SPEC.md). It is a simplified version of the [Channel Mechanism](https://tools.ietf.org/html/rfc4254#page-5) in the SSH Connection Protocol.\n\n## Implementations\n\n- [x] [Golang](https://github.com/progrium/qmux/tree/main/golang) (best reference)\n- [x] [TypeScript](https://github.com/progrium/qmux/tree/main/typescript)\n- [ ] Python\n- [ ] C# (help wanted)\n\n## Demos\n\n- [groktunnel](https://github.com/progrium/qmux/tree/main/demos/groktunnel): Ephemeral localhost public forwarding system for HTTP similar to Ngrok in less than 150 lines of Go.\n\n## About\n\nLicensed MIT\n" }, { "path": "SPEC.md", "content": "# qmux\n\nqmux is a wire protocol for multiplexing connections or streams into a single connection.\nIt is a subset of the [SSH Connection Protocol](https://tools.ietf.org/html/rfc4254#page-5).\n\nFeatures removed to simplify include channel types, channel requests, and \"extended data\"\nmessages that were used for STDERR data.\n\n## Channels\n\n Either side may open a channel. Multiple channels are multiplexed\n into a single connection.\n\n Channels are identified by numbers at each end. The number referring\n to a channel may be different on each side. Requests to open a\n channel contain the sender's channel number. Any other channel-\n related messages contain the recipient's channel number for the\n channel.\n\n Channels are flow-controlled. No data may be sent to a channel until\n a message is received to indicate that window space is available.\n\n### Opening a Channel\n\n When either side wishes to open a new channel, it allocates a local\n number for the channel. It then sends the following message to the\n other side, and includes the local channel number and initial window\n size in the message.\n\n byte QMUX_MSG_CHANNEL_OPEN\n uint32 sender channel\n uint32 initial window size\n uint32 maximum packet size\n\n The 'sender channel' is a local identifier for the channel used by the\n sender of this message. The 'initial window size' specifies how many\n bytes of channel data can be sent to the sender of this message\n without adjusting the window. The 'maximum packet size' specifies the\n maximum size of an individual data packet that can be sent to the\n sender. For example, one might want to use smaller packets for\n interactive connections to get better interactive response on slow\n links.\n\n The remote side then decides whether it can open the channel, and\n responds with either `QMUX_MSG_CHANNEL_OPEN_CONFIRMATION` or\n `QMUX_MSG_CHANNEL_OPEN_FAILURE`.\n\n byte QMUX_MSG_CHANNEL_OPEN_CONFIRMATION\n uint32 recipient channel\n uint32 sender channel\n uint32 initial window size\n uint32 maximum packet size\n\n The 'recipient channel' is the channel number given in the original\n open request, and 'sender channel' is the channel number allocated by\n the other side.\n\n byte QMUX_MSG_CHANNEL_OPEN_FAILURE\n uint32 recipient channel\n\n### Data Transfer\n\n The window size specifies how many bytes the other party can send\n before it must wait for the window to be adjusted. Both parties use\n the following message to adjust the window.\n\n byte QMUX_MSG_CHANNEL_WINDOW_ADJUST\n uint32 recipient channel\n uint32 bytes to add\n\n After receiving this message, the recipient MAY send the given number\n of bytes more than it was previously allowed to send; the window size\n is incremented. Implementations MUST correctly handle window sizes\n of up to 2^32 - 1 bytes. The window MUST NOT be increased above\n 2^32 - 1 bytes.\n\n Data transfer is done with messages of the following type.\n\n byte QMUX_MSG_CHANNEL_DATA\n uint32 recipient channel\n string data\n\n The maximum amount of data allowed is determined by the maximum\n packet size for the channel, and the current window size, whichever\n is smaller. The window size is decremented by the amount of data\n sent. Both parties MAY ignore all extra data sent after the allowed\n window is empty.\n\n Implementations are expected to have some limit on the transport\n layer packet size.\n\n### Closing a Channel\n\n When a party will no longer send more data to a channel, it SHOULD\n send `QMUX_MSG_CHANNEL_EOF`.\n\n byte QMUX_MSG_CHANNEL_EOF\n uint32 recipient channel\n\n No explicit response is sent to this message. However, the\n application may send EOF to whatever is at the other end of the\n channel. Note that the channel remains open after this message, and\n more data may still be sent in the other direction. This message\n does not consume window space and can be sent even if no window space\n is available.\n\n When either party wishes to terminate the channel, it sends\n `QMUX_MSG_CHANNEL_CLOSE`. Upon receiving this message, a party MUST\n send back an `QMUX_MSG_CHANNEL_CLOSE` unless it has already sent this\n message for the channel. The channel is considered closed for a\n party when it has both sent and received `QMUX_MSG_CHANNEL_CLOSE`, and\n the party may then reuse the channel number. A party MAY send\n `QMUX_MSG_CHANNEL_CLOSE` without having sent or received\n `QMUX_MSG_CHANNEL_EOF`.\n\n byte QMUX_MSG_CHANNEL_CLOSE\n uint32 recipient channel\n\n This message does not consume window space and can be sent even if no\n window space is available.\n\n It is RECOMMENDED that all data sent before this message be delivered\n to the actual destination, if possible.\n\n## Summary of Message Numbers\n\n The following is a summary of messages and their associated message\n number byte value.\n\n QMUX_MSG_CHANNEL_OPEN 100\n QMUX_MSG_CHANNEL_OPEN_CONFIRMATION 101\n QMUX_MSG_CHANNEL_OPEN_FAILURE 102\n QMUX_MSG_CHANNEL_WINDOW_ADJUST 103\n QMUX_MSG_CHANNEL_DATA 104\n QMUX_MSG_CHANNEL_EOF 105\n QMUX_MSG_CHANNEL_CLOSE 106\n\n## Data Type Representations Used\n\n byte\n\n A byte represents an arbitrary 8-bit value (octet). Fixed length\n data is sometimes represented as an array of bytes, written\n byte[n], where n is the number of bytes in the array.\n\n uint32\n\n Represents a 32-bit unsigned integer. Stored as four bytes in the\n order of decreasing significance (network byte order). For\n example: the value 699921578 (0x29b7f4aa) is stored as 29 b7 f4\n aa.\n\n string\n\n Arbitrary length binary string. Strings are allowed to contain\n arbitrary binary data, including null characters and 8-bit\n characters. They are stored as a uint32 containing its length\n (number of bytes that follow) and zero (= empty string) or more\n bytes that are the value of the string. Terminating null\n characters are not used." }, { "path": "demos/groktunnel/README.md", "content": "# groktunnel\n\nExpose localhost HTTP servers with a public URL\n\n## Build\n```\n$ go build\n```\n\n## Try it out\n\nFirst we run the groktunnel server. Normally this would be run on a server, but by default uses `vcap.me`\nfor a hostname which resolves all of its subdomains to localhost.\n```\n$ ./groktunnel\n2021/04/29 16:10:35 groktunnel server [vcap.me] ready!\n```\n\nNow run a local web server. Here is how to run a server listing a file directory with Python:\n```\n$ python -m SimpleHTTPServer\nServing HTTP on 0.0.0.0 port 8000 ...\n```\n\nThen we run groktunnel as a client by giving it the local port to expose.\n```\n$ ./groktunnel 8000\nport 8000 http available at:\nhttp://y8eyshnpol.vcap.me:9999\n```\n\nThat address should serve the same content as the local web server on 8000. For added effect,\nrun both client and server with `-p 80`, which will require root to run the server.\n\n## About\n\nThis uses qmux between the client and server to tunnel subdomain requests down to the client.\nThis is done over a hijacked http connection after a tunnel is established and a new subdomain\nvhost is setup. \n\nNot counting dependencies, this whole system is done in under 150 lines. This is the 5th or 6th\nimplementation of this system since the original [localtunnel](https://github.com/progrium/localtunnel)\nin 2010, which was then cloned many times. It was then commercialized by [Ngrok](https://ngrok.com/)." }, { "path": "demos/groktunnel/go.mod", "content": "module github.com/progrium/qmux/demos/groktunnel\n\ngo 1.16\n\nrequire (\n\tgithub.com/inconshreveable/go-vhost v0.0.0-20160627193104-06d84117953b\n\tgithub.com/progrium/qmux/golang v0.0.0-20210428195120-05a36e97c488\n)\n" }, { "path": "demos/groktunnel/go.sum", "content": "github.com/inconshreveable/go-vhost v0.0.0-20160627193104-06d84117953b h1:IpLPmn6Re21F0MaV6Zsc5RdSE6KuoFpWmHiUSEs3PrE=\ngithub.com/inconshreveable/go-vhost v0.0.0-20160627193104-06d84117953b/go.mod h1:aA6DnFhALT3zH0y+A39we+zbrdMC2N0X/q21e6FI0LU=\ngithub.com/progrium/qmux/golang v0.0.0-20210428195120-05a36e97c488 h1:wG/GkKO0L/98gyqxx0Jm8CyVf/bPW3XUY8abCsFc5Yw=\ngithub.com/progrium/qmux/golang v0.0.0-20210428195120-05a36e97c488/go.mod h1:Z2EPtydgPrcZxO50GhkzTGgdWjA5PPHsZkoq6KTxPVE=\ngolang.org/x/net v0.0.0-20210420210106-798c2154c571/go.mod h1:72T/g9IO56b78aLF+1Kcs5dz7/ng1VjMUvfKvpfy+jM=\ngolang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20210420072515-93ed5bcd2bfe/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=\ngolang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=\ngolang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=\n" }, { "path": "demos/groktunnel/main.go", "content": "package main\n\nimport (\n\t\"bufio\"\n\t\"context\"\n\t\"crypto/rand\"\n\t\"flag\"\n\t\"fmt\"\n\t\"io\"\n\t\"log\"\n\t\"net\"\n\t\"net/http\"\n\t\"net/http/httputil\"\n\t\"strings\"\n\t\"time\"\n\n\tvhost \"github.com/inconshreveable/go-vhost\"\n\t\"github.com/progrium/qmux/golang/session\"\n)\n\nfunc main() {\n\tvar port = flag.String(\"p\", \"9999\", \"server port to use\")\n\tvar host = flag.String(\"h\", \"vcap.me\", \"server hostname to use\")\n\tvar addr = flag.String(\"b\", \"127.0.0.1\", \"ip to bind [server only]\")\n\tflag.Parse()\n\n\t// client usage: groktunnel [-h=] \n\tif flag.Arg(0) != \"\" {\n\t\tconn, err := net.Dial(\"tcp\", net.JoinHostPort(*host, *port))\n\t\tfatal(err)\n\t\tclient := httputil.NewClientConn(conn, bufio.NewReader(conn))\n\t\treq, err := http.NewRequest(\"GET\", \"/\", nil)\n\t\treq.Host = net.JoinHostPort(*host, *port)\n\t\tfatal(err)\n\t\tclient.Write(req)\n\t\tresp, _ := client.Read(req)\n\t\tfmt.Printf(\"port %s http available at:\\n\", flag.Arg(0))\n\t\tfmt.Printf(\"http://%s\\n\", resp.Header.Get(\"X-Public-Host\"))\n\t\tc, _ := client.Hijack()\n\t\tsess := session.New(c)\n\t\tdefer sess.Close()\n\t\tfor {\n\t\t\tch, err := sess.Accept()\n\t\t\tfatal(err)\n\t\t\tconn, err := net.Dial(\"tcp\", \"127.0.0.1:\"+flag.Arg(0))\n\t\t\tfatal(err)\n\t\t\tgo join(conn, ch)\n\t\t}\n\t\treturn\n\t}\n\n\t// server usage: groktunnel [-h=] [-b=]\n\tl, err := net.Listen(\"tcp\", net.JoinHostPort(*addr, *port))\n\tfatal(err)\n\tdefer l.Close()\n\tvmux, err := vhost.NewHTTPMuxer(l, 1*time.Second)\n\tfatal(err)\n\n\tgo serve(vmux, *host, *port)\n\n\tlog.Printf(\"groktunnel server [%s] ready!\\n\", *host)\n\tfor {\n\t\tconn, err := vmux.NextError()\n\t\tfmt.Println(err)\n\t\tif conn != nil {\n\t\t\tconn.Close()\n\t\t}\n\t}\n}\n\nfunc serve(vmux *vhost.HTTPMuxer, host, port string) {\n\tml, err := vmux.Listen(net.JoinHostPort(host, port))\n\tfatal(err)\n\tsrv := &http.Server{Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tpublicHost := strings.TrimSuffix(net.JoinHostPort(newSubdomain()+host, port), \":80\")\n\t\tpl, err := vmux.Listen(publicHost)\n\t\tfatal(err)\n\t\tw.Header().Add(\"X-Public-Host\", publicHost)\n\t\tw.Header().Add(\"Connection\", \"close\")\n\t\tw.WriteHeader(http.StatusOK)\n\t\tconn, _, _ := w.(http.Hijacker).Hijack()\n\t\tsess := session.New(conn)\n\t\tdefer sess.Close()\n\t\tlog.Printf(\"%s: start session\", publicHost)\n\t\tgo func() {\n\t\t\tfor {\n\t\t\t\tconn, err := pl.Accept()\n\t\t\t\tif err != nil {\n\t\t\t\t\tlog.Println(err)\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tch, err := sess.Open(context.Background())\n\t\t\t\tif err != nil {\n\t\t\t\t\tlog.Println(err)\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tgo join(ch, conn)\n\t\t\t}\n\t\t}()\n\t\tsess.Wait()\n\t\tlog.Printf(\"%s: end session\", publicHost)\n\t})}\n\tsrv.Serve(ml)\n}\n\nfunc join(a io.ReadWriteCloser, b io.ReadWriteCloser) {\n\tgo io.Copy(b, a)\n\tio.Copy(a, b)\n\ta.Close()\n\tb.Close()\n}\n\nfunc newSubdomain() string {\n\tb := make([]byte, 10)\n\tif _, err := rand.Read(b); err != nil {\n\t\tpanic(err)\n\t}\n\tletters := []rune(\"abcdefghijklmnopqrstuvwxyz1234567890\")\n\tr := make([]rune, 10)\n\tfor i := range r {\n\t\tr[i] = letters[int(b[i])*len(letters)/256]\n\t}\n\treturn string(r) + \".\"\n}\n\nfunc fatal(err error) {\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n}\n" }, { "path": "golang/README.md", "content": "# qmux for Go\n\nAn implementation of qmux for multiplexing any reliable `io.ReadWriteCloser`.\n\n## Using qmux in Go\n\n```\ngo get github.com/progrium/qmux/golang\n```\n\nYou can create a qmux session from any `io.ReadWriteCloser` with `session.New`:\n\n```go\npackage main\n\nimport (\n \"net\"\n \"io\"\n \"context\"\n\n \"github.com/progrium/qmux/golang/session\"\n)\n\nfunc main() {\n conn, err := net.Dial(\"tcp\", \"localhost:9999\")\n if err != nil {\n panic(err)\n }\n \n sess := session.New(conn)\n defer sess.Close() // closes underlying conn\n \n ch, err := sess.Open(context.Background())\n if err != nil {\n panic(err)\n }\n defer ch.Close()\n\n io.WriteString(ch, \"Hello world\\n\")\n}\n\n```\n\nHowever it can be convenient to use the builtin transport dialers and listeners. \n\n```go\npackage main\n\nimport (\n \"io/ioutil\"\n\n \"github.com/progrium/qmux/golang/transport\"\n)\n\nfunc main() {\n t, err := transport.ListenTCP(\"localhost:9999\")\n if err != nil {\n panic(err)\n }\n defer t.Close()\n \n sess, err := t.Accept()\n if err != nil {\n panic(err)\n }\n defer sess.Close()\n \n ch, err := sess.Accept()\n if err != nil {\n panic(err)\n }\n defer ch.Close()\n \n b, err := ioutil.ReadAll(ch)\n if err != nil {\n panic(err)\n }\n os.Stdout.Write(b) // \"Hello world\\n\" if connected with earlier program\n}\n\n```" }, { "path": "golang/codec/codec.go", "content": "// Package codec implements encoding and decoding of qmux messages.\npackage codec\n\nimport \"io\"\n\nvar (\n\tDebugMessages io.Writer\n\tDebugBytes io.Writer\n)\n" }, { "path": "golang/codec/codec_test.go", "content": "package codec\n\nimport (\n\t\"bytes\"\n\t\"testing\"\n)\n\nfunc TestMarshalUnmarshal(t *testing.T) {\n\ttests := []struct {\n\t\tin Message\n\t\tout Unmarshaler\n\t}{\n\t\t{\n\t\t\tin: CloseMessage{\n\t\t\t\tChannelID: 10,\n\t\t\t},\n\t\t\tout: &CloseMessage{},\n\t\t},\n\t\t{\n\t\t\tin: DataMessage{\n\t\t\t\tChannelID: 10,\n\t\t\t\tLength: 5,\n\t\t\t\tData: []byte(\"Hello\"),\n\t\t\t},\n\t\t\tout: &DataMessage{},\n\t\t},\n\t\t{\n\t\t\tin: EOFMessage{\n\t\t\t\tChannelID: 10,\n\t\t\t},\n\t\t\tout: &EOFMessage{},\n\t\t},\n\t\t{\n\t\t\tin: OpenMessage{\n\t\t\t\tSenderID: 10,\n\t\t\t\tWindowSize: 1024,\n\t\t\t\tMaxPacketSize: 1 << 31,\n\t\t\t},\n\t\t\tout: &OpenMessage{},\n\t\t},\n\t\t{\n\t\t\tin: OpenConfirmMessage{\n\t\t\t\tChannelID: 20,\n\t\t\t\tSenderID: 10,\n\t\t\t\tWindowSize: 1024,\n\t\t\t\tMaxPacketSize: 1 << 31,\n\t\t\t},\n\t\t\tout: &OpenConfirmMessage{},\n\t\t},\n\t\t{\n\t\t\tin: OpenFailureMessage{\n\t\t\t\tChannelID: 20,\n\t\t\t},\n\t\t\tout: &OpenFailureMessage{},\n\t\t},\n\t\t{\n\t\t\tin: WindowAdjustMessage{\n\t\t\t\tChannelID: 20,\n\t\t\t\tAdditionalBytes: 1024,\n\t\t\t},\n\t\t\tout: &WindowAdjustMessage{},\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tb, err := Marshal(test.in)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif err := Unmarshal(b, test.out); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tbb, err := Marshal(test.out)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif !bytes.Equal(b, bb) {\n\t\t\tt.Fatal(\"bytes not equal\")\n\t\t}\n\t\tif test.in.String() != test.out.(Message).String() {\n\t\t\tt.Fatal(\"strings not equal\")\n\t\t}\n\t}\n\n}\n\nfunc TestEncodeDecode(t *testing.T) {\n\ttests := []struct {\n\t\tin Message\n\t\tid uint32\n\t\tok bool\n\t}{\n\t\t{\n\t\t\tin: CloseMessage{\n\t\t\t\tChannelID: 10,\n\t\t\t},\n\t\t\tid: 10,\n\t\t\tok: true,\n\t\t},\n\t\t{\n\t\t\tin: DataMessage{\n\t\t\t\tChannelID: 10,\n\t\t\t\tLength: 5,\n\t\t\t\tData: []byte(\"Hello\"),\n\t\t\t},\n\t\t\tid: 10,\n\t\t\tok: true,\n\t\t},\n\t\t{\n\t\t\tin: EOFMessage{\n\t\t\t\tChannelID: 10,\n\t\t\t},\n\t\t\tid: 10,\n\t\t\tok: true,\n\t\t},\n\t\t{\n\t\t\tin: OpenMessage{\n\t\t\t\tSenderID: 10,\n\t\t\t\tWindowSize: 1024,\n\t\t\t\tMaxPacketSize: 1 << 31,\n\t\t\t},\n\t\t\tid: 0,\n\t\t\tok: false,\n\t\t},\n\t\t{\n\t\t\tin: OpenConfirmMessage{\n\t\t\t\tChannelID: 20,\n\t\t\t\tSenderID: 10,\n\t\t\t\tWindowSize: 1024,\n\t\t\t\tMaxPacketSize: 1 << 31,\n\t\t\t},\n\t\t\tid: 20,\n\t\t\tok: true,\n\t\t},\n\t\t{\n\t\t\tin: OpenFailureMessage{\n\t\t\t\tChannelID: 20,\n\t\t\t},\n\t\t\tid: 20,\n\t\t\tok: true,\n\t\t},\n\t\t{\n\t\t\tin: WindowAdjustMessage{\n\t\t\t\tChannelID: 20,\n\t\t\t\tAdditionalBytes: 1024,\n\t\t\t},\n\t\t\tid: 20,\n\t\t\tok: true,\n\t\t},\n\t}\n\tfor _, test := range tests {\n\t\tvar buf bytes.Buffer\n\t\tenc := NewEncoder(&buf)\n\t\tif err := enc.Encode(test.in); err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tdec := NewDecoder(&buf)\n\t\tm, err := dec.Decode()\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tid, ok := m.Channel()\n\t\tif id != test.id {\n\t\t\tt.Fatal(\"id not equal\")\n\t\t}\n\t\tif ok != test.ok {\n\t\t\tt.Fatal(\"ok not equal\")\n\t\t}\n\t}\n\n}\n" }, { "path": "golang/codec/decoder.go", "content": "package codec\n\nimport (\n\t\"encoding/binary\"\n\t\"errors\"\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"sync\"\n\t\"syscall\"\n)\n\ntype Decoder struct {\n\tr io.Reader\n\tsync.Mutex\n}\n\nfunc NewDecoder(r io.Reader) *Decoder {\n\treturn &Decoder{r: r}\n}\n\nfunc (dec *Decoder) Decode() (Message, error) {\n\tdec.Lock()\n\tdefer dec.Unlock()\n\n\tpacket, err := readPacket(dec.r)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif DebugBytes != nil {\n\t\tfmt.Fprintln(DebugBytes, \">>DEC\", packet)\n\t}\n\n\treturn decode(packet)\n}\n\nfunc readPacket(c io.Reader) ([]byte, error) {\n\tmsgNum := make([]byte, 1)\n\t_, err := c.Read(msgNum)\n\tif err != nil {\n\t\tvar syscallErr *os.SyscallError\n\t\tif errors.As(err, &syscallErr) && syscallErr.Err == syscall.ECONNRESET {\n\t\t\treturn nil, io.EOF\n\t\t}\n\t\treturn nil, err\n\t}\n\n\trest := make([]byte, payloadSizes[msgNum[0]])\n\t_, err = c.Read(rest)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tpacket := append(msgNum, rest...)\n\n\tif msgNum[0] == msgChannelData {\n\t\tdataSize := binary.BigEndian.Uint32(rest[4:8])\n\t\tdata := make([]byte, dataSize)\n\t\t_, err := c.Read(data)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\n\t\tpacket = append(packet, data...)\n\t}\n\n\treturn packet, nil\n}\n\nfunc decode(packet []byte) (Message, error) {\n\tvar msg Message\n\tswitch packet[0] {\n\tcase msgChannelOpen:\n\t\tmsg = new(OpenMessage)\n\tcase msgChannelData:\n\t\tmsg = new(DataMessage)\n\tcase msgChannelOpenConfirm:\n\t\tmsg = new(OpenConfirmMessage)\n\tcase msgChannelOpenFailure:\n\t\tmsg = new(OpenFailureMessage)\n\tcase msgChannelWindowAdjust:\n\t\tmsg = new(WindowAdjustMessage)\n\tcase msgChannelEOF:\n\t\tmsg = new(EOFMessage)\n\tcase msgChannelClose:\n\t\tmsg = new(CloseMessage)\n\tdefault:\n\t\treturn nil, fmt.Errorf(\"qmux: unexpected message type %d\", packet[0])\n\t}\n\tif err := Unmarshal(packet, msg); err != nil {\n\t\treturn nil, err\n\t}\n\tif DebugMessages != nil {\n\t\tfmt.Fprintln(DebugMessages, \">>DEC\", msg)\n\t}\n\treturn msg, nil\n}\n\ntype Unmarshaler interface {\n\tUnmarshalMux([]byte) error\n}\n\nfunc Unmarshal(b []byte, v interface{}) error {\n\tu, ok := v.(Unmarshaler)\n\tif !ok {\n\t\treturn fmt.Errorf(\"qmux: unmarshal not supported for value %#v\", v)\n\t}\n\treturn u.UnmarshalMux(b)\n}\n" }, { "path": "golang/codec/encoder.go", "content": "package codec\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"sync\"\n)\n\ntype Encoder struct {\n\tw io.Writer\n\tsync.Mutex\n}\n\nfunc NewEncoder(w io.Writer) *Encoder {\n\treturn &Encoder{w: w}\n}\n\nfunc (enc *Encoder) Encode(msg interface{}) error {\n\tenc.Lock()\n\tdefer enc.Unlock()\n\n\tif DebugMessages != nil {\n\t\tfmt.Fprintln(DebugMessages, \"< 0 {\n\t\tspace := min(ch.maxRemotePayload, len(data))\n\t\tif space, err = ch.remoteWin.reserve(space); err != nil {\n\t\t\treturn n, err\n\t\t}\n\n\t\ttoSend := data[:space]\n\n\t\tif err = ch.session.enc.Encode(codec.DataMessage{\n\t\t\tChannelID: ch.remoteId,\n\t\t\tLength: uint32(len(toSend)),\n\t\t\tData: toSend,\n\t\t}); err != nil {\n\t\t\treturn n, err\n\t\t}\n\n\t\tn += len(toSend)\n\t\tdata = data[len(toSend):]\n\t}\n\n\treturn n, err\n}\n\n// Read reads up to len(data) bytes from the channel.\nfunc (c *Channel) Read(data []byte) (n int, err error) {\n\tn, err = c.pending.Read(data)\n\n\tif n > 0 {\n\t\terr = c.adjustWindow(uint32(n))\n\t\t// sendWindowAdjust can return io.EOF if the remote\n\t\t// peer has closed the connection, however we want to\n\t\t// defer forwarding io.EOF to the caller of Read until\n\t\t// the buffer has been drained.\n\t\tif n > 0 && err == io.EOF {\n\t\t\terr = nil\n\t\t}\n\t}\n\treturn n, err\n}\n\n// writePacket sends a packet. If the packet is a channel close, it updates\n// sentClose. This method takes the lock c.writeMu.\nfunc (ch *Channel) send(msg interface{}) error {\n\tch.writeMu.Lock()\n\tdefer ch.writeMu.Unlock()\n\n\tif ch.sentClose {\n\t\treturn io.EOF\n\t}\n\n\tif _, ok := msg.(codec.CloseMessage); ok {\n\t\tch.sentClose = true\n\t}\n\n\treturn ch.session.enc.Encode(msg)\n}\n\nfunc (c *Channel) adjustWindow(n uint32) error {\n\tc.windowMu.Lock()\n\t// Since myWindow is managed on our side, and can never exceed\n\t// the initial window setting, we don't worry about overflow.\n\tc.myWindow += uint32(n)\n\tc.windowMu.Unlock()\n\treturn c.send(codec.WindowAdjustMessage{\n\t\tChannelID: c.remoteId,\n\t\tAdditionalBytes: uint32(n),\n\t})\n}\n\nfunc (c *Channel) close() {\n\tc.pending.eof()\n\tclose(c.msg)\n\tc.writeMu.Lock()\n\t// This is not necessary for a normal channel teardown, but if\n\t// there was another error, it is.\n\tc.sentClose = true\n\tc.writeMu.Unlock()\n\t// Unblock writers.\n\tc.remoteWin.close()\n}\n\n// responseMessageReceived is called when a success or failure message is\n// received on a channel to check that such a message is reasonable for the\n// given channel.\nfunc (ch *Channel) responseMessageReceived() error {\n\tif ch.direction == channelInbound {\n\t\treturn errors.New(\"qmux: channel response message received on inbound channel\")\n\t}\n\treturn nil\n}\n\nfunc (ch *Channel) handle(msg codec.Message) error {\n\tswitch m := msg.(type) {\n\tcase *codec.DataMessage:\n\t\treturn ch.handleData(m)\n\n\tcase *codec.CloseMessage:\n\t\tch.send(codec.CloseMessage{\n\t\t\tChannelID: ch.remoteId,\n\t\t})\n\t\tch.session.chans.remove(ch.localId)\n\t\tch.close()\n\t\treturn nil\n\n\tcase *codec.EOFMessage:\n\t\tch.pending.eof()\n\t\treturn nil\n\n\tcase *codec.WindowAdjustMessage:\n\t\tif !ch.remoteWin.add(m.AdditionalBytes) {\n\t\t\treturn fmt.Errorf(\"qmux: invalid window update for %d bytes\", m.AdditionalBytes)\n\t\t}\n\t\treturn nil\n\n\tcase *codec.OpenConfirmMessage:\n\t\tif err := ch.responseMessageReceived(); err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif m.MaxPacketSize < minPacketLength || m.MaxPacketSize > maxPacketLength {\n\t\t\treturn fmt.Errorf(\"qmux: invalid MaxPacketSize %d from peer\", m.MaxPacketSize)\n\t\t}\n\t\tch.remoteId = m.SenderID\n\t\tch.maxRemotePayload = m.MaxPacketSize\n\t\tch.remoteWin.add(m.WindowSize)\n\t\tch.msg <- m\n\t\treturn nil\n\n\tcase *codec.OpenFailureMessage:\n\t\tif err := ch.responseMessageReceived(); err != nil {\n\t\t\treturn err\n\t\t}\n\t\tch.session.chans.remove(m.ChannelID)\n\t\tch.msg <- m\n\t\treturn nil\n\n\tdefault:\n\t\treturn fmt.Errorf(\"qmux: invalid channel message %v\", msg)\n\t}\n}\n\nfunc (ch *Channel) handleData(msg *codec.DataMessage) error {\n\tif msg.Length > ch.maxIncomingPayload {\n\t\t// TODO(hanwen): should send Disconnect?\n\t\treturn errors.New(\"qmux: incoming packet exceeds maximum payload size\")\n\t}\n\n\tif msg.Length != uint32(len(msg.Data)) {\n\t\treturn errors.New(\"qmux: wrong packet length\")\n\t}\n\n\tch.windowMu.Lock()\n\tif ch.myWindow < msg.Length {\n\t\tch.windowMu.Unlock()\n\t\t// TODO(hanwen): should send Disconnect with reason?\n\t\treturn errors.New(\"qmux: remote side wrote too much\")\n\t}\n\tch.myWindow -= msg.Length\n\tch.windowMu.Unlock()\n\n\tch.pending.write(msg.Data)\n\treturn nil\n}\n" }, { "path": "golang/session/doc.go", "content": "// Package session implements a qmux session and channel API.\npackage session\n" }, { "path": "golang/session/session.go", "content": "package session\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"io\"\n\t\"sync\"\n\n\t\"github.com/progrium/qmux/golang/codec\"\n\t\"github.com/progrium/qmux/golang/mux\"\n)\n\nconst (\n\tminPacketLength = 9\n\tmaxPacketLength = 1 << 31\n\n\t// channelMaxPacket contains the maximum number of bytes that will be\n\t// sent in a single packet. As per RFC 4253, section 6.1, 32k is also\n\t// the minimum.\n\tchannelMaxPacket = 1 << 15\n\t// We follow OpenSSH here.\n\tchannelWindowSize = 64 * channelMaxPacket\n\n\t// chanSize sets the amount of buffering qmux connections. This is\n\t// primarily for testing: setting chanSize=0 uncovers deadlocks more\n\t// quickly.\n\tchanSize = 16\n)\n\n// Session is a bi-directional channel muxing session on a given transport.\ntype Session struct {\n\tt mux.Transport\n\tchans chanList\n\n\tenc *codec.Encoder\n\tdec *codec.Decoder\n\n\tinbox chan mux.Channel\n\n\terrCond *sync.Cond\n\terr error\n\tcloseCh chan bool\n}\n\n// NewSession returns a session that runs over the given transport.\nfunc New(t mux.Transport) *Session {\n\tif t == nil {\n\t\treturn nil\n\t}\n\ts := &Session{\n\t\tt: t,\n\t\tenc: codec.NewEncoder(t),\n\t\tdec: codec.NewDecoder(t),\n\t\tinbox: make(chan mux.Channel, chanSize),\n\t\terrCond: sync.NewCond(new(sync.Mutex)),\n\t\tcloseCh: make(chan bool, 1),\n\t}\n\tgo s.loop()\n\treturn s\n}\n\n// Close closes the underlying transport.\nfunc (s *Session) Close() error {\n\ts.t.Close()\n\treturn nil\n}\n\n// Wait blocks until the transport has shut down, and returns the\n// error causing the shutdown.\nfunc (s *Session) Wait() error {\n\ts.errCond.L.Lock()\n\tdefer s.errCond.L.Unlock()\n\tfor s.err == nil {\n\t\ts.errCond.Wait()\n\t}\n\treturn s.err\n}\n\n// Accept waits for and returns the next incoming channel.\nfunc (s *Session) Accept() (mux.Channel, error) {\n\tselect {\n\tcase ch := <-s.inbox:\n\t\treturn ch, nil\n\tcase <-s.closeCh:\n\t\treturn nil, io.EOF\n\t}\n}\n\n// Open establishes a new channel with the other end.\nfunc (s *Session) Open(ctx context.Context) (mux.Channel, error) {\n\tch := s.newChannel(channelOutbound)\n\tch.maxIncomingPayload = channelMaxPacket\n\n\tif err := s.enc.Encode(codec.OpenMessage{\n\t\tWindowSize: ch.myWindow,\n\t\tMaxPacketSize: ch.maxIncomingPayload,\n\t\tSenderID: ch.localId,\n\t}); err != nil {\n\t\treturn nil, err\n\t}\n\n\tvar m codec.Message\n\n\tselect {\n\tcase <-ctx.Done():\n\t\treturn nil, ctx.Err()\n\tcase m = <-ch.msg:\n\t\tif m == nil {\n\t\t\treturn nil, fmt.Errorf(\"qmux: channel closed early during open\")\n\t\t}\n\t}\n\n\tswitch msg := m.(type) {\n\tcase *codec.OpenConfirmMessage:\n\t\treturn ch, nil\n\tcase *codec.OpenFailureMessage:\n\t\treturn nil, fmt.Errorf(\"qmux: channel open failed on remote side\")\n\tdefault:\n\t\treturn nil, fmt.Errorf(\"qmux: unexpected packet in response to channel open: %v\", msg)\n\t}\n}\n\nfunc (s *Session) newChannel(direction channelDirection) *Channel {\n\tch := &Channel{\n\t\tremoteWin: window{Cond: sync.NewCond(new(sync.Mutex))},\n\t\tmyWindow: channelWindowSize,\n\t\tpending: newBuffer(),\n\t\tdirection: direction,\n\t\tmsg: make(chan codec.Message, chanSize),\n\t\tsession: s,\n\t\tpacketBuf: make([]byte, 0),\n\t}\n\tch.localId = s.chans.add(ch)\n\treturn ch\n}\n\n// loop runs the connection machine. It will process packets until an\n// error is encountered. To synchronize on loop exit, use session.Wait.\nfunc (s *Session) loop() {\n\tvar err error\n\tfor err == nil {\n\t\terr = s.onePacket()\n\t}\n\n\tfor _, ch := range s.chans.dropAll() {\n\t\tch.close()\n\t}\n\n\ts.t.Close()\n\ts.closeCh <- true\n\n\ts.errCond.L.Lock()\n\ts.err = err\n\ts.errCond.Broadcast()\n\ts.errCond.L.Unlock()\n}\n\n// onePacket reads and processes one packet.\nfunc (s *Session) onePacket() error {\n\tvar err error\n\tvar msg codec.Message\n\n\tmsg, err = s.dec.Decode()\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tid, isChan := msg.Channel()\n\tif !isChan {\n\t\treturn s.handleOpen(msg.(*codec.OpenMessage))\n\t}\n\n\tch := s.chans.getChan(id)\n\tif ch == nil {\n\t\treturn fmt.Errorf(\"qmux: invalid channel %d\", id)\n\t}\n\n\treturn ch.handle(msg)\n}\n\n// handleChannelOpen schedules a channel to be Accept()ed.\nfunc (s *Session) handleOpen(msg *codec.OpenMessage) error {\n\tif msg.MaxPacketSize < minPacketLength || msg.MaxPacketSize > maxPacketLength {\n\t\treturn s.enc.Encode(codec.OpenFailureMessage{\n\t\t\tChannelID: msg.SenderID,\n\t\t})\n\t}\n\n\tc := s.newChannel(channelInbound)\n\tc.remoteId = msg.SenderID\n\tc.maxRemotePayload = msg.MaxPacketSize\n\tc.remoteWin.add(msg.WindowSize)\n\tc.maxIncomingPayload = channelMaxPacket\n\ts.inbox <- c\n\n\treturn s.enc.Encode(codec.OpenConfirmMessage{\n\t\tChannelID: c.remoteId,\n\t\tSenderID: c.localId,\n\t\tWindowSize: c.myWindow,\n\t\tMaxPacketSize: c.maxIncomingPayload,\n\t})\n}\n" }, { "path": "golang/session/session_test.go", "content": "package session\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"errors\"\n\t\"io/ioutil\"\n\t\"net\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/progrium/qmux/golang/mux\"\n)\n\nfunc fatal(err error, t *testing.T) {\n\tt.Helper()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\nfunc TestQmux(t *testing.T) {\n\tl, err := net.Listen(\"tcp\", \"127.0.0.1:0\")\n\tfatal(err, t)\n\tdefer l.Close()\n\n\tgo func() {\n\t\tconn, err := l.Accept()\n\t\tfatal(err, t)\n\t\tdefer conn.Close()\n\n\t\tsess := New(conn)\n\n\t\tch, err := sess.Open(context.Background())\n\t\tfatal(err, t)\n\t\tb, err := ioutil.ReadAll(ch)\n\t\tfatal(err, t)\n\t\tch.Close() // should already be closed by other end\n\n\t\tch, err = sess.Accept()\n\t\t_, err = ch.Write(b)\n\t\tfatal(err, t)\n\t\terr = ch.CloseWrite()\n\t\tfatal(err, t)\n\n\t\terr = sess.Close()\n\t\tfatal(err, t)\n\t}()\n\n\tconn, err := net.Dial(\"tcp\", l.Addr().String())\n\tfatal(err, t)\n\tdefer conn.Close()\n\n\tsess := New(conn)\n\n\tvar ch mux.Channel\n\tt.Run(\"session accept\", func(t *testing.T) {\n\t\tch, err = sess.Accept()\n\t\tfatal(err, t)\n\t})\n\n\tt.Run(\"channel write\", func(t *testing.T) {\n\t\t_, err = ch.Write([]byte(\"Hello world\"))\n\t\tfatal(err, t)\n\t\terr = ch.Close()\n\t\tfatal(err, t)\n\t})\n\n\tt.Run(\"session open\", func(t *testing.T) {\n\t\tch, err = sess.Open(context.Background())\n\t\tfatal(err, t)\n\t})\n\n\tvar b []byte\n\tt.Run(\"channel read\", func(t *testing.T) {\n\t\tb, err = ioutil.ReadAll(ch)\n\t\tfatal(err, t)\n\t\tch.Close() // should already be closed by other end\n\t})\n\n\tif !bytes.Equal(b, []byte(\"Hello world\")) {\n\t\tt.Fatalf(\"unexpected bytes: %s\", b)\n\t}\n}\n\nfunc TestSessionOpenTimeout(t *testing.T) {\n\tl, err := net.Listen(\"tcp\", \"127.0.0.1:0\")\n\tfatal(err, t)\n\tdefer l.Close()\n\n\tconn, err := net.Dial(\"tcp\", l.Addr().String())\n\tfatal(err, t)\n\tdefer conn.Close()\n\n\tsess := New(conn)\n\n\tctx, cancel := context.WithTimeout(context.Background(), 10*time.Millisecond)\n\tdefer cancel()\n\n\tch, err := sess.Open(ctx)\n\tif err != context.DeadlineExceeded {\n\t\tt.Fatalf(\"expected DeadlineExceeded, but got: %v\", err)\n\t}\n\tif ch != nil {\n\t\tch.Close()\n\t}\n}\n\nfunc TestSessionWait(t *testing.T) {\n\tl, err := net.Listen(\"tcp\", \"127.0.0.1:0\")\n\tfatal(err, t)\n\tdefer l.Close()\n\n\tconn, err := net.Dial(\"tcp\", l.Addr().String())\n\tfatal(err, t)\n\tdefer conn.Close()\n\n\tsess := New(conn)\n\tfatal(sess.Close(), t)\n\t// wait should return immediately since the connection was closed\n\terr = mux.Wait(sess)\n\tvar netErr net.Error\n\tif !errors.As(err, &netErr) {\n\t\tt.Fatalf(\"expected a network error, but got: %v\", err)\n\t}\n}\n" }, { "path": "golang/session/util.go", "content": "package session\n\nfunc min(a uint32, b int) uint32 {\n\tif a < uint32(b) {\n\t\treturn a\n\t}\n\treturn uint32(b)\n}\n" }, { "path": "golang/session/util_buffer.go", "content": "package session\n\nimport (\n\t\"io\"\n\t\"sync\"\n)\n\n// buffer provides a linked list buffer for data exchange\n// between producer and consumer. Theoretically the buffer is\n// of unlimited capacity as it does no allocation of its own.\ntype buffer struct {\n\t// protects concurrent access to head, tail and closed\n\t*sync.Cond\n\n\thead *element // the buffer that will be read first\n\ttail *element // the buffer that will be read last\n\n\tclosed bool\n}\n\n// An element represents a single link in a linked list.\ntype element struct {\n\tbuf []byte\n\tnext *element\n}\n\n// newBuffer returns an empty buffer that is not closed.\nfunc newBuffer() *buffer {\n\te := new(element)\n\tb := &buffer{\n\t\tCond: sync.NewCond(new(sync.Mutex)),\n\t\thead: e,\n\t\ttail: e,\n\t}\n\treturn b\n}\n\n// write makes buf available for Read to receive.\n// buf must not be modified after the call to write.\nfunc (b *buffer) write(buf []byte) {\n\tb.Cond.L.Lock()\n\te := &element{buf: buf}\n\tb.tail.next = e\n\tb.tail = e\n\tb.Cond.Signal()\n\tb.Cond.L.Unlock()\n}\n\n// eof closes the buffer. Reads from the buffer once all\n// the data has been consumed will receive io.EOF.\nfunc (b *buffer) eof() {\n\tb.Cond.L.Lock()\n\tb.closed = true\n\tb.Cond.Signal()\n\tb.Cond.L.Unlock()\n}\n\n// Read reads data from the internal buffer in buf. Reads will block\n// if no data is available, or until the buffer is closed.\nfunc (b *buffer) Read(buf []byte) (n int, err error) {\n\tb.Cond.L.Lock()\n\tdefer b.Cond.L.Unlock()\n\n\tfor len(buf) > 0 {\n\t\t// if there is data in b.head, copy it\n\t\tif len(b.head.buf) > 0 {\n\t\t\tr := copy(buf, b.head.buf)\n\t\t\tbuf, b.head.buf = buf[r:], b.head.buf[r:]\n\t\t\tn += r\n\t\t\tcontinue\n\t\t}\n\t\t// if there is a next buffer, make it the head\n\t\tif len(b.head.buf) == 0 && b.head != b.tail {\n\t\t\tb.head = b.head.next\n\t\t\tcontinue\n\t\t}\n\n\t\t// if at least one byte has been copied, return\n\t\tif n > 0 {\n\t\t\tbreak\n\t\t}\n\n\t\t// if nothing was read, and there is nothing outstanding\n\t\t// check to see if the buffer is closed.\n\t\tif b.closed {\n\t\t\terr = io.EOF\n\t\t\tbreak\n\t\t}\n\t\t// out of buffers, wait for producer\n\t\tb.Cond.Wait()\n\t}\n\treturn\n}\n" }, { "path": "golang/session/util_chanlist.go", "content": "package session\n\nimport \"sync\"\n\n// chanList is a thread safe channel list.\ntype chanList struct {\n\t// protects concurrent access to chans\n\tsync.Mutex\n\n\t// chans are indexed by the local id of the channel, which the\n\t// other side should send in the PeersId field.\n\tchans []*Channel\n}\n\n// Assigns a channel ID to the given channel.\nfunc (c *chanList) add(ch *Channel) uint32 {\n\tc.Lock()\n\tdefer c.Unlock()\n\tfor i := range c.chans {\n\t\tif c.chans[i] == nil {\n\t\t\tc.chans[i] = ch\n\t\t\treturn uint32(i)\n\t\t}\n\t}\n\tc.chans = append(c.chans, ch)\n\treturn uint32(len(c.chans) - 1)\n}\n\n// getChan returns the channel for the given ID.\nfunc (c *chanList) getChan(id uint32) *Channel {\n\tc.Lock()\n\tdefer c.Unlock()\n\tif id < uint32(len(c.chans)) {\n\t\treturn c.chans[id]\n\t}\n\treturn nil\n}\n\nfunc (c *chanList) remove(id uint32) {\n\tc.Lock()\n\tif id < uint32(len(c.chans)) {\n\t\tc.chans[id] = nil\n\t}\n\tc.Unlock()\n}\n\n// dropAll forgets all channels it knows, returning them in a slice.\nfunc (c *chanList) dropAll() []*Channel {\n\tc.Lock()\n\tdefer c.Unlock()\n\tvar r []*Channel\n\n\tfor _, ch := range c.chans {\n\t\tif ch == nil {\n\t\t\tcontinue\n\t\t}\n\t\tr = append(r, ch)\n\t}\n\tc.chans = nil\n\treturn r\n}\n" }, { "path": "golang/session/util_window.go", "content": "package session\n\nimport (\n\t\"io\"\n\t\"sync\"\n)\n\n// window represents the buffer available to clients\n// wishing to write to a channel.\ntype window struct {\n\t*sync.Cond\n\twin uint32 // RFC 4254 5.2 says the window size can grow to 2^32-1\n\twriteWaiters int\n\tclosed bool\n}\n\n// add adds win to the amount of window available\n// for consumers.\nfunc (w *window) add(win uint32) bool {\n\t// a zero sized window adjust is a noop.\n\tif win == 0 {\n\t\treturn true\n\t}\n\tw.L.Lock()\n\tif w.win+win < win {\n\t\tw.L.Unlock()\n\t\treturn false\n\t}\n\tw.win += win\n\t// It is unusual that multiple goroutines would be attempting to reserve\n\t// window space, but not guaranteed. Use broadcast to notify all waiters\n\t// that additional window is available.\n\tw.Broadcast()\n\tw.L.Unlock()\n\treturn true\n}\n\n// close sets the window to closed, so all reservations fail\n// immediately.\nfunc (w *window) close() {\n\tw.L.Lock()\n\tw.closed = true\n\tw.Broadcast()\n\tw.L.Unlock()\n}\n\n// reserve reserves win from the available window capacity.\n// If no capacity remains, reserve will block. reserve may\n// return less than requested.\nfunc (w *window) reserve(win uint32) (uint32, error) {\n\tvar err error\n\tw.L.Lock()\n\tw.writeWaiters++\n\tw.Broadcast()\n\tfor w.win == 0 && !w.closed {\n\t\tw.Wait()\n\t}\n\tw.writeWaiters--\n\tif w.win < win {\n\t\twin = w.win\n\t}\n\tw.win -= win\n\tif w.closed {\n\t\terr = io.EOF\n\t}\n\tw.L.Unlock()\n\treturn win, err\n}\n\n// waitWriterBlocked waits until some goroutine is blocked for further\n// writes. It is used in tests only.\nfunc (w *window) waitWriterBlocked() {\n\tw.Cond.L.Lock()\n\tfor w.writeWaiters == 0 {\n\t\tw.Cond.Wait()\n\t}\n\tw.Cond.L.Unlock()\n}\n" }, { "path": "golang/transport/dial_io.go", "content": "package transport\n\nimport (\n\t\"io\"\n\t\"os\"\n\n\t\"github.com/progrium/qmux/golang/mux\"\n\t\"github.com/progrium/qmux/golang/session\"\n)\n\nfunc DialIO(out io.WriteCloser, in io.ReadCloser) (mux.Session, error) {\n\treturn session.New(&ioduplex{out, in}), nil\n}\n\nfunc DialStdio() (mux.Session, error) {\n\treturn DialIO(os.Stdout, os.Stdin)\n}\n" }, { "path": "golang/transport/dial_net.go", "content": "package transport\n\nimport (\n\t\"net\"\n\n\t\"github.com/progrium/qmux/golang/mux\"\n\t\"github.com/progrium/qmux/golang/session\"\n)\n\nfunc dialNet(proto, addr string) (mux.Session, error) {\n\tconn, err := net.Dial(proto, addr)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn session.New(conn), nil\n}\n\nfunc DialTCP(addr string) (mux.Session, error) {\n\treturn dialNet(\"tcp\", addr)\n}\n\nfunc DialUnix(addr string) (mux.Session, error) {\n\treturn dialNet(\"unix\", addr)\n}\n" }, { "path": "golang/transport/dial_ws.go", "content": "package transport\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/progrium/qmux/golang/mux\"\n\t\"github.com/progrium/qmux/golang/session\"\n\t\"golang.org/x/net/websocket\"\n)\n\nfunc DialWS(addr string) (mux.Session, error) {\n\tws, err := websocket.Dial(fmt.Sprintf(\"ws://%s/\", addr), \"\", fmt.Sprintf(\"http://%s/\", addr))\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tws.PayloadType = websocket.BinaryFrame\n\treturn session.New(ws), nil\n}\n" }, { "path": "golang/transport/doc.go", "content": "// Package transport provides several dialers and listeners for getting qmux sessions over TCP, Unix sockets, WebSocket, and stdio.\npackage transport\n" }, { "path": "golang/transport/listen.go", "content": "package transport\n\nimport \"github.com/progrium/qmux/golang/mux\"\n\ntype Listener interface {\n\t// Close closes the listener.\n\t// Any blocked Accept operations will be unblocked and return errors.\n\tClose() error\n\n\t// Accept waits for and returns the next incoming session.\n\tAccept() (mux.Session, error)\n}\n" }, { "path": "golang/transport/listen_io.go", "content": "package transport\n\nimport (\n\t\"io\"\n\t\"os\"\n\n\t\"github.com/progrium/qmux/golang/mux\"\n\t\"github.com/progrium/qmux/golang/session\"\n)\n\ntype IOListener struct {\n\tio.ReadWriteCloser\n}\n\nfunc (l *IOListener) Accept() (mux.Session, error) {\n\treturn session.New(l.ReadWriteCloser), nil\n}\n\ntype ioduplex struct {\n\tio.WriteCloser\n\tio.ReadCloser\n}\n\nfunc (d *ioduplex) Close() error {\n\tif err := d.WriteCloser.Close(); err != nil {\n\t\treturn err\n\t}\n\tif err := d.ReadCloser.Close(); err != nil {\n\t\treturn err\n\t}\n\treturn nil\n}\n\nfunc ListenIO(out io.WriteCloser, in io.ReadCloser) (*IOListener, error) {\n\treturn &IOListener{\n\t\t&ioduplex{out, in},\n\t}, nil\n}\n\nfunc ListenStdio() (*IOListener, error) {\n\treturn ListenIO(os.Stdout, os.Stdin)\n}\n" }, { "path": "golang/transport/listen_net.go", "content": "package transport\n\nimport (\n\t\"io\"\n\t\"net\"\n\n\t\"github.com/progrium/qmux/golang/mux\"\n\t\"github.com/progrium/qmux/golang/session\"\n)\n\ntype NetListener struct {\n\tnet.Listener\n\taccepted chan mux.Session\n\tcloser chan bool\n\terrs chan error\n}\n\nfunc (l *NetListener) Accept() (mux.Session, error) {\n\tselect {\n\tcase <-l.closer:\n\t\treturn nil, io.EOF\n\tcase err := <-l.errs:\n\t\treturn nil, err\n\tcase sess := <-l.accepted:\n\t\treturn sess, nil\n\t}\n}\n\n// func (l *NetListener) Addr() net.Addr {\n// \treturn l.Addr()\n// }\n\nfunc (l *NetListener) Close() error {\n\tif l.closer != nil {\n\t\tl.closer <- true\n\t}\n\treturn l.Listener.Close()\n}\n\nfunc listenNet(proto, addr string) (*NetListener, error) {\n\tl, err := net.Listen(proto, addr)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tcloser := make(chan bool, 1)\n\terrs := make(chan error, 1)\n\taccepted := make(chan mux.Session)\n\tgo func(l net.Listener) {\n\t\tfor {\n\t\t\tconn, err := l.Accept()\n\t\t\tif err != nil {\n\t\t\t\terrs <- err\n\t\t\t\treturn\n\t\t\t}\n\t\t\taccepted <- session.New(conn)\n\t\t}\n\t}(l)\n\treturn &NetListener{\n\t\tListener: l,\n\t\terrs: errs,\n\t\taccepted: accepted,\n\t\tcloser: closer,\n\t}, nil\n}\n\nfunc ListenTCP(addr string) (*NetListener, error) {\n\treturn listenNet(\"tcp\", addr)\n}\n\nfunc ListenUnix(addr string) (*NetListener, error) {\n\treturn listenNet(\"unix\", addr)\n}\n" }, { "path": "golang/transport/listen_ws.go", "content": "package transport\n\nimport (\n\t\"net\"\n\t\"net/http\"\n\n\t\"github.com/progrium/qmux/golang/mux\"\n\t\"github.com/progrium/qmux/golang/session\"\n\t\"golang.org/x/net/websocket\"\n)\n\nfunc HandleWS(l *NetListener, ws *websocket.Conn) {\n\tws.PayloadType = websocket.BinaryFrame\n\tsess := session.New(ws)\n\tdefer sess.Close()\n\tl.accepted <- sess\n\tl.errs <- mux.Wait(sess)\n}\n\nfunc ListenWS(addr string) (*NetListener, error) {\n\tl, err := net.Listen(\"tcp\", addr)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tnl := &NetListener{\n\t\tListener: l,\n\t\taccepted: make(chan mux.Session),\n\t\terrs: make(chan error, 2),\n\t\tcloser: make(chan bool, 1),\n\t}\n\ts := &http.Server{\n\t\tAddr: addr,\n\t\tHandler: websocket.Handler(func(ws *websocket.Conn) {\n\t\t\tHandleWS(nl, ws)\n\t\t}),\n\t}\n\tgo func() {\n\t\tnl.errs <- s.Serve(l)\n\t}()\n\treturn nl, nil\n}\n" }, { "path": "golang/transport/transport_test.go", "content": "package transport\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"path\"\n\t\"testing\"\n\n\t\"github.com/progrium/qmux/golang/mux\"\n)\n\nfunc fatal(err error, t *testing.T) {\n\tt.Helper()\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\nfunc testExchange(t *testing.T, sess mux.Session) {\n\tvar err error\n\tvar ch mux.Channel\n\tt.Run(\"session accept\", func(t *testing.T) {\n\t\tch, err = sess.Accept()\n\t\tfatal(err, t)\n\t})\n\n\tt.Run(\"channel write\", func(t *testing.T) {\n\t\t_, err = ch.Write([]byte(\"Hello world\"))\n\t\tfatal(err, t)\n\t\terr = ch.Close()\n\t\tfatal(err, t)\n\t})\n\n\tt.Run(\"session open\", func(t *testing.T) {\n\t\tch, err = sess.Open(context.Background())\n\t\tfatal(err, t)\n\t})\n\n\tvar b []byte\n\tt.Run(\"channel read\", func(t *testing.T) {\n\t\tb, err = ioutil.ReadAll(ch)\n\t\tfatal(err, t)\n\t\terr = ch.Close()\n\t\tfatal(err, t)\n\t})\n\n\tif !bytes.Equal(b, []byte(\"Hello world\")) {\n\t\tt.Fatalf(\"unexpected bytes: %s\", b)\n\t}\n\n\tt.Run(\"session close\", func(t *testing.T) {\n\t\terr = sess.Close()\n\t\tfatal(err, t)\n\t})\n}\n\nfunc startListener(t *testing.T, l Listener) {\n\tt.Helper()\n\n\tt.Cleanup(func() {\n\t\tfatal(l.Close(), t)\n\t})\n\n\tgo func() {\n\t\tsess, err := l.Accept()\n\t\tfatal(err, t)\n\t\tt.Cleanup(func() {\n\t\t\t// Synchronizes cleanup, waiting for the client to disconnect before\n\t\t\t// closing the stream. This prevents errors in the Pipe-based test with\n\t\t\t// closing one end of the pipe before the other has read the data.\n\t\t\t// Registering as a test cleanup function also avoids a race condition\n\t\t\t// with the test exiting before closing the session.\n\t\t\tif err := mux.Wait(sess); err != io.EOF {\n\t\t\t\tt.Errorf(\"Wait returned unexpected error: %v\", err)\n\t\t\t}\n\t\t\terr = sess.Close()\n\t\t\tfatal(err, t)\n\t\t})\n\n\t\tch, err := sess.Open(context.Background())\n\t\tfatal(err, t)\n\t\tb, err := ioutil.ReadAll(ch)\n\t\tfatal(err, t)\n\t\tch.Close()\n\n\t\tch, err = sess.Accept()\n\t\t_, err = ch.Write(b)\n\t\tfatal(err, t)\n\t\terr = ch.CloseWrite()\n\t\tfatal(err, t)\n\t}()\n}\n\nfunc TestTCP(t *testing.T) {\n\tl, err := ListenTCP(\"127.0.0.1:0\")\n\tfatal(err, t)\n\tstartListener(t, l)\n\n\tsess, err := DialTCP(l.Addr().String())\n\tfatal(err, t)\n\ttestExchange(t, sess)\n}\n\nfunc TestUnix(t *testing.T) {\n\ttmp := t.TempDir()\n\tsockPath := path.Join(tmp, \"qmux.sock\")\n\tl, err := ListenUnix(sockPath)\n\tfatal(err, t)\n\tstartListener(t, l)\n\n\tsess, err := DialUnix(sockPath)\n\tfatal(err, t)\n\ttestExchange(t, sess)\n}\n\nfunc TestIO(t *testing.T) {\n\tpr1, pw1 := io.Pipe()\n\tpr2, pw2 := io.Pipe()\n\n\tl, err := ListenIO(pw1, pr2)\n\tfatal(err, t)\n\tstartListener(t, l)\n\n\tsess, err := DialIO(pw2, pr1)\n\tfatal(err, t)\n\ttestExchange(t, sess)\n}\n\nfunc TestWS(t *testing.T) {\n\tl, err := ListenWS(\"127.0.0.1:0\")\n\tfatal(err, t)\n\tstartListener(t, l)\n\n\tsess, err := DialWS(l.Addr().String())\n\tfatal(err, t)\n\ttestExchange(t, sess)\n}\n" }, { "path": "typescript/Makefile", "content": "\nbuild: dist/qmux.js\n\ndist/qmux.js: **.ts\n\tmkdir -p dist\n\tdeno bundle -c tsconfig.json index.ts dist/qmux.js" }, { "path": "typescript/README.md", "content": "# qmux for TypeScript" }, { "path": "typescript/api.ts", "content": "\nexport interface IConn {\n read(len: number): Promise;\n write(buffer: Uint8Array): Promise;\n close(): Promise;\n}\n\nexport interface ISession {\n open(): Promise;\n accept(): Promise;\n close(): Promise;\n}\n\nexport interface IChannel extends IConn {\n ident(): number\n closeWrite(): Promise\n}\n\nexport interface IConnListener {\n accept(): Promise;\n close(): Promise;\n}\n" }, { "path": "typescript/channel.ts", "content": "// @ts-ignore\nimport * as util from \"./util.ts\";\n// @ts-ignore\nimport * as codec from \"./codec/index.ts\";\n// @ts-ignore\nimport * as internal from \"./internal.ts\";\n\nexport const channelMaxPacket = 1 << 15;\nexport const channelWindowSize = 64 * channelMaxPacket;\n\n// channel represents a virtual muxed connection\nexport class Channel {\n localId: number;\n remoteId: number;\n maxIncomingPayload: number;\n maxRemotePayload: number;\n session: internal.Session;\n ready: util.queue;\n sentEOF: boolean;\n sentClose: boolean;\n remoteWin: number;\n myWindow: number;\n readBuf: util.ReadBuffer;\n writers: Array<() => void>;\n\n constructor(sess: internal.Session) {\n this.localId = 0;\n this.remoteId = 0;\n this.maxIncomingPayload = 0;\n this.maxRemotePayload = 0;\n this.sentEOF = false;\n this.sentClose = false;\n this.remoteWin = 0;\n this.myWindow = 0;\n this.ready = new util.queue();\n this.session = sess;\n this.writers = [];\n this.readBuf = new util.ReadBuffer();\n }\n\n ident(): number {\n return this.localId;\n }\n\n async read(len: number): Promise {\n let data = await this.readBuf.read(len);\n if (data !== undefined) {\n try {\n await this.adjustWindow(data.byteLength)\n } catch (e) {\n if (e !== \"EOF\") {\n throw e;\n }\n }\n }\n return data;\n }\n\n reserveWindow(win: number): number {\n if (this.remoteWin < win) {\n win = this.remoteWin;\n }\n this.remoteWin -= win;\n return win;\n }\n\n addWindow(win: number) {\n this.remoteWin += win;\n while (this.remoteWin > 0) {\n let writer = this.writers.shift();\n if (!writer) break;\n writer();\n }\n }\n\n write(buffer: Uint8Array): Promise {\n if (this.sentEOF) {\n return Promise.reject(\"EOF\");\n }\n\n return new Promise((resolve, reject) => {\n let n = 0;\n let tryWrite = () => {\n if (this.sentEOF || this.sentClose) {\n reject(\"EOF\");\n return;\n }\n if (buffer.byteLength == 0) {\n resolve(n);\n return;\n }\n let space = Math.min(this.maxRemotePayload, buffer.byteLength);\n let reserved = this.reserveWindow(space);\n if (reserved == 0) {\n this.writers.push(tryWrite);\n return;\n }\n\n let toSend = buffer.slice(0, reserved);\n\n this.send({\n ID: codec.DataID,\n channelID: this.remoteId,\n length: toSend.byteLength,\n data: toSend,\n }).then(() => {\n n += toSend.byteLength;\n buffer = buffer.slice(toSend.byteLength);\n if (buffer.byteLength == 0) {\n resolve(n);\n return;\n }\n this.writers.push(tryWrite);\n })\n }\n tryWrite();\n })\n }\n\n async closeWrite() {\n this.sentEOF = true;\n await this.send({\n ID: codec.EofID,\n channelID: this.remoteId\n });\n this.writers.forEach(writer => writer());\n this.writers = [];\n }\n\n async close(): Promise {\n if (!this.sentClose) {\n await this.send({\n ID: codec.CloseID,\n channelID: this.remoteId\n });\n this.sentClose = true;\n while (await this.ready.shift() !== undefined) { }\n return;\n }\n this.shutdown();\n }\n\n shutdown(): void {\n this.readBuf.close();\n this.writers.forEach(writer => writer());\n this.ready.close();\n this.session.rmCh(this.localId);\n }\n\n async adjustWindow(n: number) {\n // Since myWindow is managed on our side, and can never exceed\n // the initial window setting, we don't worry about overflow.\n this.myWindow += n;\n await this.send({\n ID: codec.WindowAdjustID,\n channelID: this.remoteId,\n additionalBytes: n,\n })\n }\n\n send(msg: codec.ChannelMessage): Promise {\n if (this.sentClose) {\n throw \"EOF\";\n }\n\n this.sentClose = (msg.ID === codec.CloseID);\n\n return this.session.enc.encode(msg);\n }\n\n handle(msg: codec.ChannelMessage): void {\n if (msg.ID === codec.DataID) {\n this.handleData(msg as codec.DataMessage);\n return;\n }\n if (msg.ID === codec.CloseID) {\n this.close(); // is this right?\n return;\n }\n if (msg.ID === codec.EofID) {\n this.readBuf.eof();\n }\n if (msg.ID === codec.OpenFailureID) {\n this.session.rmCh(msg.channelID);\n this.ready.push(false);\n return;\n }\n if (msg.ID === codec.OpenConfirmID) {\n if (msg.maxPacketSize < internal.minPacketLength || msg.maxPacketSize > internal.maxPacketLength) {\n throw \"invalid max packet size\";\n }\n this.remoteId = msg.senderID;\n this.maxRemotePayload = msg.maxPacketSize;\n this.addWindow(msg.windowSize);\n this.ready.push(true);\n return;\n }\n if (msg.ID === codec.WindowAdjustID) {\n this.addWindow(msg.additionalBytes);\n }\n }\n\n handleData(msg: codec.DataMessage) {\n if (msg.length > this.maxIncomingPayload) {\n throw \"incoming packet exceeds maximum payload size\";\n }\n\n // TODO: check packet length\n if (this.myWindow < msg.length) {\n throw \"remote side wrote too much\";\n }\n\n this.myWindow -= msg.length;\n\n this.readBuf.write(msg.data)\n }\n\n}\n\n" }, { "path": "typescript/codec/codec_test.ts", "content": "import {\n assertEquals,\n} from \"https://deno.land/std/testing/asserts.ts\";\n\n// @ts-ignore\nimport * as codec from \"./index.ts\";\nimport * as msg from \"./message.ts\";\n\nDeno.test(\"hello world #1\", () => {\n let packet = new Uint8Array(5);\n packet.set([105, 0, 0, 0, 0]);\n let obj = codec.Unmarshal(packet) as msg.AnyMessage;\n let buf = codec.Marshal(obj);\n console.log(\"Hello\", obj, buf);\n});\n" }, { "path": "typescript/codec/decoder.ts", "content": "// @ts-ignore\nimport * as msg from \"./message.ts\";\n// @ts-ignore\nimport * as api from \"../api.ts\";\n// @ts-ignore\nimport * as util from \"../util.ts\";\n\nexport class Decoder {\n conn: api.IConn;\n debug: boolean;\n\n constructor(conn: api.IConn, debug: boolean = false) {\n this.conn = conn;\n this.debug = debug;\n }\n\n async decode(): Promise {\n let packet = await readPacket(this.conn);\n if (packet === undefined) {\n return Promise.resolve(undefined);\n }\n let msg = Unmarshal(packet);\n if (this.debug) {\n console.log(\">>\", msg);\n }\n return msg;\n }\n}\n\nasync function readPacket(conn: api.IConn): Promise {\n let head = await conn.read(1);\n if (head === undefined) {\n return Promise.resolve(undefined);\n }\n let msgID = head[0];\n\n let size = msg.payloadSizes.get(msgID);\n if (size === undefined || msgID < msg.OpenID || msgID > msg.CloseID) {\n return Promise.reject(`bad packet: ${msgID}`);\n }\n\n let rest = await conn.read(size);\n if (rest === undefined) {\n return Promise.reject(\"unexpected EOF\");\n }\n\n if (msgID === msg.DataID) {\n let view = new DataView(rest.buffer);\n let length = view.getUint32(4);\n let data = await conn.read(length);\n if (data === undefined) {\n return Promise.reject(\"unexpected EOF\");\n }\n return util.concat([head, rest, data], length + rest.length + 1);\n }\n\n return util.concat([head, rest], rest.length + 1);\n}\n\nexport function Unmarshal(packet: Uint8Array): msg.Message {\n let data = new DataView(packet.buffer);\n switch (packet[0]) {\n case msg.CloseID:\n return {\n ID: packet[0],\n channelID: data.getUint32(1)\n } as msg.CloseMessage;\n case msg.DataID:\n let dataLength = data.getUint32(5);\n let rest = new Uint8Array(packet.buffer.slice(9));\n return {\n ID: packet[0],\n channelID: data.getUint32(1),\n length: dataLength,\n data: rest,\n } as msg.DataMessage;\n case msg.EofID:\n return {\n ID: packet[0],\n channelID: data.getUint32(1)\n } as msg.EOFMessage;\n case msg.OpenID:\n return {\n ID: packet[0],\n senderID: data.getUint32(1),\n windowSize: data.getUint32(5),\n maxPacketSize: data.getUint32(9),\n } as msg.OpenMessage;\n case msg.OpenConfirmID:\n return {\n ID: packet[0],\n channelID: data.getUint32(1),\n senderID: data.getUint32(5),\n windowSize: data.getUint32(9),\n maxPacketSize: data.getUint32(13),\n } as msg.OpenConfirmMessage;\n case msg.OpenFailureID:\n return {\n ID: packet[0],\n channelID: data.getUint32(1),\n } as msg.OpenFailureMessage;\n case msg.WindowAdjustID:\n return {\n ID: packet[0],\n channelID: data.getUint32(1),\n additionalBytes: data.getUint32(5),\n } as msg.WindowAdjustMessage;\n default:\n throw `unmarshal of unknown type: ${packet[0]}`;\n }\n}\n" }, { "path": "typescript/codec/encoder.ts", "content": "// @ts-ignore\nimport * as api from \"../api.ts\";\n// @ts-ignore\nimport * as msg from \"./message.ts\";\n\nexport class Encoder {\n conn: api.IConn;\n debug: boolean;\n\n constructor(conn: api.IConn, debug: boolean = false) {\n this.conn = conn;\n this.debug = debug;\n }\n\n async encode(m: msg.AnyMessage): Promise {\n if (this.debug) {\n console.log(\"<<\", m);\n }\n return this.conn.write(Marshal(m));\n }\n}\n\nexport function Marshal(obj: msg.AnyMessage): Uint8Array {\n if (obj.ID === msg.CloseID) {\n let m = obj as msg.CloseMessage;\n let data = new DataView(new ArrayBuffer(5));\n data.setUint8(0, m.ID);\n data.setUint32(1, m.channelID);\n return new Uint8Array(data.buffer);\n }\n if (obj.ID === msg.DataID) {\n let m = obj as msg.DataMessage;\n let data = new DataView(new ArrayBuffer(9));\n data.setUint8(0, m.ID);\n data.setUint32(1, m.channelID);\n data.setUint32(5, m.length);\n let buf = new Uint8Array(9 + m.length);\n buf.set(new Uint8Array(data.buffer), 0);\n buf.set(m.data, 9);\n return buf;\n }\n if (obj.ID === msg.EofID) {\n let m = obj as msg.EOFMessage;\n let data = new DataView(new ArrayBuffer(5));\n data.setUint8(0, m.ID);\n data.setUint32(1, m.channelID);\n return new Uint8Array(data.buffer);\n }\n if (obj.ID === msg.OpenID) {\n let m = obj as msg.OpenMessage;\n let data = new DataView(new ArrayBuffer(13));\n data.setUint8(0, m.ID);\n data.setUint32(1, m.senderID);\n data.setUint32(5, m.windowSize);\n data.setUint32(9, m.maxPacketSize);\n return new Uint8Array(data.buffer);\n }\n if (obj.ID === msg.OpenConfirmID) {\n let m = obj as msg.OpenConfirmMessage;\n let data = new DataView(new ArrayBuffer(17));\n data.setUint8(0, m.ID);\n data.setUint32(1, m.channelID);\n data.setUint32(5, m.senderID);\n data.setUint32(9, m.windowSize);\n data.setUint32(13, m.maxPacketSize);\n return new Uint8Array(data.buffer);\n }\n if (obj.ID === msg.OpenFailureID) {\n let m = obj as msg.OpenFailureMessage;\n let data = new DataView(new ArrayBuffer(5));\n data.setUint8(0, m.ID);\n data.setUint32(1, m.channelID);\n return new Uint8Array(data.buffer);\n }\n if (obj.ID === msg.WindowAdjustID) {\n let m = obj as msg.WindowAdjustMessage;\n let data = new DataView(new ArrayBuffer(9));\n data.setUint8(0, m.ID);\n data.setUint32(1, m.channelID);\n data.setUint32(5, m.additionalBytes);\n return new Uint8Array(data.buffer);\n }\n throw `marshal of unknown type: ${obj}`;\n}\n" }, { "path": "typescript/codec/index.ts", "content": "// @ts-ignore\nexport * from \"./message.ts\";\n// @ts-ignore\nexport * from \"./encoder.ts\";\n// @ts-ignore\nexport * from \"./decoder.ts\";\n" }, { "path": "typescript/codec/message.ts", "content": "\nexport const OpenID = 100;\nexport const OpenConfirmID = 101;\nexport const OpenFailureID = 102;\nexport const WindowAdjustID = 103;\nexport const DataID = 104;\nexport const EofID = 105;\nexport const CloseID = 106;\n\nexport var payloadSizes = new Map([\n [OpenID, 12],\n [OpenConfirmID, 16],\n [OpenFailureID, 4],\n [WindowAdjustID, 8],\n [DataID, 8],\n [EofID, 4],\n [CloseID, 4],\n]);\n\nexport interface Message {\n ID: number;\n}\n\nexport interface OpenMessage {\n ID: 100;\n senderID: number;\n windowSize: number;\n maxPacketSize: number;\n}\n\nexport interface OpenConfirmMessage {\n ID: 101;\n channelID: number;\n senderID: number;\n windowSize: number;\n maxPacketSize: number;\n}\n\nexport interface OpenFailureMessage {\n ID: 102;\n channelID: number;\n}\n\nexport interface WindowAdjustMessage {\n ID: 103;\n channelID: number;\n additionalBytes: number;\n}\n\nexport interface DataMessage {\n ID: 104;\n channelID: number;\n length: number;\n data: Uint8Array;\n}\n\nexport interface EOFMessage {\n ID: 105;\n channelID: number;\n}\n\nexport interface CloseMessage {\n ID: 106;\n channelID: number;\n}\n\nexport type ChannelMessage = (\n OpenConfirmMessage |\n OpenFailureMessage |\n WindowAdjustMessage |\n DataMessage |\n EOFMessage |\n CloseMessage);\n\nexport type AnyMessage = ChannelMessage | OpenMessage;\n" }, { "path": "typescript/index.ts", "content": "// https://github.com/Microsoft/TypeScript/issues/27481\n// @ts-ignore\nexport * from \"./internal.ts\";\n// @ts-ignore\nexport * from \"./transport/websocket.ts\";\n" }, { "path": "typescript/internal.ts", "content": "// @ts-ignore\nexport * from \"./session.ts\";\n// @ts-ignore\nexport * from \"./channel.ts\";\n" }, { "path": "typescript/session.ts", "content": "// @ts-ignore\nimport * as api from \"./api.ts\";\n// @ts-ignore\nimport * as codec from \"./codec/index.ts\";\n// @ts-ignore\nimport * as util from \"./util.ts\";\n// @ts-ignore\nimport * as internal from \"./internal.ts\";\n\nexport const minPacketLength = 9;\nexport const maxPacketLength = Number.MAX_VALUE;\n\n\nexport class Session implements api.ISession {\n conn: api.IConn;\n channels: Array;\n incoming: util.queue;\n enc: codec.Encoder;\n dec: codec.Decoder;\n done: Promise;\n\n constructor(conn: api.IConn, debug: boolean = false) {\n this.conn = conn;\n this.enc = new codec.Encoder(conn, debug);\n this.dec = new codec.Decoder(conn, debug);\n this.channels = [];\n this.incoming = new util.queue();\n this.done = this.loop();\n }\n\n async open(): Promise {\n let ch = this.newChannel();\n ch.maxIncomingPayload = internal.channelMaxPacket;\n await this.enc.encode({\n ID: codec.OpenID,\n windowSize: ch.myWindow,\n maxPacketSize: ch.maxIncomingPayload,\n senderID: ch.localId\n });\n if (await ch.ready.shift()) {\n return ch;\n }\n throw \"failed to open\";\n }\n\n accept(): Promise {\n return this.incoming.shift();\n }\n\n async close(): Promise {\n for (const ids of Object.keys(this.channels)) {\n let id = parseInt(ids);\n if (this.channels[id] !== undefined) {\n this.channels[id].shutdown();\n }\n }\n await this.conn.close();\n await this.done;\n }\n\n async loop() {\n try {\n while (true) {\n let msg = await this.dec.decode();\n if (msg === undefined) {\n this.close();\n return;\n }\n if (msg.ID === codec.OpenID) {\n await this.handleOpen(msg as codec.OpenMessage);\n continue;\n }\n\n let cmsg: codec.ChannelMessage = msg as codec.ChannelMessage;\n\n let ch = this.getCh(cmsg.channelID);\n if (ch === undefined) {\n throw `invalid channel (${cmsg.channelID}) on op ${cmsg.ID}`;\n }\n await ch.handle(cmsg);\n }\n } catch (e) {\n throw new Error(`session readloop: ${e}`);\n }\n // catch {\n // \tthis.channels.forEach(async (ch) => {\n // \t\tawait ch.close();\n // \t})\n // \tthis.channels = [];\n // \tawait this.conn.close();\n // }\n }\n\n async handleOpen(msg: codec.OpenMessage) {\n if (msg.maxPacketSize < minPacketLength || msg.maxPacketSize > maxPacketLength) {\n await this.enc.encode({\n ID: codec.OpenFailureID,\n channelID: msg.senderID\n });\n return;\n }\n let c = this.newChannel();\n c.remoteId = msg.senderID;\n c.maxRemotePayload = msg.maxPacketSize;\n c.remoteWin = msg.windowSize;\n c.maxIncomingPayload = internal.channelMaxPacket;\n this.incoming.push(c);\n await this.enc.encode({\n ID: codec.OpenConfirmID,\n channelID: c.remoteId,\n senderID: c.localId,\n windowSize: c.myWindow,\n maxPacketSize: c.maxIncomingPayload\n });\n }\n\n newChannel(): internal.Channel {\n let ch = new internal.Channel(this);\n ch.remoteWin = 0;\n ch.myWindow = internal.channelWindowSize;\n ch.localId = this.addCh(ch);\n return ch;\n }\n\n getCh(id: number): internal.Channel {\n let ch = this.channels[id];\n if (ch && ch.localId !== id) {\n console.log(\"bad ids:\", id, ch.localId, ch.remoteId);\n }\n return ch;\n }\n\n addCh(ch: internal.Channel): number {\n this.channels.forEach((v, i) => {\n if (v === undefined) {\n this.channels[i] = ch;\n return i;\n }\n });\n this.channels.push(ch);\n return this.channels.length - 1;\n }\n\n rmCh(id: number): void {\n delete this.channels[id];\n }\n\n}\n\n" }, { "path": "typescript/session_test.ts", "content": "import {\n assertEquals,\n} from \"https://deno.land/std/testing/asserts.ts\";\n\nimport * as session from \"./session.ts\";\nimport * as api from \"./api.ts\";\nimport * as util from \"./util.ts\";\nimport * as tcp from \"./transport/deno/tcp.ts\";\nimport * as websocket from \"./transport/deno/websocket.ts\";\n\nasync function readAll(conn: api.IConn): Promise {\n let buff = new Uint8Array();\n while (true) {\n let next = await conn.read(100);\n if (next === undefined) {\n return buff;\n }\n buff = util.concat([buff, next], buff.byteLength + next.byteLength);\n }\n}\n\nasync function startListener(listener: api.IConnListener) {\n let conn = await listener.accept();\n if (!conn) {\n throw new Error(\"accept failed\")\n }\n let sess = new session.Session(conn);\n let ch = await sess.open();\n let b = await readAll(ch);\n await ch.close();\n\n let ch2 = await sess.accept();\n if (ch2 === undefined) {\n throw new Error(\"accept failed\")\n }\n await ch2.write(b);\n await ch2.close();\n try {\n await sess.close();\n await listener.close();\n } catch (e) {\n console.log(e);\n }\n}\n\nasync function testExchange(conn: api.IConn) {\n let sess = new session.Session(conn);\n let ch = await sess.accept();\n if (ch === undefined) {\n throw new Error(\"accept failed\")\n }\n\n await ch.write(new TextEncoder().encode(\"Hello world\"));\n await ch.closeWrite();\n await ch.close();\n\n let ch2 = await sess.open();\n let b = await readAll(ch2);\n await ch2.close();\n\n assertEquals(new TextEncoder().encode(\"Hello world\"), b);\n try {\n await sess.close();\n } catch (e) {\n console.log(e);\n }\n}\n\nDeno.test(\"tcp\", async () => {\n let listener = new tcp.Listener({ port: 0 });\n let port = (listener.listener.addr as Deno.NetAddr).port;\n await Promise.all([\n startListener(listener),\n tcp.Dial({ port }).then(conn => {\n return testExchange(conn);\n }),\n ]);\n});\n\nDeno.test(\"websocket\", async () => {\n let endpoint = \"ws://127.0.0.1:9999\";\n let listener = new websocket.Listener(9999);\n await Promise.all([\n startListener(listener),\n websocket.Dial(endpoint).then(conn => {\n return testExchange(conn);\n }),\n ]);\n});\n\n\nDeno.test(\"multiple pending reads\", async () => {\n let listener = Deno.listen({ port: 0 });\n\n let port = (listener.addr as Deno.NetAddr).port;\n\n let lConn = listener.accept();\n\n let sess1 = new session.Session(new tcp.Conn(await Deno.connect({ port })));\n let sess2 = new session.Session(new tcp.Conn(await lConn));\n\n let ch1p = sess1.accept();\n let ch2 = await sess2.open();\n let ch1 = await ch1p;\n if (ch1 === undefined) {\n throw new Error(\"accept failed\");\n }\n\n let a = ch1.read(1);\n let bc = ch1.read(2);\n\n await ch2.write(new TextEncoder().encode(\"abc\"));\n\n assertEquals(await a, new TextEncoder().encode(\"a\"))\n assertEquals(await bc, new TextEncoder().encode(\"bc\"))\n\n await ch2.closeWrite();\n await ch2.close();\n await sess2.close();\n\n await ch1.close();\n await sess1.close();\n\n listener.close();\n});\n" }, { "path": "typescript/transport/deno/tcp.ts", "content": "// @ts-ignore\nimport * as api from \"../../api.ts\";\n\n\nexport class Listener implements api.IConnListener {\n listener: Deno.Listener;\n\n constructor(opts: Deno.ListenOptions) {\n this.listener = Deno.listen(opts);\n }\n\n async accept(): Promise {\n return new Conn(await this.listener.accept());\n }\n\n close(): Promise {\n this.listener.close();\n return Promise.resolve();\n }\n}\n\nexport async function Dial(opts: Deno.ConnectOptions): Promise {\n return new Conn(await Deno.connect(opts));\n}\n\nexport class Conn implements api.IConn {\n conn: Deno.Conn;\n\n constructor(conn: Deno.Conn) {\n this.conn = conn;\n }\n\n async read(len: number): Promise {\n let buff = new Uint8Array(len);\n let n: number | null;\n try {\n n = await this.conn.read(buff);\n } catch (e) {\n if (e instanceof Deno.errors.Interrupted || e instanceof Deno.errors.BadResource) {\n return undefined;\n }\n throw e;\n }\n if (n == null) {\n return undefined;\n }\n if (buff.byteLength > n) {\n buff = buff.slice(0, n);\n }\n return buff;\n }\n\n write(buffer: Uint8Array): Promise {\n return this.conn.write(buffer)\n }\n\n close(): Promise {\n try {\n this.conn.close();\n } catch (e) {\n if (!(e instanceof Deno.errors.BadResource)) {\n throw e;\n }\n }\n return Promise.resolve();\n }\n}\n" }, { "path": "typescript/transport/deno/websocket.ts", "content": "import { StandardWebSocketClient, WebSocketClient, WebSocketServer } from \"https://deno.land/x/websocket@v0.1.2/mod.ts\";\n\n// @ts-ignore\nimport * as api from \"./../../api.ts\";\n// @ts-ignore\nimport * as internal from \"./../../internal.ts\";\n// @ts-ignore\nimport * as util from \"./../../util.ts\";\n\nexport class Listener implements api.IConnListener {\n wss: WebSocketServer\n q: util.queue\n\n constructor(port: number) {\n this.q = new util.queue();\n this.wss = new WebSocketServer(port);\n this.wss.on(\"connection\", (ws: WebSocketClient) => {\n this.q.push(new Conn(ws));\n })\n }\n\n accept(): Promise {\n return this.q.shift();\n }\n\n async close(): Promise {\n await this.wss.close();\n this.q.close();\n }\n}\n\nexport function Dial(endpoint: string): Promise {\n let ws = new StandardWebSocketClient(endpoint);\n return new Promise((resolve) => {\n // TODO errors?\n ws.on(\"open\", function () {\n resolve(new Conn(ws));\n });\n })\n}\n\nexport class Conn implements api.IConn {\n socket: WebSocketClient\n buf: util.ReadBuffer\n isClosed: boolean\n\n constructor(socket: WebSocketClient) {\n this.isClosed = false;\n this.socket = socket;\n this.buf = new util.ReadBuffer();\n this.socket.on(\"message\", (event: MessageEvent | Uint8Array) => {\n if (event instanceof Uint8Array) {\n this.buf.write(event);\n return;\n }\n event.data.arrayBuffer().then((data) => {\n let buf = new Uint8Array(data);\n this.buf.write(buf);\n });\n });\n this.socket.on(\"close\", () => {\n this.close();\n });\n //this.socket.onerror = (err) => console.error(\"qtalk\", err);\n }\n\n read(len: number): Promise {\n return this.buf.read(len);\n }\n\n write(buffer: Uint8Array): Promise {\n this.socket.send(buffer);\n return Promise.resolve(buffer.byteLength);\n }\n\n async close(): Promise {\n if (this.isClosed) {\n return;\n }\n this.isClosed = true;\n this.buf.close();\n await this.socket.close(1000); // Code 1000: Normal Closure\n }\n}\n" }, { "path": "typescript/transport/websocket.ts", "content": "// @ts-ignore\nimport * as api from \"./../api.ts\";\n// @ts-ignore\nimport * as internal from \"./../internal.ts\";\n// @ts-ignore\nimport * as util from \"./../util.ts\";\n\nexport function Dial(addr: string, debug: boolean = false, onclose?: () => void): Promise {\n return new Promise((resolve) => {\n var socket = new WebSocket(addr);\n socket.onopen = () => resolve(new internal.Session(new Conn(socket), debug));\n //socket.onerror = (err) => console.error(\"qtalk\", err);\n if (onclose) socket.onclose = onclose;\n })\n}\n\nexport class Conn implements api.IConn {\n socket: WebSocket\n error: any\n waiters: Array<() => void>\n buf: Uint8Array;\n isClosed: boolean\n\n constructor(socket: WebSocket) {\n this.isClosed = false;\n this.buf = new Uint8Array(0);\n this.waiters = [];\n this.socket = socket;\n this.socket.binaryType = \"arraybuffer\";\n this.socket.onmessage = (event) => {\n var buf = new Uint8Array(event.data);\n this.buf = util.concat([this.buf, buf], this.buf.length + buf.length);\n if (this.waiters.length > 0) {\n let waiter = this.waiters.shift();\n if (waiter) waiter();\n }\n };\n let onclose = this.socket.onclose;\n this.socket.onclose = (e: CloseEvent) => {\n if (onclose) onclose.bind(this.socket)(e);\n this.close();\n }\n //this.socket.onerror = (err) => console.error(\"qtalk\", err);\n }\n\n read(len: number): Promise {\n return new Promise((resolve) => {\n var tryRead = () => {\n if (this.isClosed) {\n resolve(undefined);\n return;\n }\n if (this.buf.length >= len) {\n var data = this.buf.slice(0, len);\n this.buf = this.buf.slice(len);\n resolve(data);\n return;\n }\n this.waiters.push(tryRead);\n }\n tryRead();\n })\n }\n\n write(buffer: Uint8Array): Promise {\n this.socket.send(buffer);\n return Promise.resolve(buffer.byteLength);\n }\n\n close(): Promise {\n if (this.isClosed) return Promise.resolve();\n return new Promise((resolve) => {\n this.isClosed = true;\n this.waiters.forEach(waiter => waiter());\n this.socket.close();\n resolve();\n });\n }\n}\n" }, { "path": "typescript/tsconfig.json", "content": "{\n \"compilerOptions\": {\n \"lib\": [\"es2016\", \"dom\", \"es5\"],\n \"noImplicitAny\": true,\n \"allowJs\": true,\n }\n}\n" }, { "path": "typescript/util.ts", "content": "\nexport function concat(list: Uint8Array[], totalLength: number): Uint8Array {\n let buf = new Uint8Array(totalLength);\n let offset = 0;\n list.forEach((el) => {\n buf.set(el, offset);\n offset += el.length;\n });\n return buf;\n}\n\n// queue primitive for incoming connections and\n// signaling channel ready state\nexport class queue {\n q: Array\n waiters: Array<(a: ValueType | undefined) => void>\n closed: boolean\n\n constructor() {\n this.q = [];\n this.waiters = [];\n this.closed = false;\n }\n\n push(obj: ValueType) {\n if (this.closed) throw \"closed queue\";\n if (this.waiters.length > 0) {\n let waiter = this.waiters.shift()\n if (waiter) waiter(obj);\n return;\n }\n this.q.push(obj);\n }\n\n shift(): Promise {\n if (this.closed) return Promise.resolve(undefined);\n return new Promise(resolve => {\n if (this.q.length > 0) {\n resolve(this.q.shift());\n return;\n }\n this.waiters.push(resolve);\n })\n }\n\n close() {\n if (this.closed) return;\n this.closed = true;\n this.waiters.forEach(waiter => {\n waiter(undefined);\n });\n }\n}\n\nexport class ReadBuffer {\n gotEOF: boolean;\n readBuf: Uint8Array | undefined;\n readers: Array<() => void>;\n\n constructor() {\n this.readBuf = new Uint8Array(0);\n this.gotEOF = false;\n this.readers = [];\n }\n\n read(len: number): Promise {\n return new Promise(resolve => {\n let tryRead = () => {\n if (this.readBuf === undefined) {\n resolve(undefined);\n return;\n }\n if (this.readBuf.length == 0) {\n if (this.gotEOF) {\n this.readBuf = undefined;\n resolve(undefined);\n return;\n }\n this.readers.push(tryRead);\n return;\n }\n let data = this.readBuf.slice(0, len);\n this.readBuf = this.readBuf.slice(data.byteLength);\n if (this.readBuf.length == 0 && this.gotEOF) {\n this.readBuf = undefined;\n }\n resolve(data);\n }\n tryRead();\n });\n }\n\n write(data: Uint8Array) {\n if (this.readBuf) {\n this.readBuf = concat([this.readBuf, data], this.readBuf.length + data.length);\n }\n\n while (!this.readBuf || this.readBuf.length > 0) {\n let reader = this.readers.shift();\n if (!reader) break\n reader();\n }\n }\n\n eof() {\n this.gotEOF = true;\n this.flushReaders();\n }\n\n close() {\n this.readBuf = undefined;\n this.flushReaders();\n }\n\n protected flushReaders() {\n while (true) {\n let reader = this.readers.shift();\n if (reader === undefined) {\n return;\n }\n reader();\n }\n }\n}\n" } ]