[
  {
    "path": ".gitignore",
    "content": ".idea/\ncorpus/\ncrashers/\nsuppressions/\nreliable-fuzz.zip\n"
  },
  {
    "path": "LICENSE",
    "content": "MIT License\n\nCopyright (c) 2020 Kenta Iwasaki <kenta@lithdew.net>\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n"
  },
  {
    "path": "README.md",
    "content": "# reliable\n\n[![MIT License](https://img.shields.io/apm/l/atomic-design-ui.svg?)](LICENSE)\n[![go.dev reference](https://img.shields.io/badge/go.dev-reference-007d9c?logo=go&logoColor=white&style=flat-square)](https://pkg.go.dev/github.com/lithdew/reliable)\n[![Discord Chat](https://img.shields.io/discord/697002823123992617)](https://discord.gg/HZEbkeQ)\n\n**reliable** is a reliability layer for UDP connections in Go.\n\nWith only 9 bytes of packet overhead at most, what **reliable** does for your UDP-based application is:\n\n1. handle acknowledgement over the recipient of packets you sent,\n2. handle sending acknowledgements when too many are being buffered up,\n3. handle resending sent packets whose recipient hasn't been acknowledged after some timeout, and\n4. handle stopping/buffering up packets to be sent when the recipients read buffer is suspected to be full.\n\n** This project is still a WIP! Scrub through the source code, write some unit tests, help out with documentation, or open up a Github issue if you would like to help out or have any questions!\n\n## Protocol\n\n### Packet Header\n\n**reliable** uses the same packet header layout described in [`networkprotocol/reliable.io`](https://github.com/networkprotocol/reliable.io).\n\nAll packets start with a single byte (8 bits) representing 8 different flags. Packets are sequential, and are numbered using an unsigned 16-bit integer included in the packet header unless the packet is marked to be unreliable.\n\nPacket acknowledgements (ACKs) are redundantly included in every sent packet using a total of 5 bytes: two bytes representing an unsigned 16-bit packet sequence number (ack), and three bytes representing a 32-bit bitfield (ackBits).\n\nThe packet header layout, much like [`networkprotocol/reliable.io`](https://github.com/networkprotocol/reliable.io), is delta-encoded and RLE-encoded to reduce the size overhead per packet.\n\n### Packet ACKs\n\nGiven a packet we have just received from our peer, for each set bit (i) in the bitfield (ackBits), we mark a packet we have sent to be acknowledged if its sequence number is (ack - i).\n\nIn the case of peer A sending packets to B, with B not sending any packets at all to A, B will send an empty packet for every 32 packets received from A so that A will be aware that B has acknowledged its packets.\n\nMore explicitly, a counter (lui) is maintained representing the last consecutive packet sequence number that we have received whose acknowledgement we have told to our peer about.\n\nFor example, if (lui) is 0, and we have sent acknowledgements for packets whose sequence numbers are 2, 3, 4, and 6, and we have then acknowledged packet sequence number 1, then lui would be 4.\n\nUpon updating (lui), if the next 32 consecutive sequence numbers are sequence numbers of packets we have previously received, we will increment (lui) by 32 and send a single empty packet containing the following packet acknowledgements: (ack=lui+31, ackBits=[lui,lui+31]).\n\n### Packet Buffering\n\nTwo fixed-sized sequence buffers are maintained for packets that we have sent (wq), and packets that we have received (rq). The size fixed for these buffers must evenly divide into the max value of an unsigned 16-bit integer (65536). The data structure is described in [this blog post by Glenn Fiedler](https://gafferongames.com/post/reliable_ordered_messages/).\n\nWe keep track of a counter (oui), representing the last consecutive sequence number of a packet we have sent that was acknowledged by our peer. For example, if we have sent packets whose sequence numbers are in the range [0, 256], and we have received acknowledgements for packets (0, 1, 2, 3, 4, 8, 9, 10, 11, 12), then (oui) would be 4.\n\nLet cap(q) be the fixed size or capacity of sequence buffer q.\n\nWhile sending packets, we intermittently stop and buffer the sending of packets if we believe sending more packets would overflow the read buffer of our recipient. More explicitly, if the next packet we sent is assigned a packet number greater than (oui + cap(rq)), we stop all sends until (oui) has incremented through the recipient of a packet from our peer.\n\n### Retransmitting Lost Packets\n\nThe logic for retransmitting stale, unacknowledged sent packets and maintaining acknowledgements was taken with credit to [this blog post by Glenn Fiedler](https://gafferongames.com/post/reliable_ordered_messages/).\n\nPackets are suspected to be lost if they are not acknowledged by their recipient after 100ms. Once a packet is suspected to be lost, it is resent. As of right now, packets are resent for a maximum of 10 times.\n\nIt might be wise to not allow packets to be resent a capped number of times, and to leave it up to the developer. However, that is open for discussion which I am happy to have over on my Discord server or through a Github issue.\n\n## Rationale\n\nOn my quest for finding a feasible solution against TCP head-of-line blocking, I looked through a _lot_ of reliable UDP libraries in Go, with the majority primarily suited for either file transfer or gaming:\n\n1. A direct port of the reference C implementation of [networkprotocol/reliable.io](https://github.com/networkprotocol/reliable.io): [jakecoffman/rely](https://github.com/jakecoffman/rely)\n2. A realtime multiplayer network gaming protocol: [obsilp/rmnp](https://github.com/obsilp/rmnp)\n3. A reliable, production-grade ARQ protocol: [xtaci/kcp-go](https://github.com/xtaci/kcp-go)\n4. An encrypted session-based streaming protocol: [ooclab/es](https://github.com/ooclab/es/tree/master/proto/udp)\n5. A game networking protocol: [arl/udpnet](https://github.com/arl/udpnet)\n6. A direct port of uTP (Micro Transport Protocol): [warjiang/utp](https://github.com/warjiang/utp/tree/master/utp)\n7. A protocol for sending arbitrarily large, chunked amounts of data: [go-guoyk/sptp](https://github.com/go-guoyk/sptp)\n8. A small-scale fast transmission protocol: [spance/suft](https://github.com/spance/suft/)\n9. A direct port of QUIC: [lucas-clemente/quic-go](https://github.com/lucas-clemente/quic-go)\n\nGoing through all of them, I felt that they did just a little too much for me. For my work and side projects, I have been working heavily on decentralized p2p networking protocols. The nature of these protocols is that they suffer heavily from TCP head-of-line blocking operating in high-latency/high packet loss environments.\n\nIn many cases, a lot of the features provided by these libraries were either not needed, or honestly felt like they would best be handled and thought through by the developer using these libraries. For example:\n \n1. handshaking/session management\n2. packet fragmentation/reassembly\n3. packet encryption/decryption\n\nSo, I began working on a modular approach and decided to abstract away the reliability portion of protocols I have built into a separate library.\n\nI feel that this approach is best versus the popular alternatives like QUIC or SCTP that may, depending on your circumstances, do just a bit too much for you. After all, getting _just_ the reliability bits of a UDP-based protocol correct and well-tested is hard enough.\n\n## Todo\n\n1. Estimate the round-trip time (RTT) and adjust the system's packet re-transmission delay based off of it.\n2. Encapsulate away protocol logic and `net.PacketConn`-related bits for a finer abstraction.\n3. Keep a cache of the string representations of passed-in `net.UDPAddr`.\n4. Reduce locking in as many code hot paths as possible.\n5. Networking statistics (packet loss, RTT, etc.).\n6. More unit tests.\n\n## Usage\n\n**reliable** uses Go modules. To include it in your project, run the following command:\n\n```\n$ go get github.com/lithdew/reliable\n```\n\nShould you just be looking to quickly get a project or demo up and running, use `Endpoint`. If you require more flexibility, consider directly working with `Conn`.\n\nNote that some sort of keep-alive mechanism or heartbeat system needs to be bootstrapped on top, otherwise packets may indefinitely be resent as they will have failed to be acknowledged. \n\n## Options\n\n1. The read buffer size may be configured using `WithReadBufferSize`. The default read buffer size is 256.\n2. The write buffer size may be configured using `WithWriteBufferSize`. The default write buffer size is 256.\n3. The minimum period of time before we retransmit an packet that has yet to be acknowledged may be configured using `WithResendTimeout`. The default resend timeout is 100 milliseconds.\n4. A packet handler which is to be called back when a packet is received may be configured using `WithEndpointPacketHandler` or `WithProtocolPacketHandler`. By default, a nil handler is provided which ignores all incoming packets.\n5. An error handler which is called when errors occur on a connection that may be configured using `WithEndpointErrorHandler` or `WithProtocolErrorHandler`. By default, a nil handler is provided which ignores all errors.\n6. A byte buffer pool may be passed in using `WithBufferPool`. By default, a new byte buffer pool is instantiated.\n\n## Benchmarks\n\nA benchmark was done using [`cmd/benchmark`](examples/benchmark) from Japan to a DigitalOcean 2GB / 60 GB Disk / NYC3 server.\n\nThe benchmark task was to spam 1400 byte packets from Japan to New York. Given a ping latency of roughly 220 milliseconds, the throughput was roughly 1.2 MiB/sec.\n\nUnit test benchmarks have also been performed, as shown below.\n\n```\n$ cat /proc/cpuinfo | grep 'model name' | uniq\nmodel name : Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz\n\n$ go test -bench=. -benchtime=10s\ngoos: linux\ngoarch: amd64\npkg: github.com/lithdew/reliable\nBenchmarkEndpointWriteReliablePacket-8           2053717              5941 ns/op             183 B/op          9 allocs/op\nBenchmarkEndpointWriteUnreliablePacket-8         2472392              4866 ns/op             176 B/op          8 allocs/op\nBenchmarkMarshalPacketHeader-8                  749060137               15.7 ns/op             0 B/op          0 allocs/op\nBenchmarkUnmarshalPacketHeader-8                835547473               14.6 ns/op             0 B/op          0 allocs/op\n```\n\n## Example\n\nYou may run the example below by executing the following command:\n\n```\n$ go run github.com/lithdew/reliable/examples/basic\n```\n\nThis example demonstrates:\n\n1. how to quickly construct two UDP endpoints listening on ports 44444 and 55555, and\n2. how to have the UDP endpoint at port 44444 spam 1400-byte packets to the UDP endpoint at port 55555 as fast as possible.\n\n```go\npackage main\n\nimport (\n\t\"bytes\"\n\t\"errors\"\n\t\"github.com/davecgh/go-spew/spew\"\n\t\"github.com/lithdew/reliable\"\n\t\"io\"\n\t\"log\"\n\t\"net\"\n\t\"os\"\n\t\"os/signal\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t\"time\"\n)\n\nvar (\n\tPacketData = bytes.Repeat([]byte(\"x\"), 1400)\n\tNumPackets = uint64(0)\n)\n\nfunc check(err error) {\n\tif err != nil && !errors.Is(err, io.EOF) {\n\t\tlog.Panic(err)\n\t}\n}\n\nfunc listen(addr string) net.PacketConn {\n\tconn, err := net.ListenPacket(\"udp\", addr)\n\tcheck(err)\n\treturn conn\n}\n\nfunc handler(buf []byte, _ net.Addr) {\n\tif bytes.Equal(buf, PacketData) || len(buf) == 0 {\n\t\treturn\n\t}\n\tspew.Dump(buf)\n\tos.Exit(1)\n}\n\nfunc main() {\n\texit := make(chan struct{})\n\n\tvar wg sync.WaitGroup\n\twg.Add(2)\n\n\tca := listen(\"127.0.0.1:44444\")\n\tcb := listen(\"127.0.0.1:55555\")\n\n\ta := reliable.NewEndpoint(ca, reliable.WithEndpointPacketHandler(handler))\n\tb := reliable.NewEndpoint(cb, reliable.WithEndpointPacketHandler(handler))\n\n\tdefer func() {\n\t\tcheck(ca.SetDeadline(time.Now().Add(1 * time.Millisecond)))\n\t\tcheck(cb.SetDeadline(time.Now().Add(1 * time.Millisecond)))\n\n\t\tclose(exit)\n\n\t\tcheck(a.Close())\n\t\tcheck(b.Close())\n\n\t\tcheck(ca.Close())\n\t\tcheck(cb.Close())\n\n\t\twg.Wait()\n\t}()\n\n\tgo a.Listen()\n\tgo b.Listen()\n\n\t// The two goroutines below have endpoint A spam endpoint B, and print out how\n\t// many packets of data are being sent per second.\n\n\tgo func() {\n\t\tdefer wg.Done()\n\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-exit:\n\t\t\t\treturn\n\t\t\tdefault:\n\t\t\t}\n\n\t\t\tcheck(a.WriteReliablePacket(PacketData, b.Addr()))\n\t\t\tatomic.AddUint64(&NumPackets, 1)\n\t\t}\n\t}()\n\n\tgo func() {\n\t\tdefer wg.Done()\n\n\t\tticker := time.NewTicker(1 * time.Second)\n\t\tdefer ticker.Stop()\n\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-exit:\n\t\t\t\treturn\n\t\t\tcase <-ticker.C:\n\t\t\t\tnumPackets := atomic.SwapUint64(&NumPackets, 0)\n\t\t\t\tnumBytes := float64(numPackets) * 1400.0 / 1024.0 / 1024.0\n\n\t\t\t\tlog.Printf(\n\t\t\t\t\t\"Sent %d packet(s) comprised of %.2f MiB worth of data.\",\n\t\t\t\t\tnumPackets,\n\t\t\t\t\tnumBytes,\n\t\t\t\t)\n\t\t\t}\n\t\t}\n\t}()\n\n\tch := make(chan os.Signal, 1)\n\tsignal.Notify(ch, os.Interrupt)\n\t<-ch\n}\n```"
  },
  {
    "path": "conn.go",
    "content": "package reliable\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"net\"\n)\n\ntype transmitFunc func(buf []byte) (bool, error)\n\ntype Conn struct {\n\taddr     net.Addr\n\tconn     net.PacketConn\n\tprotocol *Protocol\n}\n\nfunc NewConn(addr net.Addr, conn net.PacketConn, opts ...ProtocolOption) *Conn {\n\tp := NewProtocol(opts...)\n\treturn &Conn{addr: addr, conn: conn, protocol: p}\n}\n\nfunc (c *Conn) WriteReliablePacket(buf []byte) error {\n\tbuf, err := c.protocol.WritePacket(true, buf)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t_, err = c.transmit(buf)\n\treturn err\n}\n\nfunc (c *Conn) WriteUnreliablePacket(buf []byte) error {\n\tbuf, err := c.protocol.WritePacket(false, buf)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t_, err = c.transmit(buf)\n\treturn err\n}\n\nfunc (c *Conn) Read(header PacketHeader, buf []byte) error {\n\tbuf = c.protocol.ReadPacket(header, buf)\n\n\tif len(buf) != 0 {\n\t\t_, err := c.transmit(buf)\n\t\treturn err\n\t}\n\n\treturn nil\n}\n\nfunc (c *Conn) Close() {\n\tc.protocol.Close()\n}\n\nfunc (c *Conn) Run() {\n\tc.protocol.Run(c.transmit)\n}\n\nfunc (c *Conn) transmit(buf []byte) (EOF bool, err error) {\n\tn, err := c.conn.WriteTo(buf, c.addr)\n\n\tif err == nil && n != len(buf) {\n\t\terr = io.ErrShortWrite\n\t}\n\n\tEOF = isEOF(err)\n\n\tif err != nil && !EOF {\n\t\terr = fmt.Errorf(\"failed to transmit packet: %w\", err)\n\t\treturn\n\t}\n\n\treturn\n}\n"
  },
  {
    "path": "conn_test.go",
    "content": "package reliable\n\nimport (\n\t\"bytes\"\n\t\"github.com/stretchr/testify/require\"\n\t\"go.uber.org/goleak\"\n\t\"math\"\n\t\"net\"\n\t\"sync/atomic\"\n\t\"testing\"\n\t\"time\"\n)\n\nfunc TestConnWriteReliablePacket(t *testing.T) {\n\tdefer goleak.VerifyNone(t)\n\n\tdata := bytes.Repeat([]byte(\"x\"), 1400)\n\n\tactual := uint64(0)\n\texpected := uint64(65536)\n\n\ta, _ := net.ListenPacket(\"udp\", \"127.0.0.1:0\")\n\tb, _ := net.ListenPacket(\"udp\", \"127.0.0.1:0\")\n\n\thandler := func(buf []byte, _ uint16) {\n\t\tatomic.AddUint64(&actual, 1)\n\t\trequire.EqualValues(t, data, buf)\n\t}\n\n\tca := NewConn(a.LocalAddr(), a, WithProtocolPacketHandler(handler))\n\tcb := NewConn(b.LocalAddr(), b, WithProtocolPacketHandler(handler))\n\n\tgo readLoop(t, a, ca)\n\tgo readLoop(t, b, cb)\n\n\tdefer func() {\n\t\trequire.NoError(t, a.SetDeadline(time.Now().Add(1*time.Millisecond)))\n\t\trequire.NoError(t, b.SetDeadline(time.Now().Add(1*time.Millisecond)))\n\n\t\trequire.NoError(t, a.Close())\n\t\trequire.NoError(t, b.Close())\n\n\t\tca.Close()\n\t\tcb.Close()\n\n\t\trequire.EqualValues(t, expected, atomic.LoadUint64(&actual))\n\t}()\n\n\tfor i := uint64(0); i < expected; i++ {\n\t\trequire.NoError(t, ca.WriteReliablePacket(data))\n\t}\n}\n\nfunc readLoop(t *testing.T, pc net.PacketConn, c *Conn) {\n\tvar (\n\t\tn   int\n\t\terr error\n\t)\n\n\tbuf := make([]byte, math.MaxUint16+1)\n\tfor {\n\t\tn, _, err = pc.ReadFrom(buf)\n\t\tif err != nil {\n\t\t\tbreak\n\t\t}\n\n\t\theader, buf, err := UnmarshalPacketHeader(buf[:n])\n\t\trequire.NoError(t, err)\n\n\t\tif err == nil {\n\t\t\terr = c.Read(header, buf)\n\t\t\trequire.NoError(t, err)\n\t\t}\n\t}\n}\n"
  },
  {
    "path": "endpoint.go",
    "content": "package reliable\n\nimport (\n\t\"io\"\n\t\"math\"\n\t\"net\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t\"time\"\n)\n\ntype EndpointPacketHandler func(buf []byte, addr net.Addr)\ntype EndpointErrorHandler func(err error, addr net.Addr)\n\ntype Endpoint struct {\n\twriteBufferSize uint16 // write buffer size that must be a divisor of 65536\n\treadBufferSize  uint16 // read buffer size that must be a divisor of 65536\n\n\tupdatePeriod  time.Duration // how often time-dependant parts of the protocol get checked\n\tresendTimeout time.Duration // how long we wait until unacked packets should be resent\n\n\tmu sync.Mutex\n\twg sync.WaitGroup\n\n\tpool *Pool\n\n\tph EndpointPacketHandler\n\teh EndpointErrorHandler\n\n\taddr  net.Addr\n\tconn  net.PacketConn\n\tconns map[string]*Conn\n\n\tclosing uint32\n}\n\nfunc NewEndpoint(conn net.PacketConn, opts ...EndpointOption) *Endpoint {\n\te := &Endpoint{conn: conn, addr: conn.LocalAddr(), conns: make(map[string]*Conn)}\n\n\tfor _, opt := range opts {\n\t\topt.applyEndpoint(e)\n\t}\n\n\tif e.writeBufferSize == 0 {\n\t\te.writeBufferSize = DefaultWriteBufferSize\n\t}\n\n\tif e.readBufferSize == 0 {\n\t\te.readBufferSize = DefaultReadBufferSize\n\t}\n\n\tif e.resendTimeout == 0 {\n\t\te.resendTimeout = DefaultResendTimeout\n\t}\n\n\tif e.updatePeriod == 0 {\n\t\te.updatePeriod = DefaultUpdatePeriod\n\t}\n\n\tif e.pool == nil {\n\t\te.pool = new(Pool)\n\t}\n\n\treturn e\n}\n\nfunc (e *Endpoint) getConn(addr net.Addr) *Conn {\n\tid := addr.String()\n\n\te.mu.Lock()\n\tdefer e.mu.Unlock()\n\n\tconn := e.conns[id]\n\tif conn == nil {\n\t\tif atomic.LoadUint32(&e.closing) == 1 {\n\t\t\treturn nil\n\t\t}\n\n\t\tconn = NewConn(\n\t\t\taddr,\n\t\t\te.conn,\n\t\t\tWithWriteBufferSize(e.writeBufferSize),\n\t\t\tWithReadBufferSize(e.readBufferSize),\n\t\t\tWithUpdatePeriod(e.updatePeriod),\n\t\t\tWithResendTimeout(e.resendTimeout),\n\t\t\tWithBufferPool(e.pool),\n\t\t)\n\n\t\te.wg.Add(1)\n\t\tgo func() {\n\t\t\tdefer e.wg.Done()\n\t\t\tconn.Run()\n\t\t}()\n\n\t\te.conns[id] = conn\n\t}\n\n\treturn conn\n}\n\nfunc (e *Endpoint) clearConn(addr net.Addr) {\n\tid := addr.String()\n\n\te.mu.Lock()\n\tconn := e.conns[id]\n\tdelete(e.conns, id)\n\te.mu.Unlock()\n\n\tconn.Close()\n}\n\nfunc (e *Endpoint) clearConns() {\n\te.mu.Lock()\n\tconns := make([]*Conn, 0, len(e.conns))\n\tfor id, conn := range e.conns {\n\t\tconns = append(conns, conn)\n\t\tdelete(e.conns, id)\n\t}\n\te.mu.Unlock()\n\n\tfor _, conn := range conns {\n\t\tconn.Close()\n\t}\n}\n\nfunc (e *Endpoint) Addr() net.Addr {\n\treturn e.addr\n}\n\nfunc (e *Endpoint) WriteReliablePacket(buf []byte, addr net.Addr) error {\n\tconn := e.getConn(addr)\n\tif conn == nil {\n\t\treturn io.EOF\n\t}\n\treturn conn.WriteReliablePacket(buf)\n}\n\nfunc (e *Endpoint) WriteUnreliablePacket(buf []byte, addr net.Addr) error {\n\tconn := e.getConn(addr)\n\tif conn == nil {\n\t\treturn io.EOF\n\t}\n\treturn conn.WriteUnreliablePacket(buf)\n}\n\nfunc (e *Endpoint) Listen() {\n\te.mu.Lock()\n\te.wg.Add(1)\n\te.mu.Unlock()\n\n\tdefer e.wg.Done()\n\n\tvar (\n\t\tn    int\n\t\taddr net.Addr\n\t\terr  error\n\t)\n\n\tbuf := make([]byte, math.MaxUint16+1)\n\tfor {\n\t\tn, addr, err = e.conn.ReadFrom(buf)\n\t\tif err != nil {\n\t\t\tbreak\n\t\t}\n\n\t\tconn := e.getConn(addr)\n\t\tif conn == nil {\n\t\t\tbreak\n\t\t}\n\n\t\theader, buf, err := UnmarshalPacketHeader(buf[:n])\n\t\tif err != nil {\n\t\t\te.clearConn(addr)\n\n\t\t\tif e.eh != nil {\n\t\t\t\te.eh(err, e.addr)\n\t\t\t}\n\n\t\t\tcontinue\n\t\t}\n\n\t\terr = conn.Read(header, buf)\n\t\tif err != nil {\n\t\t\te.clearConn(addr)\n\n\t\t\tif e.eh != nil {\n\t\t\t\te.eh(err, e.addr)\n\t\t\t}\n\n\t\t\tcontinue\n\t\t}\n\n\t\tif e.ph != nil {\n\t\t\te.ph(buf, e.addr)\n\t\t}\n\t}\n\n\te.clearConns()\n}\n\nfunc (e *Endpoint) Close() error {\n\tatomic.StoreUint32(&e.closing, 1)\n\te.wg.Wait()\n\treturn nil\n}\n"
  },
  {
    "path": "endpoint_test.go",
    "content": "package reliable\n\nimport (\n\t\"bytes\"\n\t\"github.com/stretchr/testify/require\"\n\t\"go.uber.org/goleak\"\n\t\"net\"\n\t\"sort\"\n\t\"strconv\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t\"testing\"\n\t\"time\"\n)\n\nfunc newPacketConn(t testing.TB, addr string) net.PacketConn {\n\tt.Helper()\n\tconn, err := net.ListenPacket(\"udp\", addr)\n\trequire.NoError(t, err)\n\treturn conn\n}\n\nfunc BenchmarkEndpointWriteReliablePacket(b *testing.B) {\n\tca := newPacketConn(b, \"127.0.0.1:0\")\n\tcb := newPacketConn(b, \"127.0.0.1:0\")\n\n\tea := NewEndpoint(ca)\n\teb := NewEndpoint(cb)\n\n\tgo ea.Listen()\n\tgo eb.Listen()\n\n\tdefer func() {\n\t\trequire.NoError(b, ca.SetDeadline(time.Now().Add(1*time.Millisecond)))\n\t\trequire.NoError(b, cb.SetDeadline(time.Now().Add(1*time.Millisecond)))\n\n\t\trequire.NoError(b, ea.Close())\n\t\trequire.NoError(b, eb.Close())\n\n\t\trequire.NoError(b, ca.Close())\n\t\trequire.NoError(b, cb.Close())\n\t}()\n\n\tdata := bytes.Repeat([]byte(\"x\"), 1400)\n\n\tb.ReportAllocs()\n\tb.ResetTimer()\n\n\tfor i := 0; i < b.N; i++ {\n\t\tif err := ea.WriteReliablePacket(data, eb.Addr()); err != nil && !isEOF(err) {\n\t\t\tb.Fatal(err)\n\t\t}\n\t}\n}\n\nfunc BenchmarkEndpointWriteUnreliablePacket(b *testing.B) {\n\tca := newPacketConn(b, \"127.0.0.1:0\")\n\tcb := newPacketConn(b, \"127.0.0.1:0\")\n\n\tea := NewEndpoint(ca)\n\teb := NewEndpoint(cb)\n\n\tgo ea.Listen()\n\tgo eb.Listen()\n\n\tdefer func() {\n\t\trequire.NoError(b, ca.SetDeadline(time.Now().Add(1*time.Millisecond)))\n\t\trequire.NoError(b, cb.SetDeadline(time.Now().Add(1*time.Millisecond)))\n\n\t\trequire.NoError(b, ea.Close())\n\t\trequire.NoError(b, eb.Close())\n\n\t\trequire.NoError(b, ca.Close())\n\t\trequire.NoError(b, cb.Close())\n\t}()\n\n\tdata := bytes.Repeat([]byte(\"x\"), 1400)\n\n\tb.ReportAllocs()\n\tb.ResetTimer()\n\n\tfor i := 0; i < b.N; i++ {\n\t\tif err := ea.WriteUnreliablePacket(data, eb.Addr()); err != nil && !isEOF(err) {\n\t\t\tb.Fatal(err)\n\t\t}\n\t}\n}\n\nfunc TestEndpointWriteReliablePacket(t *testing.T) {\n\tdefer goleak.VerifyNone(t)\n\n\tvar mu sync.Mutex\n\n\tvalues := make(map[string]struct{})\n\n\tactual := uint64(0)\n\texpected := uint64(65536)\n\n\thandler := func(buf []byte, _ net.Addr) {\n\t\tif len(buf) == 0 {\n\t\t\treturn\n\t\t}\n\n\t\tatomic.AddUint64(&actual, 1)\n\n\t\tmu.Lock()\n\t\t_, exists := values[string(buf)]\n\t\tdelete(values, string(buf))\n\t\tmu.Unlock()\n\n\t\trequire.True(t, exists)\n\t}\n\n\tca := newPacketConn(t, \"127.0.0.1:0\")\n\tcb := newPacketConn(t, \"127.0.0.1:0\")\n\n\ta := NewEndpoint(ca, WithEndpointPacketHandler(handler))\n\tb := NewEndpoint(cb, WithEndpointPacketHandler(handler))\n\n\tgo a.Listen()\n\tgo b.Listen()\n\n\tdefer func() {\n\t\trequire.NoError(t, ca.SetDeadline(time.Now().Add(1*time.Millisecond)))\n\t\trequire.NoError(t, cb.SetDeadline(time.Now().Add(1*time.Millisecond)))\n\n\t\trequire.NoError(t, a.Close())\n\t\trequire.NoError(t, b.Close())\n\n\t\trequire.NoError(t, ca.Close())\n\t\trequire.NoError(t, cb.Close())\n\n\t\trequire.EqualValues(t, expected, atomic.LoadUint64(&actual))\n\t}()\n\n\tfor i := uint64(0); i < expected; i++ {\n\t\tdata := strconv.AppendUint(nil, i, 10)\n\n\t\tmu.Lock()\n\t\tvalues[string(data)] = struct{}{}\n\t\tmu.Unlock()\n\n\t\trequire.NoError(t, a.WriteReliablePacket(data, b.Addr()))\n\t}\n}\n\nfunc TestEndpointWriteReliablePacketEndToEnd(t *testing.T) {\n\tdefer goleak.VerifyNone(t)\n\n\tactual := uint64(0)\n\texpected := uint64(512)\n\n\thandler := func(buf []byte, _ net.Addr) {\n\t\tif len(buf) == 0 {\n\t\t\treturn\n\t\t}\n\t\tatomic.AddUint64(&actual, 1)\n\t}\n\n\tca := newPacketConn(t, \"127.0.0.1:0\")\n\tcb := newPacketConn(t, \"127.0.0.1:0\")\n\n\ta := NewEndpoint(ca, WithEndpointPacketHandler(handler))\n\tb := NewEndpoint(cb, WithEndpointPacketHandler(handler))\n\n\tgo a.Listen()\n\tgo b.Listen()\n\n\tdefer func() {\n\t\trequire.NoError(t, ca.SetDeadline(time.Now().Add(1*time.Millisecond)))\n\t\trequire.NoError(t, cb.SetDeadline(time.Now().Add(1*time.Millisecond)))\n\n\t\trequire.NoError(t, a.Close())\n\t\trequire.NoError(t, b.Close())\n\n\t\trequire.NoError(t, ca.Close())\n\t\trequire.NoError(t, cb.Close())\n\n\t\trequire.EqualValues(t, expected*2, atomic.LoadUint64(&actual))\n\t}()\n\n\tfor i := uint64(0); i < expected; i++ {\n\t\tdata := strconv.AppendUint(nil, i, 10)\n\n\t\trequire.NoError(t, a.WriteReliablePacket(data, b.Addr()))\n\t\trequire.NoError(t, b.WriteReliablePacket(data, a.Addr()))\n\t}\n}\n\n// Check whether race condition happen\n// Simulate write and read heavy condition by sending packet concurrently\nfunc TestRaceConditions(t *testing.T) {\n\tdefer goleak.VerifyNone(t)\n\n\tvar expected int = 1000\n\ttr := newTestRaceConditions(expected)\n\n\thandler := func(buf []byte, _ net.Addr) {\n\t\tif len(buf) == 0 {\n\t\t\treturn\n\t\t}\n\t\ttr.append(buf)\n\t}\n\n\tca := newPacketConn(t, \"127.0.0.1:0\")\n\tcb := newPacketConn(t, \"127.0.0.1:0\")\n\tcc := newPacketConn(t, \"127.0.0.1:0\")\n\tcd := newPacketConn(t, \"127.0.0.1:0\")\n\tce := newPacketConn(t, \"127.0.0.1:0\")\n\n\ta := NewEndpoint(ca, WithEndpointPacketHandler(handler))\n\tb := NewEndpoint(cb, WithEndpointPacketHandler(handler))\n\tc := NewEndpoint(cc, WithEndpointPacketHandler(handler))\n\td := NewEndpoint(cd, WithEndpointPacketHandler(handler))\n\te := NewEndpoint(ce, WithEndpointPacketHandler(handler))\n\n\tgo a.Listen()\n\tgo b.Listen()\n\tgo c.Listen()\n\tgo d.Listen()\n\tgo e.Listen()\n\n\tdefer func() {\n\t\ttr.wait()\n\n\t\t// Note: Guarantee that all messages are deliverd\n\t\ttime.Sleep(100 * time.Millisecond)\n\n\t\trequire.NoError(t, ca.SetDeadline(time.Now().Add(1*time.Millisecond)))\n\t\trequire.NoError(t, cb.SetDeadline(time.Now().Add(1*time.Millisecond)))\n\t\trequire.NoError(t, cc.SetDeadline(time.Now().Add(1*time.Millisecond)))\n\t\trequire.NoError(t, cd.SetDeadline(time.Now().Add(1*time.Millisecond)))\n\t\trequire.NoError(t, ce.SetDeadline(time.Now().Add(1*time.Millisecond)))\n\n\t\trequire.NoError(t, a.Close())\n\t\trequire.NoError(t, b.Close())\n\t\trequire.NoError(t, c.Close())\n\t\trequire.NoError(t, d.Close())\n\t\trequire.NoError(t, e.Close())\n\n\t\trequire.NoError(t, ca.Close())\n\t\trequire.NoError(t, cb.Close())\n\t\trequire.NoError(t, cc.Close())\n\t\trequire.NoError(t, cd.Close())\n\t\trequire.NoError(t, ce.Close())\n\n\t\trequire.EqualValues(t, tr.expected, uniqSort(tr.actual))\n\t}()\n\n\ttr.wg.Add(1)\n\tsB := tr.expected[0 : len(tr.expected)/4]\n\tgo func() {\n\t\tdefer tr.done()\n\t\tfor i := 0; i < len(sB); i++ {\n\t\t\tdata := []byte(strconv.Itoa(sB[i]))\n\n\t\t\terr := a.WriteReliablePacket(data, b.Addr())\n\t\t\tif err != nil {\n\t\t\t\trequire.True(t, isEOF(err))\n\t\t\t}\n\t\t}\n\t}()\n\n\ttr.wg.Add(1)\n\tsC := tr.expected[len(tr.expected)/4 : len(tr.expected)*2/4]\n\tgo func() {\n\t\tdefer tr.done()\n\t\tfor i := 0; i < len(sC); i++ {\n\t\t\tdata := []byte(strconv.Itoa(sC[i]))\n\n\t\t\terr := a.WriteReliablePacket(data, c.Addr())\n\t\t\tif err != nil {\n\t\t\t\trequire.True(t, isEOF(err))\n\t\t\t}\n\t\t}\n\t}()\n\n\ttr.wg.Add(1)\n\tsD := tr.expected[len(tr.expected)*2/4 : len(tr.expected)*3/4]\n\tgo func() {\n\t\tdefer tr.done()\n\t\tfor i := 0; i < len(sD); i++ {\n\t\t\tdata := []byte(strconv.Itoa(sD[i]))\n\n\t\t\terr := a.WriteReliablePacket(data, d.Addr())\n\t\t\tif err != nil {\n\t\t\t\trequire.True(t, isEOF(err))\n\t\t\t}\n\t\t}\n\t}()\n\n\ttr.wg.Add(1)\n\tsE := tr.expected[len(tr.expected)*3/4:]\n\tgo func() {\n\t\tdefer tr.done()\n\t\tfor i := 0; i < len(sE); i++ {\n\t\t\tdata := []byte(strconv.Itoa(sE[i]))\n\n\t\t\terr := a.WriteReliablePacket(data, e.Addr())\n\t\t\tif err != nil {\n\t\t\t\trequire.True(t, isEOF(err))\n\t\t\t}\n\t\t}\n\t}()\n}\n\n// Note: This struct is test for TestRaceConditions\n// The purpose for this struct is to prevent race condition of WaitGroup\ntype testRaceConditions struct {\n\tmu       sync.Mutex\n\twg       sync.WaitGroup\n\texpected []int\n\tactual   []int\n}\n\nfunc newTestRaceConditions(cap int) *testRaceConditions {\n\treturn &testRaceConditions{expected: genNumSlice(cap)}\n}\n\nfunc (t *testRaceConditions) done() {\n\tt.mu.Lock()\n\tdefer t.mu.Unlock()\n\tt.wg.Done()\n}\n\nfunc (t *testRaceConditions) wait() {\n\tt.wg.Wait()\n}\n\nfunc (t *testRaceConditions) append(buf []byte) {\n\tt.mu.Lock()\n\tdefer t.mu.Unlock()\n\n\tnum, _ := strconv.Atoi(string(buf))\n\tt.actual = append(t.actual, num)\n}\n\nfunc genNumSlice(len int) (s []int) {\n\tfor i := 0; i < len; i++ {\n\t\ts = append(s, i)\n\t}\n\treturn\n}\n\nfunc uniqSort(s []int) (result []int) {\n\tsort.Ints(s)\n\tvar pre int\n\tfor i := 0; i < len(s); i++ {\n\t\tif i == 0 || s[i] != pre {\n\t\t\tresult = append(result, s[i])\n\t\t}\n\t\tpre = s[i]\n\t}\n\treturn\n}\n"
  },
  {
    "path": "error.go",
    "content": "package reliable\n\nimport (\n\t\"errors\"\n\t\"io\"\n\t\"net\"\n)\n\nfunc isEOF(err error) bool {\n\tif errors.Is(err, io.EOF) {\n\t\treturn true\n\t}\n\n\tvar netErr *net.OpError\n\tif errors.As(err, &netErr) {\n\t\tif netErr.Err.Error() == \"use of closed network connection\" {\n\t\t\treturn true\n\t\t}\n\t\tif netErr.Timeout() {\n\t\t\treturn true\n\t\t}\n\t}\n\n\treturn false\n}\n"
  },
  {
    "path": "examples/basic/main.go",
    "content": "package main\n\nimport (\n\t\"bytes\"\n\t\"errors\"\n\t\"github.com/davecgh/go-spew/spew\"\n\t\"github.com/lithdew/reliable\"\n\t\"io\"\n\t\"log\"\n\t\"net\"\n\t\"os\"\n\t\"os/signal\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t\"time\"\n)\n\nvar (\n\tPacketData = bytes.Repeat([]byte(\"x\"), 1400)\n\tNumPackets = uint64(0)\n)\n\nfunc check(err error) {\n\tif err != nil && !errors.Is(err, io.EOF) {\n\t\tlog.Panic(err)\n\t}\n}\n\nfunc listen(addr string) net.PacketConn {\n\tconn, err := net.ListenPacket(\"udp\", addr)\n\tcheck(err)\n\treturn conn\n}\n\nfunc handler(buf []byte, _ net.Addr) {\n\tif bytes.Equal(buf, PacketData) || len(buf) == 0 {\n\t\treturn\n\t}\n\tspew.Dump(buf)\n\tos.Exit(1)\n}\n\nfunc main() {\n\texit := make(chan struct{})\n\n\tvar wg sync.WaitGroup\n\twg.Add(2)\n\n\tca := listen(\"127.0.0.1:44444\")\n\tcb := listen(\"127.0.0.1:55555\")\n\n\ta := reliable.NewEndpoint(ca, reliable.WithEndpointPacketHandler(handler))\n\tb := reliable.NewEndpoint(cb, reliable.WithEndpointPacketHandler(handler))\n\n\tdefer func() {\n\t\tcheck(ca.SetDeadline(time.Now().Add(1 * time.Millisecond)))\n\t\tcheck(cb.SetDeadline(time.Now().Add(1 * time.Millisecond)))\n\n\t\tclose(exit)\n\n\t\tcheck(a.Close())\n\t\tcheck(b.Close())\n\n\t\tcheck(ca.Close())\n\t\tcheck(cb.Close())\n\n\t\twg.Wait()\n\t}()\n\n\tgo a.Listen()\n\tgo b.Listen()\n\n\t// The two goroutines below have endpoint A spam endpoint B, and print out how\n\t// many packets of data are being sent per second.\n\n\tgo func() {\n\t\tdefer wg.Done()\n\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-exit:\n\t\t\t\treturn\n\t\t\tdefault:\n\t\t\t}\n\n\t\t\tcheck(a.WriteReliablePacket(PacketData, b.Addr()))\n\t\t\tatomic.AddUint64(&NumPackets, 1)\n\t\t}\n\t}()\n\n\tgo func() {\n\t\tdefer wg.Done()\n\n\t\tticker := time.NewTicker(1 * time.Second)\n\t\tdefer ticker.Stop()\n\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase <-exit:\n\t\t\t\treturn\n\t\t\tcase <-ticker.C:\n\t\t\t\tnumPackets := atomic.SwapUint64(&NumPackets, 0)\n\t\t\t\tnumBytes := float64(numPackets) * 1400.0 / 1024.0 / 1024.0\n\n\t\t\t\tlog.Printf(\n\t\t\t\t\t\"Sent %d packet(s) comprised of %.2f MiB worth of data.\",\n\t\t\t\t\tnumPackets,\n\t\t\t\t\tnumBytes,\n\t\t\t\t)\n\t\t\t}\n\t\t}\n\t}()\n\n\tch := make(chan os.Signal, 1)\n\tsignal.Notify(ch, os.Interrupt)\n\t<-ch\n}\n"
  },
  {
    "path": "examples/benchmark/main.go",
    "content": "package main\n\nimport (\n\t\"bytes\"\n\t\"errors\"\n\t\"flag\"\n\t\"github.com/lithdew/reliable\"\n\t\"io\"\n\t\"log\"\n\t\"net\"\n\t\"sync/atomic\"\n\t\"time\"\n)\n\nvar (\n\tlistener bool\n)\n\nfunc check(err error) {\n\tif err != nil && !errors.Is(err, io.EOF) {\n\t\tlog.Panic(err)\n\t}\n}\n\nfunc listen(addr string) net.PacketConn {\n\tconn, err := net.ListenPacket(\"udp\", addr)\n\tcheck(err)\n\n\tlog.Printf(\"%s: Listening for peers.\", conn.LocalAddr())\n\n\treturn conn\n}\n\nfunc main() {\n\tflag.BoolVar(&listener, \"l\", false, \"either listen or dial\")\n\tflag.Parse()\n\n\thost := flag.Arg(0)\n\tif !listener || host == \"\" {\n\t\thost = \":0\"\n\t}\n\n\tconn := listen(host)\n\n\tcounter := uint64(0)\n\n\thandler := func(buf []byte, addr net.Addr) {\n\t\tif len(buf) == 0 {\n\t\t\treturn\n\t\t}\n\t\t//log.Printf(\"%s->%s: (seq=%d) (size=%d)\", addr.String(), conn.LocalAddr().String(), seq, len(buf))\n\t\tatomic.AddUint64(&counter, 1)\n\t}\n\n\tendpoint := reliable.NewEndpoint(conn, reliable.WithEndpointPacketHandler(handler))\n\tgo endpoint.Listen()\n\n\tdefer func() {\n\t\tcheck(endpoint.Close())\n\t\tcheck(conn.Close())\n\t}()\n\n\tif listener {\n\t\tfor range time.Tick(1 * time.Second) {\n\t\t\tnumPackets := atomic.SwapUint64(&counter, 0)\n\t\t\tnumBytes := float64(numPackets) * 1400.0 / 1024.0 / 1024.0\n\n\t\t\tlog.Printf(\"%s: Received %d packets (%.2f MiB).\", conn.LocalAddr(), numPackets, numBytes)\n\t\t}\n\t}\n\n\taddr, err := net.ResolveUDPAddr(\"udp\", flag.Arg(0))\n\tcheck(err)\n\n\tdata := bytes.Repeat([]byte(\"x\"), 1400)\n\n\tgo func() {\n\t\tfor range time.Tick(1 * time.Second) {\n\t\t\tnumPackets := atomic.SwapUint64(&counter, 0)\n\t\t\tnumBytes := float64(numPackets) * 1400.0 / 1024.0 / 1024.0\n\n\t\t\tlog.Printf(\"%s: Sent %d packets (%.2f MiB).\", conn.LocalAddr(), numPackets, numBytes)\n\t\t}\n\t}()\n\n\tfor {\n\t\tcheck(endpoint.WriteReliablePacket(data, addr))\n\t\tatomic.AddUint64(&counter, 1)\n\t}\n}\n"
  },
  {
    "path": "fuzz.go",
    "content": "// +build gofuzz\n\npackage reliable\n\nimport (\n\t\"bytes\"\n\t\"errors\"\n\t\"net\"\n\t\"time\"\n)\n\nfunc Fuzz(data []byte) int {\n\tca, err := net.ListenPacket(\"udp\", \"127.0.0.1:0\")\n\tif err != nil {\n\t\treturn -1\n\t}\n\tcb, err := net.ListenPacket(\"udp\", \"127.0.0.1:0\")\n\tif err != nil {\n\t\treturn -1\n\t}\n\n\tchErr := make(chan error)\n\n\thandler := func(buf []byte, _ net.Addr) {\n\t\tif len(buf) == 0 || bytes.Equal(buf, data) {\n\t\t\treturn\n\t\t}\n\t\tchErr <- errors.New(\"data miss match\")\n\t}\n\n\tea := NewEndpoint(ca, reliable.WithEndpointPacketHandler(handler))\n\teb := NewEndpoint(cb, reliable.WithEndpointPacketHandler(handler))\n\n\tgo ea.Listen()\n\tgo eb.Listen()\n\n\tfor i := 0; i < 65536; i++ {\n\t\tselect {\n\t\tcase <-chErr:\n\t\t\treturn 0\n\t\tdefault:\n\t\t\tif err := ea.WriteReliablePacket(data, eb.Addr()); err != nil && !isEOF(err) {\n\t\t\t\treturn 0\n\t\t\t}\n\t\t}\n\t}\n\n\tif err := ca.SetDeadline(time.Now().Add(1 * time.Millisecond)); err != nil {\n\t\treturn 0\n\t}\n\n\tif err := cb.SetDeadline(time.Now().Add(1 * time.Millisecond)); err != nil {\n\t\treturn 0\n\t}\n\n\tif err := ea.Close(); err != nil {\n\t\treturn 0\n\t}\n\tif err := eb.Close(); err != nil {\n\t\treturn 0\n\t}\n\n\tif err := ca.Close(); err != nil {\n\t\treturn 0\n\t}\n\tif err := cb.Close(); err != nil {\n\t\treturn 0\n\t}\n\n\treturn 1\n}\n"
  },
  {
    "path": "go.mod",
    "content": "module github.com/lithdew/reliable\n\ngo 1.14\n\nrequire (\n\tgithub.com/davecgh/go-spew v1.1.1\n\tgithub.com/dvyukov/go-fuzz v0.0.0-20200318091601-be3528f3a813 // indirect\n\tgithub.com/lithdew/bytesutil v0.0.0-20200409052507-d98389230a59\n\tgithub.com/lithdew/seq v0.0.0-20200504083424-74d5d8117a05\n\tgithub.com/stretchr/testify v1.5.1\n\tgithub.com/valyala/bytebufferpool v1.0.0\n\tgo.uber.org/goleak v1.0.0\n\tgolang.org/x/lint v0.0.0-20200302205851-738671d3881b // indirect\n\tgolang.org/x/tools v0.0.0-20200501005904-d351ea090f9b // indirect\n)\n"
  },
  {
    "path": "go.sum",
    "content": "github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8=\ngithub.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=\ngithub.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=\ngithub.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=\ngithub.com/dvyukov/go-fuzz v0.0.0-20200318091601-be3528f3a813 h1:NgO45/5mBLRVfiXerEFzH6ikcZ7DNRPS639xFg3ENzU=\ngithub.com/dvyukov/go-fuzz v0.0.0-20200318091601-be3528f3a813/go.mod h1:11Gm+ccJnvAhCNLlf5+cS9KjtbaD5I5zaZpFMsTHWTw=\ngithub.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=\ngithub.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=\ngithub.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=\ngithub.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=\ngithub.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=\ngithub.com/lithdew/bytesutil v0.0.0-20200409052507-d98389230a59 h1:CQpoOQecHxhvgOU/ijue/yWuShZYDtNpI9bsD4Dkzrk=\ngithub.com/lithdew/bytesutil v0.0.0-20200409052507-d98389230a59/go.mod h1:89JlULMIJ/+YWzAp5aHXgAD2d02S2mY+a+PMgXDtoNs=\ngithub.com/lithdew/seq v0.0.0-20200504083424-74d5d8117a05 h1:j1UtG8NYCupA5xUwQ/vrTf/zjuNlZ0D1n7UtM8LhS58=\ngithub.com/lithdew/seq v0.0.0-20200504083424-74d5d8117a05/go.mod h1:4vVgbfmYc+ZIh0dy99HRrM6knnAtQXNI8MOx+1pUYso=\ngithub.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=\ngithub.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=\ngithub.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=\ngithub.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=\ngithub.com/stretchr/testify v1.5.1 h1:nOGnQDM7FYENwehXlg/kFVnos3rEvtKTjRvOWSzb6H4=\ngithub.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=\ngithub.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw=\ngithub.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=\ngithub.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=\ngo.uber.org/goleak v1.0.0 h1:qsup4IcBdlmsnGfqyLl4Ntn3C2XCCuKAE7DwHpScyUo=\ngo.uber.org/goleak v1.0.0/go.mod h1:8a7PlsEVH3e/a/GLqe5IIrQx6GzcnRmZEufDUTk4A7A=\ngolang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=\ngolang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=\ngolang.org/x/lint v0.0.0-20190930215403-16217165b5de h1:5hukYrvBGR8/eNkX5mdUezrA6JiaEZDtJb9Ei+1LlBs=\ngolang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=\ngolang.org/x/lint v0.0.0-20200302205851-738671d3881b h1:Wh+f8QHJXR411sJR8/vRBTZ7YapZaRvUcLFFJhusH0k=\ngolang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=\ngolang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=\ngolang.org/x/mod v0.2.0 h1:KU7oHjnv3XNWfa5COkzUifxZmxp1TyI7ImMXqFxLwvQ=\ngolang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=\ngolang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=\ngolang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=\ngolang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e h1:vcxGaoTs7kV8m5Np9uUNQin4BrLOthgV7252N8V+FwY=\ngolang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=\ngolang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=\ngolang.org/x/tools v0.0.0-20191108193012-7d206e10da11 h1:Yq9t9jnGoR+dBuitxdo9l6Q7xh/zOyNnYUtDKaQ3x0E=\ngolang.org/x/tools v0.0.0-20191108193012-7d206e10da11/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=\ngolang.org/x/tools v0.0.0-20200501005904-d351ea090f9b h1:2hSR2MyOaYEy6yJYg/CpErymr/m7xJEJpm9kfT7ZMg4=\ngolang.org/x/tools v0.0.0-20200501005904-d351ea090f9b/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=\ngolang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\ngolang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\ngolang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=\ngolang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\ngopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=\ngopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=\ngopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=\ngopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=\ngopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=\ngopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\n"
  },
  {
    "path": "options.go",
    "content": "package reliable\n\nimport \"time\"\n\nconst (\n\tDefaultWriteBufferSize uint16 = 256\n\tDefaultReadBufferSize  uint16 = 256\n\n\tDefaultUpdatePeriod  = 100 * time.Millisecond\n\tDefaultResendTimeout = 100 * time.Millisecond\n)\n\ntype ProtocolOption interface {\n\tapplyProtocol(p *Protocol)\n}\n\ntype EndpointOption interface {\n\tapplyEndpoint(e *Endpoint)\n}\n\ntype Option interface {\n\tProtocolOption\n\tEndpointOption\n}\n\ntype withBufferPool struct{ pool *Pool }\n\nfunc (o withBufferPool) applyProtocol(p *Protocol) { p.pool = o.pool }\nfunc (o withBufferPool) applyEndpoint(e *Endpoint) { e.pool = o.pool }\n\nfunc WithBufferPool(pool *Pool) Option { return withBufferPool{pool: pool} }\n\ntype withWriteBufferSize struct{ writeBufferSize uint16 }\n\nfunc (o withWriteBufferSize) applyProtocol(p *Protocol) { p.writeBufferSize = o.writeBufferSize }\nfunc (o withWriteBufferSize) applyEndpoint(e *Endpoint) { e.writeBufferSize = o.writeBufferSize }\n\nfunc WithWriteBufferSize(writeBufferSize uint16) Option {\n\tif 65536%uint32(writeBufferSize) != 0 {\n\t\tpanic(\"write buffer size must be smaller than 65536 and a power of two\")\n\t}\n\treturn withWriteBufferSize{writeBufferSize: writeBufferSize}\n}\n\ntype withReadBufferSize struct{ readBufferSize uint16 }\n\nfunc (o withReadBufferSize) applyProtocol(p *Protocol) { p.readBufferSize = o.readBufferSize }\nfunc (o withReadBufferSize) applyEndpoint(e *Endpoint) { e.readBufferSize = o.readBufferSize }\n\nfunc WithReadBufferSize(readBufferSize uint16) Option {\n\tif 65536%uint32(readBufferSize) != 0 {\n\t\tpanic(\"read buffer size must be smaller than 65536 and a power of two\")\n\t}\n\treturn withReadBufferSize{readBufferSize: readBufferSize}\n}\n\ntype withProtocolPacketHandler struct{ ph ProtocolPacketHandler }\ntype withEndpointPacketHandler struct{ ph EndpointPacketHandler }\n\nfunc (o withProtocolPacketHandler) applyProtocol(p *Protocol) { p.ph = o.ph }\nfunc (o withEndpointPacketHandler) applyEndpoint(e *Endpoint) { e.ph = o.ph }\n\nfunc WithProtocolPacketHandler(ph ProtocolPacketHandler) ProtocolOption {\n\treturn withProtocolPacketHandler{ph: ph}\n}\nfunc WithEndpointPacketHandler(ph EndpointPacketHandler) EndpointOption {\n\treturn withEndpointPacketHandler{ph: ph}\n}\n\ntype withProtocolErrorHandler struct{ eh ProtocolErrorHandler }\ntype withEndpointErrorHandler struct{ eh EndpointErrorHandler }\n\nfunc (o withProtocolErrorHandler) applyProtocol(p *Protocol) { p.eh = o.eh }\nfunc (o withEndpointErrorHandler) applyEndpoint(e *Endpoint) { e.eh = o.eh }\n\nfunc WithProtocolErrorHandler(eh ProtocolErrorHandler) ProtocolOption {\n\treturn withProtocolErrorHandler{eh: eh}\n}\nfunc WithEndpointErrorHandler(eh EndpointErrorHandler) EndpointOption {\n\treturn withEndpointErrorHandler{eh: eh}\n}\n\ntype withUpdatePeriod struct{ updatePeriod time.Duration }\n\nfunc (o withUpdatePeriod) applyProtocol(p *Protocol) { p.updatePeriod = o.updatePeriod }\nfunc (o withUpdatePeriod) applyEndpoint(e *Endpoint) { e.updatePeriod = o.updatePeriod }\n\nfunc WithUpdatePeriod(updatePeriod time.Duration) Option {\n\tif updatePeriod == 0 {\n\t\tpanic(\"update period of zero is not supported yet\")\n\t}\n\treturn withUpdatePeriod{updatePeriod: updatePeriod}\n}\n\ntype withResendTimeout struct{ resendTimeout time.Duration }\n\nfunc (o withResendTimeout) applyProtocol(p *Protocol) { p.resendTimeout = o.resendTimeout }\nfunc (o withResendTimeout) applyEndpoint(e *Endpoint) { e.resendTimeout = o.resendTimeout }\n\nfunc WithResendTimeout(resendTimeout time.Duration) Option {\n\tif resendTimeout == 0 {\n\t\tpanic(\"ack timeout of zero is not supported yet\")\n\t}\n\treturn withResendTimeout{resendTimeout: resendTimeout}\n}\n"
  },
  {
    "path": "packet.go",
    "content": "package reliable\n\nimport (\n\t\"github.com/lithdew/bytesutil\"\n\t\"github.com/valyala/bytebufferpool\"\n\t\"io\"\n\t\"math/bits\"\n\t\"time\"\n)\n\nconst ACKBitsetSize = 32\n\ntype (\n\tBuffer = bytebufferpool.ByteBuffer\n\tPool   = bytebufferpool.Pool\n)\n\ntype writtenPacket struct {\n\tbuf     *Buffer   // pooled contents of this packet\n\tacked   bool      // whether or not this packet was acked\n\twritten time.Time // last time the packet was written\n\tresent  byte      // total number of times this packet was resent\n}\n\nfunc (p writtenPacket) shouldResend(now time.Time, resendTimeout time.Duration) bool {\n\treturn !p.acked && p.resent < 10 && now.Sub(p.written) >= resendTimeout\n}\n\ntype PacketHeaderFlag uint8\n\nconst (\n\tFlagFragment PacketHeaderFlag = 1 << iota\n\tFlagA\n\tFlagB\n\tFlagC\n\tFlagD\n\tFlagACKEncoded\n\tFlagEmpty\n\tFlagUnordered\n)\n\nfunc (p PacketHeaderFlag) Toggle(flag PacketHeaderFlag) PacketHeaderFlag {\n\treturn p | flag\n}\n\nfunc (p PacketHeaderFlag) Toggled(flag PacketHeaderFlag) bool {\n\treturn p&flag != 0\n}\n\nfunc (p PacketHeaderFlag) AppendTo(dst []byte) []byte {\n\treturn append(dst, byte(p))\n}\n\ntype PacketHeader struct {\n\tSequence  uint16\n\tACK       uint16\n\tACKBits   uint32\n\tUnordered bool\n\tEmpty     bool\n}\n\nfunc (p PacketHeader) AppendTo(dst []byte) []byte {\n\t// Mark a flag byte to RLE-encode the ACK bitset.\n\n\tflag := PacketHeaderFlag(0)\n\tif p.ACKBits&0x000000FF != 0x000000FF {\n\t\tflag = flag.Toggle(FlagA)\n\t}\n\tif p.ACKBits&0x0000FF00 != 0x0000FF00 {\n\t\tflag = flag.Toggle(FlagB)\n\t}\n\tif p.ACKBits&0x00FF0000 != 0x00FF0000 {\n\t\tflag = flag.Toggle(FlagC)\n\t}\n\tif p.ACKBits&0xFF000000 != 0xFF000000 {\n\t\tflag = flag.Toggle(FlagD)\n\t}\n\tif p.Empty {\n\t\tflag = flag.Toggle(FlagEmpty)\n\t}\n\tif p.Unordered {\n\t\tflag = flag.Toggle(FlagUnordered)\n\t}\n\n\tdiff := int(p.Sequence) - int(p.ACK)\n\tif diff < 0 {\n\t\tdiff += 65536\n\t}\n\tif diff <= 255 {\n\t\tflag = flag.Toggle(FlagACKEncoded)\n\t}\n\n\t// If the difference between the sequence number and the latest ACK'd sequence number can be represented by a\n\t// single byte, then represent it as a single byte and set the 5th bit of flag.\n\n\t// Marshal the flag and sequence number and latest ACK'd sequence number.\n\n\tdst = flag.AppendTo(dst)\n\n\tif p.Unordered {\n\t\tdst = bytesutil.AppendUint16BE(dst, p.ACK)\n\t} else {\n\t\tdst = bytesutil.AppendUint16BE(dst, p.Sequence)\n\n\t\tif diff <= 255 {\n\t\t\tdst = append(dst, uint8(diff))\n\t\t} else {\n\t\t\tdst = bytesutil.AppendUint16BE(dst, p.ACK)\n\t\t}\n\t}\n\n\t// Marshal ACK bitset.\n\n\tif p.ACKBits&0x000000FF != 0x000000FF {\n\t\tdst = append(dst, uint8(p.ACKBits&0x000000FF))\n\t}\n\tif p.ACKBits&0x0000FF00 != 0x0000FF00 {\n\t\tdst = append(dst, uint8((p.ACKBits&0x0000FF00)>>8))\n\t}\n\tif p.ACKBits&0x00FF0000 != 0x00FF0000 {\n\t\tdst = append(dst, uint8((p.ACKBits&0x00FF0000)>>16))\n\t}\n\tif p.ACKBits&0xFF000000 != 0xFF000000 {\n\t\tdst = append(dst, uint8((p.ACKBits&0xFF000000)>>24))\n\t}\n\n\treturn dst\n}\n\nfunc UnmarshalPacketHeader(buf []byte) (header PacketHeader, leftover []byte, err error) {\n\tflag := PacketHeaderFlag(0)\n\n\t// Read first 3 bytes (header, flag).\n\n\tif len(buf) < 3 {\n\t\treturn header, buf, io.ErrUnexpectedEOF\n\t}\n\n\tflag, buf = PacketHeaderFlag(buf[0]), buf[1:]\n\n\tif flag.Toggled(FlagFragment) {\n\t\treturn header, buf, io.ErrUnexpectedEOF\n\t}\n\n\theader.Empty = flag.Toggled(FlagEmpty)\n\theader.Unordered = flag.Toggled(FlagUnordered)\n\n\tif header.Unordered {\n\t\tif len(buf) < 2 {\n\t\t\treturn header, buf, io.ErrUnexpectedEOF\n\t\t}\n\t\theader.ACK, buf = bytesutil.Uint16BE(buf[:2]), buf[2:]\n\t} else {\n\t\theader.Sequence, buf = bytesutil.Uint16BE(buf[:2]), buf[2:]\n\n\t\t// Read and decode the latest ACK'ed sequence number (either 1 or 2 bytes) using the RLE flag marker.\n\n\t\tif flag.Toggled(FlagACKEncoded) {\n\t\t\tif len(buf) < 1 {\n\t\t\t\treturn header, buf, io.ErrUnexpectedEOF\n\t\t\t}\n\t\t\theader.ACK, buf = header.Sequence-uint16(buf[0]), buf[1:]\n\t\t} else {\n\t\t\tif len(buf) < 2 {\n\t\t\t\treturn header, buf, io.ErrUnexpectedEOF\n\t\t\t}\n\t\t\theader.ACK, buf = bytesutil.Uint16BE(buf[:2]), buf[2:]\n\t\t}\n\t}\n\n\tif len(buf) < bits.OnesCount8(uint8(flag&(FlagA|FlagB|FlagC|FlagD))) {\n\t\treturn header, buf, io.ErrUnexpectedEOF\n\t}\n\n\t// Read and decode ACK bitset using the RLE flag marker.\n\n\theader.ACKBits = 0xFFFFFFFF\n\n\tif flag.Toggled(FlagA) {\n\t\theader.ACKBits &= 0xFFFFFF00\n\t\theader.ACKBits |= uint32(buf[0])\n\t\tbuf = buf[1:]\n\t}\n\n\tif flag.Toggled(FlagB) {\n\t\theader.ACKBits &= 0xFFFF00FF\n\t\theader.ACKBits |= uint32(buf[0]) << 8\n\t\tbuf = buf[1:]\n\t}\n\n\tif flag.Toggled(FlagC) {\n\t\theader.ACKBits &= 0xFF00FFFF\n\t\theader.ACKBits |= uint32(buf[0]) << 16\n\t\tbuf = buf[1:]\n\t}\n\n\tif flag.Toggled(FlagD) {\n\t\theader.ACKBits &= 0x00FFFFFF\n\t\theader.ACKBits |= uint32(buf[0]) << 24\n\t\tbuf = buf[1:]\n\t}\n\n\treturn header, buf, nil\n}\n"
  },
  {
    "path": "packet_test.go",
    "content": "package reliable\n\nimport (\n\t\"github.com/stretchr/testify/assert\"\n\t\"github.com/stretchr/testify/require\"\n\t\"github.com/valyala/bytebufferpool\"\n\t\"math\"\n\t\"testing\"\n\t\"testing/quick\"\n)\n\nfunc TestEncodeDecodePacketHeader(t *testing.T) {\n\tbuf := bytebufferpool.Get()\n\tdefer bytebufferpool.Put(buf)\n\n\tf := func(seq, ack uint16, ackBits uint32) bool {\n\t\theader := PacketHeader{Sequence: seq, ACK: ack, ACKBits: ackBits}\n\t\trecovered, leftover, err := UnmarshalPacketHeader(header.AppendTo(buf.B[:0]))\n\t\treturn assert.NoError(t, err) && assert.Len(t, leftover, 0) && assert.EqualValues(t, header, recovered)\n\t}\n\n\trequire.NoError(t, quick.Check(f, &quick.Config{MaxCount: 1000}))\n}\n\nfunc BenchmarkMarshalPacketHeader(b *testing.B) {\n\theader := PacketHeader{Sequence: math.MaxUint16, ACK: math.MaxUint16, ACKBits: math.MaxUint32}\n\n\tbuf := bytebufferpool.Get()\n\tdefer bytebufferpool.Put(buf)\n\n\tb.ResetTimer()\n\tb.ReportAllocs()\n\n\tfor i := 0; i < b.N; i++ {\n\t\tbuf.B = header.AppendTo(buf.B[:0])\n\t}\n}\n\nfunc BenchmarkUnmarshalPacketHeader(b *testing.B) {\n\theader := PacketHeader{Sequence: math.MaxUint16, ACK: math.MaxUint16, ACKBits: math.MaxUint32}\n\n\tbuf := bytebufferpool.Get()\n\tdefer bytebufferpool.Put(buf)\n\n\tbuf.B = header.AppendTo(buf.B)\n\n\tvar (\n\t\trecovered PacketHeader\n\t\tleftover  []byte\n\t\terr       error\n\t)\n\n\tb.ResetTimer()\n\tb.ReportAllocs()\n\n\tfor i := 0; i < b.N; i++ {\n\t\trecovered, leftover, err = UnmarshalPacketHeader(buf.B)\n\t\tif err != nil {\n\t\t\tb.Fatalf(\"failed to unmarshal packet header: %s\", err)\n\t\t}\n\t\tif leftover := len(leftover); leftover != 0 {\n\t\t\tb.Fatalf(\"got %d byte(s) leftover\", leftover)\n\t\t}\n\t\tif recovered.Sequence != header.Sequence || recovered.ACK != header.ACK || recovered.ACKBits != header.ACKBits {\n\t\t\tb.Fatalf(\"got %#v, expected %#v\", recovered, header)\n\t\t}\n\t}\n\n\t_ = recovered\n}\n"
  },
  {
    "path": "pool.go",
    "content": "package reliable\n\nimport \"math\"\n\nvar (\n\temptyBufferIndexCache [math.MaxUint16]uint32\n)\n\nfunc init() {\n\temptyBufferIndexCache[0] = math.MaxUint32\n\tfor i := 1; i < math.MaxUint16; i *= 2 {\n\t\tcopy(emptyBufferIndexCache[i:], emptyBufferIndexCache[:i])\n\t}\n}\n\nfunc emptyBufferIndices(indices []uint32) {\n\tcopy(indices[:], emptyBufferIndexCache[:len(indices)])\n}\n"
  },
  {
    "path": "protocol.go",
    "content": "package reliable\n\nimport (\n\t\"fmt\"\n\t\"github.com/lithdew/seq\"\n\t\"io\"\n\t\"sync\"\n\t\"time\"\n)\n\ntype ProtocolPacketHandler func(buf []byte, seq uint16)\ntype ProtocolErrorHandler func(err error)\n\ntype Protocol struct {\n\twriteBufferSize uint16 // write buffer size that must be a divisor of 65536\n\treadBufferSize  uint16 // read buffer size that must be a divisor of 65536\n\n\tupdatePeriod  time.Duration // how often time-dependant parts of the protocol get checked\n\tresendTimeout time.Duration // how long we wait until unacked packets should be resent\n\n\tpool *Pool\n\n\tph ProtocolPacketHandler\n\teh ProtocolErrorHandler\n\n\tmu   sync.Mutex    // mutex over everything\n\tdie  bool          // is this conn closed?\n\texit chan struct{} // signal channel to close the conn\n\n\tlui uint16    // last sent packet index that hasn't been sent via an ack yet\n\toui uint16    // oldest sent packet index that hasn't been acked yet\n\touc sync.Cond // stop writes if the next write given oui may flood our peers read buffer\n\tls  time.Time // last time data was sent to our peer\n\n\twi uint16 // write index\n\tri uint16 // read index\n\n\twq []uint32 // write queue\n\trq []uint32 // read queue\n\n\twqe []writtenPacket // write queue entries\n}\n\nfunc NewProtocol(opts ...ProtocolOption) *Protocol {\n\tp := &Protocol{exit: make(chan struct{})}\n\n\tfor _, opt := range opts {\n\t\topt.applyProtocol(p)\n\t}\n\n\tif p.writeBufferSize == 0 {\n\t\tp.writeBufferSize = DefaultWriteBufferSize\n\t}\n\n\tif p.readBufferSize == 0 {\n\t\tp.readBufferSize = DefaultReadBufferSize\n\t}\n\n\tif p.resendTimeout == 0 {\n\t\tp.resendTimeout = DefaultResendTimeout\n\t}\n\n\tif p.updatePeriod == 0 {\n\t\tp.updatePeriod = DefaultUpdatePeriod\n\t}\n\n\tif p.pool == nil {\n\t\tp.pool = new(Pool)\n\t}\n\n\tp.wq = make([]uint32, p.writeBufferSize)\n\tp.rq = make([]uint32, p.readBufferSize)\n\n\temptyBufferIndices(p.wq)\n\temptyBufferIndices(p.rq)\n\n\tp.wqe = make([]writtenPacket, p.writeBufferSize)\n\n\tp.ouc.L = &p.mu\n\n\treturn p\n}\n\nfunc (p *Protocol) WritePacket(reliable bool, buf []byte) ([]byte, error) {\n\tp.mu.Lock()\n\tdefer p.mu.Unlock()\n\n\tvar (\n\t\tidx     uint16\n\t\tack     uint16\n\t\tackBits uint32\n\t\tok      = true\n\t)\n\n\tif reliable {\n\t\tidx, ack, ackBits, ok = p.waitForNextWriteDetails()\n\t} else {\n\t\tack, ackBits = p.nextAckDetails()\n\t}\n\n\tif !ok {\n\t\treturn nil, io.EOF\n\t}\n\n\tp.trackAcked(ack)\n\n\t// log.Printf(\"%v: send    (seq=%05d) (ack=%05d) (ack_bits=%032b) (size=%d) (reliable=%t)\", &p, idx, ack, ackBits, len(buf), reliable)\n\n\treturn p.write(PacketHeader{Sequence: idx, ACK: ack, ACKBits: ackBits, Unordered: !reliable}, buf), nil\n}\n\nfunc (p *Protocol) waitUntilReaderAvailable() {\n\tfor !p.die && seq.GT(p.wi+1, p.oui+uint16(len(p.rq))) {\n\t\tp.ouc.Wait()\n\t}\n}\n\nfunc (p *Protocol) waitForNextWriteDetails() (idx uint16, ack uint16, ackBits uint32, ok bool) {\n\tp.waitUntilReaderAvailable()\n\n\tidx, ok = p.nextWriteIndex(), !p.die\n\tack, ackBits = p.nextAckDetails()\n\treturn idx, ack, ackBits, ok\n}\n\nfunc (p *Protocol) nextWriteIndex() (idx uint16) {\n\tidx, p.wi = p.wi, p.wi+1\n\treturn idx\n}\n\nfunc (p *Protocol) nextAckDetails() (ack uint16, ackBits uint32) {\n\tack = p.ri - 1\n\tackBits = p.prepareAckBits(ack)\n\treturn ack, ackBits\n}\n\nfunc (p *Protocol) prepareAckBits(ack uint16) (ackBits uint32) {\n\tfor i, m := uint16(0), uint32(1); i < ACKBitsetSize; i, m = i+1, m<<1 {\n\t\tif p.rq[(ack-i)%uint16(len(p.rq))] != uint32(ack-i) {\n\t\t\tcontinue\n\t\t}\n\n\t\tackBits |= m\n\t}\n\treturn ackBits\n}\n\nfunc (p *Protocol) write(header PacketHeader, buf []byte) []byte {\n\tb := p.pool.Get()\n\n\tb.B = header.AppendTo(b.B)\n\tb.B = append(b.B, buf...)\n\n\tif header.Unordered {\n\t\tdefer p.pool.Put(b)\n\t}\n\n\tif !header.Unordered {\n\t\tp.trackWrite(header.Sequence, b)\n\t}\n\n\treturn b.B\n}\n\nfunc (p *Protocol) trackWrite(idx uint16, buf *Buffer) {\n\tif seq.GT(idx+1, p.wi) {\n\t\tp.clearWrites(p.wi, idx)\n\t\tp.wi = idx + 1\n\t}\n\n\ti := idx % uint16(len(p.wq))\n\tp.wq[i] = uint32(idx)\n\tif p.wqe[i].buf != nil {\n\t\tp.pool.Put(p.wqe[i].buf)\n\t}\n\tp.wqe[i].buf = buf\n\tp.wqe[i].acked = false\n\tp.wqe[i].written = time.Now()\n\tp.wqe[i].resent = 0\n}\n\nfunc (p *Protocol) clearWrites(start, end uint16) {\n\tcount, size := end-start+1, uint16(len(p.wq))\n\n\tif count >= size {\n\t\temptyBufferIndices(p.wq)\n\t\treturn\n\t}\n\n\tfirst := p.wq[start%size:]\n\tlength := uint16(len(first))\n\n\tif count <= length {\n\t\temptyBufferIndices(first[:count])\n\t\treturn\n\t}\n\n\tsecond := p.wq[:count-length]\n\n\temptyBufferIndices(first)\n\temptyBufferIndices(second)\n}\n\nfunc (p *Protocol) ReadPacket(header PacketHeader, buf []byte) []byte {\n\tp.mu.Lock()\n\tdefer p.mu.Unlock()\n\n\tp.readAckBits(header.ACK, header.ACKBits)\n\n\tif !header.Unordered && !p.trackRead(header.Sequence) {\n\t\treturn nil\n\t}\n\n\tp.trackUnacked()\n\n\tif header.Empty {\n\t\treturn nil\n\t}\n\n\tif p.ph != nil {\n\t\tp.ph(buf, header.Sequence)\n\t}\n\n\t// log.Printf(\"%v: recv    (seq=%05d) (ack=%05d) (ack_bits=%032b) (size=%d) (reliable=%t)\", &p, header.Sequence, header.ACK, header.ACKBits, len(buf), !header.Unordered)\n\n\treturn p.writeAcksIfNecessary()\n}\n\nfunc (p *Protocol) createAckIfNecessary() (header PacketHeader, needed bool) {\n\tlui := p.lui\n\n\tfor i := uint16(0); i < ACKBitsetSize; i++ {\n\t\tif p.rq[(lui+i)%uint16(len(p.rq))] != uint32(lui+i) {\n\t\t\treturn header, needed\n\t\t}\n\t}\n\n\tlui += ACKBitsetSize\n\tp.lui = lui\n\tp.ls = time.Now()\n\n\tp.waitUntilReaderAvailable()\n\n\theader.Sequence, header.ACK = p.nextWriteIndex(), lui-1\n\theader.ACKBits = p.prepareAckBits(header.ACK)\n\theader.Empty = true\n\n\tneeded = !p.die\n\n\treturn header, needed\n}\n\nfunc (p *Protocol) writeAcksIfNecessary() []byte {\n\tfor {\n\t\theader, needed := p.createAckIfNecessary()\n\t\tif !needed {\n\t\t\treturn nil\n\t\t}\n\n\t\t// log.Printf(\"%v: ack     (seq=%05d) (ack=%05d) (ack_bits=%032b)\", &p, header.Sequence, header.ACK, header.ACKBits)\n\n\t\treturn p.write(header, nil)\n\t}\n}\n\nfunc (p *Protocol) readAckBits(ack uint16, ackBits uint32) {\n\tfor idx := uint16(0); idx < ACKBitsetSize; idx, ackBits = idx+1, ackBits>>1 {\n\t\tif ackBits&1 == 0 {\n\t\t\tcontinue\n\t\t}\n\n\t\ti := (ack - idx) % uint16(len(p.wq))\n\t\tif p.wq[i] != uint32(ack-idx) || p.wqe[i].acked {\n\t\t\tcontinue\n\t\t}\n\n\t\tif p.wqe[i].buf != nil {\n\t\t\tp.pool.Put(p.wqe[i].buf)\n\t\t}\n\n\t\tp.wqe[i].buf = nil\n\t\tp.wqe[i].acked = true\n\t}\n}\n\nfunc (p *Protocol) trackRead(idx uint16) bool {\n\ti := idx % uint16(len(p.rq))\n\n\tif p.rq[i] == uint32(idx) { // duplicate packet\n\t\treturn false\n\t}\n\n\tif seq.GT(idx+1, p.ri) {\n\t\tp.clearReads(p.ri, idx)\n\t\tp.ri = idx + 1\n\t}\n\n\tp.rq[i] = uint32(idx)\n\n\treturn true\n}\n\nfunc (p *Protocol) clearReads(start, end uint16) {\n\tcount, size := end-start+1, uint16(len(p.rq))\n\n\tif count >= size {\n\t\temptyBufferIndices(p.rq)\n\t\treturn\n\t}\n\n\tfirst := p.rq[start%size:]\n\tlength := uint16(len(first))\n\n\tif count <= length {\n\t\temptyBufferIndices(first[:count])\n\t\treturn\n\t}\n\n\tsecond := p.rq[:count-length]\n\n\temptyBufferIndices(first)\n\temptyBufferIndices(second)\n}\n\nfunc (p *Protocol) trackAcked(ack uint16) {\n\tlui := p.lui\n\n\tfor lui <= ack {\n\t\tif p.rq[lui%uint16(len(p.rq))] != uint32(lui) {\n\t\t\tbreak\n\t\t}\n\t\tlui++\n\t}\n\n\tp.lui = lui\n\tp.ls = time.Now()\n}\n\nfunc (p *Protocol) trackUnacked() {\n\toui := p.oui\n\n\tfor {\n\t\ti := oui % uint16(len(p.wq))\n\t\tif p.wq[i] != uint32(oui) || !p.wqe[i].acked {\n\t\t\tbreak\n\t\t}\n\t\toui++\n\t}\n\tp.oui = oui\n\n\tp.ouc.Broadcast()\n}\n\nfunc (p *Protocol) close() bool {\n\tif p.die {\n\t\treturn false\n\t}\n\tclose(p.exit)\n\tp.die = true\n\tp.ouc.Broadcast()\n\n\treturn true\n}\n\nfunc (p *Protocol) Close() {\n\tp.mu.Lock()\n\tdefer p.mu.Unlock()\n\n\tif !p.close() {\n\t\treturn\n\t}\n}\n\nfunc (p *Protocol) Run(transmit transmitFunc) {\n\tticker := time.NewTicker(p.updatePeriod)\n\tdefer ticker.Stop()\n\n\tfor {\n\t\tselect {\n\t\tcase <-p.exit:\n\t\t\treturn\n\t\tcase <-ticker.C:\n\t\t\tif err := p.retransmitUnackedPackets(transmit); err != nil && p.eh != nil {\n\t\t\t\tp.eh(err)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc (p *Protocol) retransmitUnackedPackets(transmit transmitFunc) error {\n\tp.mu.Lock()\n\tdefer p.mu.Unlock()\n\n\tfor idx := uint16(0); idx < uint16(len(p.wq)); idx++ {\n\t\ti := (p.oui + idx) % uint16(len(p.wq))\n\t\tif p.wq[i] != uint32(p.oui+idx) || !p.wqe[i].shouldResend(time.Now(), p.resendTimeout) {\n\t\t\tcontinue\n\t\t}\n\n\t\t// log.Printf(\"%v: resend  (seq=%d)\", &p, p.oui+idx)\n\n\t\tif isEOF, err := transmit(p.wqe[i].buf.B); err != nil {\n\t\t\treturn fmt.Errorf(\"failed to retransmit unacked packet: %w\", err)\n\t\t} else if isEOF {\n\t\t\tbreak\n\t\t}\n\n\t\tp.wqe[i].written = time.Now()\n\t\tp.wqe[i].resent++\n\t}\n\n\treturn nil\n}\n"
  },
  {
    "path": "protocol_test.go",
    "content": "package reliable\n\nimport (\n\t\"github.com/stretchr/testify/require\"\n\t\"go.uber.org/goleak\"\n\t\"sync\"\n\t\"testing\"\n)\n\nfunc testConnWaitForWriteDetails(inc uint16) func(t testing.TB) {\n\treturn func(t testing.TB) {\n\t\tdefer goleak.VerifyNone(t)\n\n\t\tp := NewProtocol()\n\t\tp.wi = uint16(len(p.rq))\n\n\t\tvar wg sync.WaitGroup\n\t\twg.Add(8)\n\n\t\tch := make(chan uint16, 8)\n\n\t\tfor i := 0; i < 8; i++ {\n\t\t\tgo func() {\n\t\t\t\tdefer wg.Done()\n\n\t\t\t\tp.ouc.L.Lock()\n\t\t\t\tidx, _, _, _ := p.waitForNextWriteDetails()\n\t\t\t\tp.ouc.L.Unlock()\n\t\t\t\tch <- idx\n\t\t\t}()\n\t\t}\n\n\t\tfor i := 0; i < 8; i++ {\n\t\t\tp.ouc.L.Lock()\n\t\t\tp.oui += inc\n\t\t\tp.ouc.Broadcast()\n\t\t\tp.ouc.L.Unlock()\n\t\t}\n\n\t\twg.Wait()\n\n\t\texpected := make(map[uint16]struct{}, 8)\n\n\t\tclose(ch)\n\t\tfor idx := range ch {\n\t\t\texpected[idx] = struct{}{}\n\t\t}\n\n\t\tfor i := 0; i < 8; i++ {\n\t\t\tactual := uint16(len(p.wq) + i)\n\t\t\trequire.Contains(t, expected, actual)\n\t\t\tdelete(expected, actual)\n\t\t}\n\t}\n}\n\nfunc TestConnWaitForWriteDetails(t *testing.T) {\n\ttestConnWaitForWriteDetails(1)(t)\n\ttestConnWaitForWriteDetails(2)(t)\n\ttestConnWaitForWriteDetails(4)(t)\n}\n"
  }
]