Repository: Terry-Mao/goim Branch: master Commit: 1800484a4533 Files: 116 Total size: 335.2 KB Directory structure: gitextract_ggltw923/ ├── .github/ │ └── workflows/ │ └── go.yml ├── .gitignore ├── CHANGELOG.md ├── LICENSE ├── Makefile ├── README.md ├── README_cn.md ├── README_en.md ├── api/ │ ├── comet/ │ │ ├── comet.pb.go │ │ └── comet.proto │ ├── generate.go │ ├── logic/ │ │ ├── logic.pb.go │ │ └── logic.proto │ └── protocol/ │ ├── operation.go │ ├── protocol.go │ ├── protocol.pb.go │ └── protocol.proto ├── benchmarks/ │ ├── client/ │ │ └── main.go │ ├── multi_push/ │ │ └── main.go │ ├── push/ │ │ └── main.go │ ├── push_room/ │ │ └── main.go │ └── push_rooms/ │ └── main.go ├── cmd/ │ ├── comet/ │ │ ├── comet-example.toml │ │ └── main.go │ ├── job/ │ │ ├── job-example.toml │ │ └── main.go │ └── logic/ │ ├── logic-example.toml │ └── main.go ├── codecov.sh ├── docs/ │ ├── benchmark_cn.md │ ├── benchmark_en.md │ ├── en/ │ │ ├── proto.md │ │ └── push.md │ ├── goim.graffle │ ├── proto.md │ └── push.md ├── examples/ │ ├── cert.pem │ ├── javascript/ │ │ ├── client.js │ │ ├── index.html │ │ └── main.go │ └── private.pem ├── go.mod ├── go.sum ├── internal/ │ ├── comet/ │ │ ├── bucket.go │ │ ├── channel.go │ │ ├── conf/ │ │ │ └── conf.go │ │ ├── errors/ │ │ │ └── errors.go │ │ ├── grpc/ │ │ │ └── server.go │ │ ├── operation.go │ │ ├── ring.go │ │ ├── room.go │ │ ├── round.go │ │ ├── server.go │ │ ├── server_tcp.go │ │ ├── server_websocket.go │ │ └── whitelist.go │ ├── job/ │ │ ├── comet.go │ │ ├── conf/ │ │ │ └── conf.go │ │ ├── job.go │ │ ├── push.go │ │ └── room.go │ └── logic/ │ ├── balancer.go │ ├── balancer_test.go │ ├── conf/ │ │ └── conf.go │ ├── conn.go │ ├── conn_test.go │ ├── dao/ │ │ ├── dao.go │ │ ├── dao_test.go │ │ ├── kafka.go │ │ ├── kafka_test.go │ │ ├── redis.go │ │ ├── redis_test.go │ │ └── size_coverage.out │ ├── grpc/ │ │ └── server.go │ ├── http/ │ │ ├── middleware.go │ │ ├── nodes.go │ │ ├── online.go │ │ ├── push.go │ │ ├── result.go │ │ └── server.go │ ├── logic.go │ ├── logic_test.go │ ├── model/ │ │ ├── metadata.go │ │ ├── online.go │ │ └── room.go │ ├── nodes.go │ ├── nodes_test.go │ ├── online.go │ ├── online_test.go │ ├── push.go │ └── push_test.go ├── pkg/ │ ├── bufio/ │ │ ├── bufio.go │ │ └── bufio_test.go │ ├── bytes/ │ │ ├── buffer.go │ │ ├── buffer_test.go │ │ ├── writer.go │ │ └── writer_test.go │ ├── encoding/ │ │ └── binary/ │ │ ├── endian.go │ │ └── endian_test.go │ ├── ip/ │ │ ├── ip.go │ │ └── ip_test.go │ ├── strings/ │ │ ├── ints.go │ │ └── ints_test.go │ ├── time/ │ │ ├── debug.go │ │ ├── duration.go │ │ ├── duration_test.go │ │ ├── timer.go │ │ └── timer_test.go │ └── websocket/ │ ├── conn.go │ ├── request.go │ ├── server.go │ └── server_test.go └── scripts/ ├── README.md ├── jdk8.sh ├── kafka.sh └── zk.sh ================================================ FILE CONTENTS ================================================ ================================================ FILE: .github/workflows/go.yml ================================================ name: Go on: push: branches: [ master ] pull_request: branches: [ master ] jobs: build: name: Build runs-on: ubuntu-latest steps: - name: Set up Go 1.x uses: actions/setup-go@v2 with: go-version: ^1.13 - name: Check out code into the Go module directory uses: actions/checkout@v2 - name: Get dependencies run: | go get -v -t -d ./... if [ -f Gopkg.toml ]; then curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh dep ensure fi - name: Build run: go build -v ./... ================================================ FILE: .gitignore ================================================ # IDE ignore .idea/ *.ipr *.iml *.iws .vscode/ # temp ignore *.log *.cache *.diff *.exe *.exe~ *.patch *.tmp *.swp # system ignore .DS_Store Thumbs.db # build /cmd/comet/comet /cmd/logic/logic /cmd/job/job /target /configs /dist ================================================ FILE: CHANGELOG.md ================================================ #### goim ##### Version 2.0.0 > 1.router has been changed to redis > 2.Support node with redis online heartbeat maintenance > 3.Support for gRPC and Discovery services > 4.Support node connection number and weight scheduling > 5.Support node scheduling by region > 6.Support instruction subscription > 7.Support the current connection room switch > 8.Support multiple room types ({type}://{room_id}) > 9.Support sending messages by device_id > 10.Support for room message aggregation > 11.Supports IPv6 ================================================ FILE: LICENSE ================================================ The MIT License (MIT) Copyright (c) 2015 Terry.Mao Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ================================================ FILE: Makefile ================================================ # Go parameters GOCMD=GO111MODULE=on go GOBUILD=$(GOCMD) build GOTEST=$(GOCMD) test all: test build build: rm -rf target/ mkdir target/ cp cmd/comet/comet-example.toml target/comet.toml cp cmd/logic/logic-example.toml target/logic.toml cp cmd/job/job-example.toml target/job.toml $(GOBUILD) -o target/comet cmd/comet/main.go $(GOBUILD) -o target/logic cmd/logic/main.go $(GOBUILD) -o target/job cmd/job/main.go test: $(GOTEST) -v ./... clean: rm -rf target/ run: nohup target/logic -conf=target/logic.toml -region=sh -zone=sh001 -deploy.env=dev -weight=10 2>&1 > target/logic.log & nohup target/comet -conf=target/comet.toml -region=sh -zone=sh001 -deploy.env=dev -weight=10 -addrs=127.0.0.1 -debug=true 2>&1 > target/comet.log & nohup target/job -conf=target/job.toml -region=sh -zone=sh001 -deploy.env=dev 2>&1 > target/job.log & stop: pkill -f target/logic pkill -f target/job pkill -f target/comet ================================================ FILE: README.md ================================================ goim v2.0 ============== [![Language](https://img.shields.io/badge/Language-Go-blue.svg)](https://golang.org/) [![Build Status](https://github.com/Terry-Mao/goim/workflows/Go/badge.svg)](https://github.com/Terry-Mao/goim/actions) [![GoDoc](https://godoc.org/github.com/go-kratos/kratos?status.svg)](https://pkg.go.dev/github.com/Terry-Mao/goim) [![Go Report Card](https://goreportcard.com/badge/github.com/Terry-Mao/goim)](https://goreportcard.com/report/github.com/Terry-Mao/goim) goim is an im server writen in golang. ## Features * Light weight * High performance * Pure Golang * Supports single push, multiple push and broadcasting * Supports one key to multiple subscribers (Configurable maximum subscribers count) * Supports heartbeats (Application heartbeats, TCP, KeepAlive, HTTP long pulling) * Supports authentication (Unauthenticated user can't subscribe) * Supports multiple protocols (WebSocket，TCP，HTTP） * Scalable architecture (Unlimited dynamic job and logic modules) * Asynchronous push notification based on Kafka ## Architecture ![arch](./docs/arch.png) ## Quick Start ### Build ``` make build ``` ### Run ``` make run make stop // or nohup target/logic -conf=target/logic.toml -region=sh -zone=sh001 -deploy.env=dev -weight=10 2>&1 > target/logic.log & nohup target/comet -conf=target/comet.toml -region=sh -zone=sh001 -deploy.env=dev -weight=10 -addrs=127.0.0.1 2>&1 > target/logic.log & nohup target/job -conf=target/job.toml -region=sh -zone=sh001 -deploy.env=dev 2>&1 > target/logic.log & ``` ### Environment ``` env: export REGION=sh export ZONE=sh001 export DEPLOY_ENV=dev supervisor: environment=REGION=sh,ZONE=sh001,DEPLOY_ENV=dev go flag: -region=sh -zone=sh001 deploy.env=dev ``` ### Configuration You can view the comments in target/comet.toml,logic.toml,job.toml to understand the meaning of the config. ### Dependencies [Discovery](https://github.com/bilibili/discovery) [Kafka](https://kafka.apache.org/quickstart) ## Document [Protocol](./docs/protocol.png) [English](./README_en.md) [中文](./README_cn.md) ## Examples Websocket: [Websocket Client Demo](https://github.com/Terry-Mao/goim/tree/master/examples/javascript) Android: [Android](https://github.com/roamdy/goim-sdk) iOS: [iOS](https://github.com/roamdy/goim-oc-sdk) ## Benchmark ![benchmark](./docs/benchmark.jpg) ### Benchmark Server | CPU | Memory | OS | Instance | | :---- | :---- | :---- | :---- | | Intel(R) Xeon(R) CPU E5-2630 v2 @ 2.60GHz | DDR3 32GB | Debian GNU/Linux 8 | 1 | ### Benchmark Case * Online: 1,000,000 * Duration: 15min * Push Speed: 40/s (broadcast room) * Push Message: {"test":1} * Received calc mode: 1s per times, total 30 times ### Benchmark Resource * CPU: 2000%~2300% * Memory: 14GB * GC Pause: 504ms * Network: Incoming(450MBit/s), Outgoing(4.39GBit/s) ### Benchmark Result * Received: 35,900,000/s [中文](./docs/benchmark_cn.md) [English](./docs/benchmark_en.md) ## LICENSE goim is is distributed under the terms of the MIT License. ================================================ FILE: README_cn.md ================================================ goim v2.0 ============== `Terry-Mao/goim` 是一个支持集群的im及实时推送服务。 --------------------------------------- * [特性](#特性) * [安装](#安装) * [配置](#配置) * [例子](#例子) * [文档](#文档) * [集群](#集群) * [更多](#更多) --------------------------------------- ## 特性 * 轻量级 * 高性能 * 纯Golang实现 * 支持单个、多个、单房间以及广播消息推送 * 支持单个Key多个订阅者（可限制订阅者最大人数） * 心跳支持（应用心跳和tcp、keepalive） * 支持安全验证（未授权用户不能订阅） * 多协议支持（websocket，tcp） * 可拓扑的架构（job、logic模块可动态无限扩展） * 基于Kafka做异步消息推送 ## 安装 ### 一、安装依赖 ```sh $ yum -y install java-1.7.0-openjdk ``` ### 二、安装Kafka消息队列服务 kafka在官网已经描述的非常详细，在这里就不过多说明，安装、启动请查看[这里](http://kafka.apache.org/documentation.html#quickstart). ### 三、搭建golang环境 1.下载源码(根据自己的系统下载对应的[安装包](http://golang.org/dl/)) ```sh $ cd /data/programfiles $ wget -c --no-check-certificate https://storage.googleapis.com/golang/go1.5.2.linux-amd64.tar.gz $ tar -xvf go1.5.2.linux-amd64.tar.gz -C /usr/local ``` 2.配置GO环境变量 (这里我加在/etc/profile.d/golang.sh) ```sh $ vi /etc/profile.d/golang.sh # 将以下环境变量添加到profile最后面 export GOROOT=/usr/local/go export PATH=$PATH:$GOROOT/bin export GOPATH=/data/apps/go $ source /etc/profile ``` ### 四、部署goim 1.下载goim及依赖包 ```sh $ yum install hg $ go get -u github.com/Terry-Mao/goim $ mv $GOPATH/src/github.com/Terry-Mao/goim $GOPATH/src/goim $ cd $GOPATH/src/goim $ go get ./... ``` 2.安装router、logic、comet、job模块(配置文件请依据实际机器环境配置) ```sh $ cd $GOPATH/src/goim/router $ go install $ cp router-example.conf $GOPATH/bin/router.conf $ cp router-log.xml $GOPATH/bin/ $ cd ../logic/ $ go install $ cp logic-example.conf $GOPATH/bin/logic.conf $ cp logic-log.xml $GOPATH/bin/ $ cd ../comet/ $ go install $ cp comet-example.conf $GOPATH/bin/comet.conf $ cp comet-log.xml $GOPATH/bin/ $ cd ../logic/job/ $ go install $ cp job-example.conf $GOPATH/bin/job.conf $ cp job-log.xml $GOPATH/bin/ ``` 到此所有的环境都搭建完成！ ### 五、启动goim ```sh $ cd /$GOPATH/bin $ nohup $GOPATH/bin/router -c $GOPATH/bin/router.conf 2>&1 > /data/logs/goim/panic-router.log & $ nohup $GOPATH/bin/logic -c $GOPATH/bin/logic.conf 2>&1 > /data/logs/goim/panic-logic.log & $ nohup $GOPATH/bin/comet -c $GOPATH/bin/comet.conf 2>&1 > /data/logs/goim/panic-comet.log & $ nohup $GOPATH/bin/job -c $GOPATH/bin/job.conf 2>&1 > /data/logs/goim/panic-job.log & ``` 如果启动失败，默认配置可通过查看panic-xxx.log日志文件来排查各个模块问题. ### 六、测试 推送协议可查看[push http协议文档](./docs/push.md) ## 配置 TODO ## 例子 Websocket: [Websocket Client Demo](https://github.com/Terry-Mao/goim/tree/master/examples/javascript) Android: [Android](https://github.com/roamdy/goim-sdk) iOS: [iOS](https://github.com/roamdy/goim-oc-sdk) ## 文档 [push http协议文档](./docs/push.md)推送接口 ## 集群 ### comet comet 属于接入层，非常容易扩展，直接开启多个comet节点，修改配置文件中的base节点下的server.id修改成不同值（注意一定要保证不同的comet进程值唯一），前端接入可以使用LVS 或者 DNS来转发 ### logic logic 属于无状态的逻辑层，可以随意增加节点，使用nginx upstream来扩展http接口，内部rpc部分，可以使用LVS四层转发 ### kafka kafka 可以使用多broker，或者多partition来扩展队列 ### router router 属于有状态节点，logic可以使用一致性hash配置节点，增加多个router节点（目前还不支持动态扩容），提前预估好在线和压力情况 ### job job 根据kafka的partition来扩展多job工作方式，具体可以参考下kafka的partition负载 ## 更多 TODO ================================================ FILE: README_en.md ================================================ goim ============== `Terry-Mao/goim` is a IM and push notification server cluster. --------------------------------------- * [Features](#features) * [Installing](#installing) * [Configurations](#configurations) * [Examples](#examples) * [Documents](#documents) * [More](#more) --------------------------------------- ## Features * Light weight * High performance * Pure Golang * Supports single push, multiple push, room push and broadcasting * Supports one key to multiple subscribers (Configurable maximum subscribers count) * Supports heartbeats (Application heartbeats, TCP, KeepAlive) * Supports authentication (Unauthenticated user can't subscribe) * Supports multiple protocols (WebSocket，TCP） * Scalable architecture (Unlimited dynamic job and logic modules) * Asynchronous push notification based on Kafka ## Installing ### Dependencies ```sh $ yum -y install java-1.7.0-openjdk ``` ### Install Kafka Please follow the official quick start [here](http://kafka.apache.org/documentation.html#quickstart). ### Install Golang environment Please follow the official quick start [here](https://golang.org/doc/install). ### Deploy goim 1.Download goim ```sh $ yum install git $ cd $GOPATH/src $ git clone https://github.com/Terry-Mao/goim.git $ cd $GOPATH/src/goim $ go get ./... ``` 2.Install router、logic、comet、job modules(You might need to change the configuration files based on your servers) ```sh $ cd $GOPATH/src/goim/router $ go install $ cp router-example.conf $GOPATH/bin/router.conf $ cp router-log.xml $GOPATH/bin/ $ cd ../logic/ $ go install $ cp logic-example.conf $GOPATH/bin/logic.conf $ cp logic-log.xml $GOPATH/bin/ $ cd ../comet/ $ go install $ cp comet-example.conf $GOPATH/bin/comet.conf $ cp comet-log.xml $GOPATH/bin/ $ cd ../logic/job/ $ go install $ cp job-example.conf $GOPATH/bin/job.conf $ cp job-log.xml $GOPATH/bin/ ``` Everything is DONE! ### Run goim You may need to change the log files location. ```sh $ cd /$GOPATH/bin $ nohup $GOPATH/bin/router -c $GOPATH/bin/router.conf 2>&1 > /data/logs/goim/panic-router.log & $ nohup $GOPATH/bin/logic -c $GOPATH/bin/logic.conf 2>&1 > /data/logs/goim/panic-logic.log & $ nohup $GOPATH/bin/comet -c $GOPATH/bin/comet.conf 2>&1 > /data/logs/goim/panic-comet.log & $ nohup $GOPATH/bin/job -c $GOPATH/bin/job.conf 2>&1 > /data/logs/goim/panic-job.log & ``` If it fails, please check the logs for debugging. ### Testing Check the push protocols here[push HTTP protocols](./docs/push.md) ## Configurations TODO ## Examples Websocket: [Websocket Client Demo](https://github.com/Terry-Mao/goim/tree/master/examples/javascript) Android: [Android SDK](https://github.com/roamdy/goim-sdk) iOS: [iOS](https://github.com/roamdy/goim-oc-sdk) ## Documents [push HTTP protocols](./docs/en/push.md) [Comet client protocols](./docs/en/proto.md) ##More TODO ================================================ FILE: api/comet/comet.pb.go ================================================ // Code generated by protoc-gen-go. DO NOT EDIT. // source: comet/comet.proto package comet import ( context "context" fmt "fmt" protocol "github.com/Terry-Mao/goim/api/protocol" proto "github.com/golang/protobuf/proto" grpc "google.golang.org/grpc" codes "google.golang.org/grpc/codes" status "google.golang.org/grpc/status" math "math" ) // Reference imports to suppress errors if they are not otherwise used. var _ = proto.Marshal var _ = fmt.Errorf var _ = math.Inf // This is a compile-time assertion to ensure that this generated file // is compatible with the proto package it is being compiled against. // A compilation error at this line likely means your copy of the // proto package needs to be updated. const _ = proto.ProtoPackageIsVersion3 // please upgrade the proto package type PushMsgReq struct { Keys []string `protobuf:"bytes,1,rep,name=keys,proto3" json:"keys,omitempty"` ProtoOp int32 `protobuf:"varint,3,opt,name=protoOp,proto3" json:"protoOp,omitempty"` Proto *protocol.Proto `protobuf:"bytes,2,opt,name=proto,proto3" json:"proto,omitempty"` XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` } func (m *PushMsgReq) Reset() { *m = PushMsgReq{} } func (m *PushMsgReq) String() string { return proto.CompactTextString(m) } func (*PushMsgReq) ProtoMessage() {} func (*PushMsgReq) Descriptor() ([]byte, []int) { return fileDescriptor_327b4a7d084564be, []int{0} } func (m *PushMsgReq) XXX_Unmarshal(b []byte) error { return xxx_messageInfo_PushMsgReq.Unmarshal(m, b) } func (m *PushMsgReq) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { return xxx_messageInfo_PushMsgReq.Marshal(b, m, deterministic) } func (m *PushMsgReq) XXX_Merge(src proto.Message) { xxx_messageInfo_PushMsgReq.Merge(m, src) } func (m *PushMsgReq) XXX_Size() int { return xxx_messageInfo_PushMsgReq.Size(m) } func (m *PushMsgReq) XXX_DiscardUnknown() { xxx_messageInfo_PushMsgReq.DiscardUnknown(m) } var xxx_messageInfo_PushMsgReq proto.InternalMessageInfo func (m *PushMsgReq) GetKeys() []string { if m != nil { return m.Keys } return nil } func (m *PushMsgReq) GetProtoOp() int32 { if m != nil { return m.ProtoOp } return 0 } func (m *PushMsgReq) GetProto() *protocol.Proto { if m != nil { return m.Proto } return nil } type PushMsgReply struct { XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` } func (m *PushMsgReply) Reset() { *m = PushMsgReply{} } func (m *PushMsgReply) String() string { return proto.CompactTextString(m) } func (*PushMsgReply) ProtoMessage() {} func (*PushMsgReply) Descriptor() ([]byte, []int) { return fileDescriptor_327b4a7d084564be, []int{1} } func (m *PushMsgReply) XXX_Unmarshal(b []byte) error { return xxx_messageInfo_PushMsgReply.Unmarshal(m, b) } func (m *PushMsgReply) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { return xxx_messageInfo_PushMsgReply.Marshal(b, m, deterministic) } func (m *PushMsgReply) XXX_Merge(src proto.Message) { xxx_messageInfo_PushMsgReply.Merge(m, src) } func (m *PushMsgReply) XXX_Size() int { return xxx_messageInfo_PushMsgReply.Size(m) } func (m *PushMsgReply) XXX_DiscardUnknown() { xxx_messageInfo_PushMsgReply.DiscardUnknown(m) } var xxx_messageInfo_PushMsgReply proto.InternalMessageInfo type BroadcastReq struct { ProtoOp int32 `protobuf:"varint,1,opt,name=protoOp,proto3" json:"protoOp,omitempty"` Proto *protocol.Proto `protobuf:"bytes,2,opt,name=proto,proto3" json:"proto,omitempty"` Speed int32 `protobuf:"varint,3,opt,name=speed,proto3" json:"speed,omitempty"` XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` } func (m *BroadcastReq) Reset() { *m = BroadcastReq{} } func (m *BroadcastReq) String() string { return proto.CompactTextString(m) } func (*BroadcastReq) ProtoMessage() {} func (*BroadcastReq) Descriptor() ([]byte, []int) { return fileDescriptor_327b4a7d084564be, []int{2} } func (m *BroadcastReq) XXX_Unmarshal(b []byte) error { return xxx_messageInfo_BroadcastReq.Unmarshal(m, b) } func (m *BroadcastReq) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { return xxx_messageInfo_BroadcastReq.Marshal(b, m, deterministic) } func (m *BroadcastReq) XXX_Merge(src proto.Message) { xxx_messageInfo_BroadcastReq.Merge(m, src) } func (m *BroadcastReq) XXX_Size() int { return xxx_messageInfo_BroadcastReq.Size(m) } func (m *BroadcastReq) XXX_DiscardUnknown() { xxx_messageInfo_BroadcastReq.DiscardUnknown(m) } var xxx_messageInfo_BroadcastReq proto.InternalMessageInfo func (m *BroadcastReq) GetProtoOp() int32 { if m != nil { return m.ProtoOp } return 0 } func (m *BroadcastReq) GetProto() *protocol.Proto { if m != nil { return m.Proto } return nil } func (m *BroadcastReq) GetSpeed() int32 { if m != nil { return m.Speed } return 0 } type BroadcastReply struct { XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` } func (m *BroadcastReply) Reset() { *m = BroadcastReply{} } func (m *BroadcastReply) String() string { return proto.CompactTextString(m) } func (*BroadcastReply) ProtoMessage() {} func (*BroadcastReply) Descriptor() ([]byte, []int) { return fileDescriptor_327b4a7d084564be, []int{3} } func (m *BroadcastReply) XXX_Unmarshal(b []byte) error { return xxx_messageInfo_BroadcastReply.Unmarshal(m, b) } func (m *BroadcastReply) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { return xxx_messageInfo_BroadcastReply.Marshal(b, m, deterministic) } func (m *BroadcastReply) XXX_Merge(src proto.Message) { xxx_messageInfo_BroadcastReply.Merge(m, src) } func (m *BroadcastReply) XXX_Size() int { return xxx_messageInfo_BroadcastReply.Size(m) } func (m *BroadcastReply) XXX_DiscardUnknown() { xxx_messageInfo_BroadcastReply.DiscardUnknown(m) } var xxx_messageInfo_BroadcastReply proto.InternalMessageInfo type BroadcastRoomReq struct { RoomID string `protobuf:"bytes,1,opt,name=roomID,proto3" json:"roomID,omitempty"` Proto *protocol.Proto `protobuf:"bytes,2,opt,name=proto,proto3" json:"proto,omitempty"` XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` } func (m *BroadcastRoomReq) Reset() { *m = BroadcastRoomReq{} } func (m *BroadcastRoomReq) String() string { return proto.CompactTextString(m) } func (*BroadcastRoomReq) ProtoMessage() {} func (*BroadcastRoomReq) Descriptor() ([]byte, []int) { return fileDescriptor_327b4a7d084564be, []int{4} } func (m *BroadcastRoomReq) XXX_Unmarshal(b []byte) error { return xxx_messageInfo_BroadcastRoomReq.Unmarshal(m, b) } func (m *BroadcastRoomReq) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { return xxx_messageInfo_BroadcastRoomReq.Marshal(b, m, deterministic) } func (m *BroadcastRoomReq) XXX_Merge(src proto.Message) { xxx_messageInfo_BroadcastRoomReq.Merge(m, src) } func (m *BroadcastRoomReq) XXX_Size() int { return xxx_messageInfo_BroadcastRoomReq.Size(m) } func (m *BroadcastRoomReq) XXX_DiscardUnknown() { xxx_messageInfo_BroadcastRoomReq.DiscardUnknown(m) } var xxx_messageInfo_BroadcastRoomReq proto.InternalMessageInfo func (m *BroadcastRoomReq) GetRoomID() string { if m != nil { return m.RoomID } return "" } func (m *BroadcastRoomReq) GetProto() *protocol.Proto { if m != nil { return m.Proto } return nil } type BroadcastRoomReply struct { XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` } func (m *BroadcastRoomReply) Reset() { *m = BroadcastRoomReply{} } func (m *BroadcastRoomReply) String() string { return proto.CompactTextString(m) } func (*BroadcastRoomReply) ProtoMessage() {} func (*BroadcastRoomReply) Descriptor() ([]byte, []int) { return fileDescriptor_327b4a7d084564be, []int{5} } func (m *BroadcastRoomReply) XXX_Unmarshal(b []byte) error { return xxx_messageInfo_BroadcastRoomReply.Unmarshal(m, b) } func (m *BroadcastRoomReply) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { return xxx_messageInfo_BroadcastRoomReply.Marshal(b, m, deterministic) } func (m *BroadcastRoomReply) XXX_Merge(src proto.Message) { xxx_messageInfo_BroadcastRoomReply.Merge(m, src) } func (m *BroadcastRoomReply) XXX_Size() int { return xxx_messageInfo_BroadcastRoomReply.Size(m) } func (m *BroadcastRoomReply) XXX_DiscardUnknown() { xxx_messageInfo_BroadcastRoomReply.DiscardUnknown(m) } var xxx_messageInfo_BroadcastRoomReply proto.InternalMessageInfo type RoomsReq struct { XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` } func (m *RoomsReq) Reset() { *m = RoomsReq{} } func (m *RoomsReq) String() string { return proto.CompactTextString(m) } func (*RoomsReq) ProtoMessage() {} func (*RoomsReq) Descriptor() ([]byte, []int) { return fileDescriptor_327b4a7d084564be, []int{6} } func (m *RoomsReq) XXX_Unmarshal(b []byte) error { return xxx_messageInfo_RoomsReq.Unmarshal(m, b) } func (m *RoomsReq) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { return xxx_messageInfo_RoomsReq.Marshal(b, m, deterministic) } func (m *RoomsReq) XXX_Merge(src proto.Message) { xxx_messageInfo_RoomsReq.Merge(m, src) } func (m *RoomsReq) XXX_Size() int { return xxx_messageInfo_RoomsReq.Size(m) } func (m *RoomsReq) XXX_DiscardUnknown() { xxx_messageInfo_RoomsReq.DiscardUnknown(m) } var xxx_messageInfo_RoomsReq proto.InternalMessageInfo type RoomsReply struct { Rooms map[string]bool `protobuf:"bytes,1,rep,name=rooms,proto3" json:"rooms,omitempty" protobuf_key:"bytes,1,opt,name=key,proto3" protobuf_val:"varint,2,opt,name=value,proto3"` XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` } func (m *RoomsReply) Reset() { *m = RoomsReply{} } func (m *RoomsReply) String() string { return proto.CompactTextString(m) } func (*RoomsReply) ProtoMessage() {} func (*RoomsReply) Descriptor() ([]byte, []int) { return fileDescriptor_327b4a7d084564be, []int{7} } func (m *RoomsReply) XXX_Unmarshal(b []byte) error { return xxx_messageInfo_RoomsReply.Unmarshal(m, b) } func (m *RoomsReply) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { return xxx_messageInfo_RoomsReply.Marshal(b, m, deterministic) } func (m *RoomsReply) XXX_Merge(src proto.Message) { xxx_messageInfo_RoomsReply.Merge(m, src) } func (m *RoomsReply) XXX_Size() int { return xxx_messageInfo_RoomsReply.Size(m) } func (m *RoomsReply) XXX_DiscardUnknown() { xxx_messageInfo_RoomsReply.DiscardUnknown(m) } var xxx_messageInfo_RoomsReply proto.InternalMessageInfo func (m *RoomsReply) GetRooms() map[string]bool { if m != nil { return m.Rooms } return nil } func init() { proto.RegisterType((*PushMsgReq)(nil), "goim.comet.PushMsgReq") proto.RegisterType((*PushMsgReply)(nil), "goim.comet.PushMsgReply") proto.RegisterType((*BroadcastReq)(nil), "goim.comet.BroadcastReq") proto.RegisterType((*BroadcastReply)(nil), "goim.comet.BroadcastReply") proto.RegisterType((*BroadcastRoomReq)(nil), "goim.comet.BroadcastRoomReq") proto.RegisterType((*BroadcastRoomReply)(nil), "goim.comet.BroadcastRoomReply") proto.RegisterType((*RoomsReq)(nil), "goim.comet.RoomsReq") proto.RegisterType((*RoomsReply)(nil), "goim.comet.RoomsReply") proto.RegisterMapType((map[string]bool)(nil), "goim.comet.RoomsReply.RoomsEntry") } func init() { proto.RegisterFile("comet/comet.proto", fileDescriptor_327b4a7d084564be) } var fileDescriptor_327b4a7d084564be = []byte{ // 404 bytes of a gzipped FileDescriptorProto 0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0x9c, 0x53, 0xcf, 0x4f, 0xe2, 0x40, 0x14, 0x4e, 0x61, 0xcb, 0x8f, 0x07, 0x4b, 0xd8, 0x49, 0x43, 0x9a, 0x66, 0xb3, 0x61, 0x7b, 0x62, 0xd7, 0xd8, 0x26, 0x18, 0x22, 0x91, 0x93, 0xa8, 0x07, 0x0f, 0x44, 0x32, 0x31, 0x1e, 0xbc, 0x15, 0x18, 0x01, 0x69, 0x9d, 0xd2, 0x16, 0x93, 0x9e, 0xfc, 0x0b, 0xfc, 0x9f, 0xcd, 0x9b, 0xa9, 0x6d, 0x31, 0x95, 0x44, 0x2f, 0xcd, 0xf7, 0xde, 0x7c, 0xf3, 0xbe, 0x6f, 0xbe, 0x07, 0xf0, 0x6b, 0xce, 0x3d, 0x16, 0xd9, 0xe2, 0x6b, 0xf9, 0x01, 0x8f, 0x38, 0x81, 0x25, 0x5f, 0x7b, 0x96, 0xe8, 0x18, 0x83, 0xe5, 0x3a, 0x5a, 0xed, 0x66, 0x58, 0xd9, 0xb7, 0x2c, 0x08, 0xe2, 0xe3, 0x89, 0xc3, 0x6d, 0x24, 0xd8, 0x8e, 0xbf, 0xb6, 0xc5, 0x85, 0x39, 0x77, 0x53, 0x20, 0x47, 0x98, 0x0f, 0x00, 0xd3, 0x5d, 0xb8, 0x9a, 0x84, 0x4b, 0xca, 0xb6, 0x84, 0xc0, 0x8f, 0x0d, 0x8b, 0x43, 0x5d, 0xe9, 0x96, 0x7b, 0x75, 0x2a, 0x30, 0xd1, 0xa1, 0x2a, 0xa8, 0x37, 0xbe, 0x5e, 0xee, 0x2a, 0x3d, 0x95, 0xbe, 0x97, 0xe4, 0x3f, 0xa8, 0x02, 0xea, 0xa5, 0xae, 0xd2, 0x6b, 0xf4, 0x35, 0x4b, 0xd8, 0x49, 0x05, 0xa6, 0x08, 0xa8, 0xa4, 0x98, 0x2d, 0x68, 0xa6, 0x3a, 0xbe, 0x1b, 0x9b, 0x8f, 0xd0, 0x1c, 0x07, 0xdc, 0x59, 0xcc, 0x9d, 0x30, 0x42, 0xe5, 0x9c, 0x8a, 0xf2, 0x6d, 0x15, 0xa2, 0x81, 0x1a, 0xfa, 0x8c, 0x2d, 0x12, 0xa7, 0xb2, 0x30, 0xdb, 0xd0, 0xca, 0x69, 0xa1, 0xfa, 0x1d, 0xb4, 0xb3, 0x0e, 0xe7, 0x1e, 0x3a, 0xe8, 0x40, 0x25, 0xe0, 0xdc, 0xbb, 0xbe, 0x14, 0x06, 0xea, 0x34, 0xa9, 0xbe, 0xf4, 0x4a, 0x0d, 0xc8, 0x87, 0xb9, 0xa8, 0x06, 0x50, 0xc3, 0x22, 0xa4, 0x6c, 0x6b, 0xbe, 0x00, 0x24, 0xd8, 0x77, 0x63, 0x72, 0x0a, 0x2a, 0xaa, 0xc8, 0xc0, 0x1b, 0xfd, 0xbf, 0x56, 0xb6, 0x50, 0x2b, 0xa3, 0x49, 0x78, 0xf5, 0x14, 0x05, 0x31, 0x95, 0x7c, 0x63, 0x98, 0x8c, 0x11, 0x4d, 0xd2, 0x86, 0xf2, 0x86, 0xc5, 0x89, 0x6f, 0x84, 0x18, 0xc4, 0xb3, 0xe3, 0xee, 0x98, 0x30, 0x5d, 0xa3, 0xb2, 0x38, 0x2b, 0x0d, 0x95, 0xfe, 0x6b, 0x09, 0xd4, 0x0b, 0x14, 0x20, 0x23, 0xa8, 0x26, 0x2b, 0x21, 0x9d, 0xbc, 0x70, 0xf6, 0x7b, 0x30, 0xf4, 0xc2, 0x3e, 0x3a, 0x3f, 0x87, 0x7a, 0xfa, 0x52, 0xb2, 0x47, 0xcb, 0xaf, 0xd5, 0x30, 0x3e, 0x39, 0xc1, 0x11, 0x13, 0xf8, 0xb9, 0x17, 0x16, 0xf9, 0x5d, 0x4c, 0x96, 0xfb, 0x31, 0xfe, 0x1c, 0x38, 0xc5, 0x71, 0x03, 0x50, 0x45, 0x24, 0x44, 0x2b, 0x48, 0x71, 0x6b, 0x74, 0x8a, 0xb3, 0x1d, 0x1f, 0xdd, 0xff, 0x3b, 0xfc, 0xcf, 0x11, 0x37, 0x46, 0xe2, 0x3b, 0xab, 0x88, 0x35, 0x9f, 0xbc, 0x05, 0x00, 0x00, 0xff, 0xff, 0xb3, 0xcb, 0xf3, 0x2a, 0x8c, 0x03, 0x00, 0x00, } // Reference imports to suppress errors if they are not otherwise used. var _ context.Context var _ grpc.ClientConn // This is a compile-time assertion to ensure that this generated file // is compatible with the grpc package it is being compiled against. const _ = grpc.SupportPackageIsVersion4 // CometClient is the client API for Comet service. // // For semantics around ctx use and closing/ending streaming RPCs, please refer to https://godoc.org/google.golang.org/grpc#ClientConn.NewStream. type CometClient interface { // PushMsg push by key or mid PushMsg(ctx context.Context, in *PushMsgReq, opts ...grpc.CallOption) (*PushMsgReply, error) // Broadcast send to every enrity Broadcast(ctx context.Context, in *BroadcastReq, opts ...grpc.CallOption) (*BroadcastReply, error) // BroadcastRoom broadcast to one room BroadcastRoom(ctx context.Context, in *BroadcastRoomReq, opts ...grpc.CallOption) (*BroadcastRoomReply, error) // Rooms get all rooms Rooms(ctx context.Context, in *RoomsReq, opts ...grpc.CallOption) (*RoomsReply, error) } type cometClient struct { cc *grpc.ClientConn } func NewCometClient(cc *grpc.ClientConn) CometClient { return &cometClient{cc} } func (c *cometClient) PushMsg(ctx context.Context, in *PushMsgReq, opts ...grpc.CallOption) (*PushMsgReply, error) { out := new(PushMsgReply) err := c.cc.Invoke(ctx, "/goim.comet.Comet/PushMsg", in, out, opts...) if err != nil { return nil, err } return out, nil } func (c *cometClient) Broadcast(ctx context.Context, in *BroadcastReq, opts ...grpc.CallOption) (*BroadcastReply, error) { out := new(BroadcastReply) err := c.cc.Invoke(ctx, "/goim.comet.Comet/Broadcast", in, out, opts...) if err != nil { return nil, err } return out, nil } func (c *cometClient) BroadcastRoom(ctx context.Context, in *BroadcastRoomReq, opts ...grpc.CallOption) (*BroadcastRoomReply, error) { out := new(BroadcastRoomReply) err := c.cc.Invoke(ctx, "/goim.comet.Comet/BroadcastRoom", in, out, opts...) if err != nil { return nil, err } return out, nil } func (c *cometClient) Rooms(ctx context.Context, in *RoomsReq, opts ...grpc.CallOption) (*RoomsReply, error) { out := new(RoomsReply) err := c.cc.Invoke(ctx, "/goim.comet.Comet/Rooms", in, out, opts...) if err != nil { return nil, err } return out, nil } // CometServer is the server API for Comet service. type CometServer interface { // PushMsg push by key or mid PushMsg(context.Context, *PushMsgReq) (*PushMsgReply, error) // Broadcast send to every enrity Broadcast(context.Context, *BroadcastReq) (*BroadcastReply, error) // BroadcastRoom broadcast to one room BroadcastRoom(context.Context, *BroadcastRoomReq) (*BroadcastRoomReply, error) // Rooms get all rooms Rooms(context.Context, *RoomsReq) (*RoomsReply, error) } // UnimplementedCometServer can be embedded to have forward compatible implementations. type UnimplementedCometServer struct { } func (*UnimplementedCometServer) PushMsg(ctx context.Context, req *PushMsgReq) (*PushMsgReply, error) { return nil, status.Errorf(codes.Unimplemented, "method PushMsg not implemented") } func (*UnimplementedCometServer) Broadcast(ctx context.Context, req *BroadcastReq) (*BroadcastReply, error) { return nil, status.Errorf(codes.Unimplemented, "method Broadcast not implemented") } func (*UnimplementedCometServer) BroadcastRoom(ctx context.Context, req *BroadcastRoomReq) (*BroadcastRoomReply, error) { return nil, status.Errorf(codes.Unimplemented, "method BroadcastRoom not implemented") } func (*UnimplementedCometServer) Rooms(ctx context.Context, req *RoomsReq) (*RoomsReply, error) { return nil, status.Errorf(codes.Unimplemented, "method Rooms not implemented") } func RegisterCometServer(s *grpc.Server, srv CometServer) { s.RegisterService(&_Comet_serviceDesc, srv) } func _Comet_PushMsg_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { in := new(PushMsgReq) if err := dec(in); err != nil { return nil, err } if interceptor == nil { return srv.(CometServer).PushMsg(ctx, in) } info := &grpc.UnaryServerInfo{ Server: srv, FullMethod: "/goim.comet.Comet/PushMsg", } handler := func(ctx context.Context, req interface{}) (interface{}, error) { return srv.(CometServer).PushMsg(ctx, req.(*PushMsgReq)) } return interceptor(ctx, in, info, handler) } func _Comet_Broadcast_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { in := new(BroadcastReq) if err := dec(in); err != nil { return nil, err } if interceptor == nil { return srv.(CometServer).Broadcast(ctx, in) } info := &grpc.UnaryServerInfo{ Server: srv, FullMethod: "/goim.comet.Comet/Broadcast", } handler := func(ctx context.Context, req interface{}) (interface{}, error) { return srv.(CometServer).Broadcast(ctx, req.(*BroadcastReq)) } return interceptor(ctx, in, info, handler) } func _Comet_BroadcastRoom_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { in := new(BroadcastRoomReq) if err := dec(in); err != nil { return nil, err } if interceptor == nil { return srv.(CometServer).BroadcastRoom(ctx, in) } info := &grpc.UnaryServerInfo{ Server: srv, FullMethod: "/goim.comet.Comet/BroadcastRoom", } handler := func(ctx context.Context, req interface{}) (interface{}, error) { return srv.(CometServer).BroadcastRoom(ctx, req.(*BroadcastRoomReq)) } return interceptor(ctx, in, info, handler) } func _Comet_Rooms_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { in := new(RoomsReq) if err := dec(in); err != nil { return nil, err } if interceptor == nil { return srv.(CometServer).Rooms(ctx, in) } info := &grpc.UnaryServerInfo{ Server: srv, FullMethod: "/goim.comet.Comet/Rooms", } handler := func(ctx context.Context, req interface{}) (interface{}, error) { return srv.(CometServer).Rooms(ctx, req.(*RoomsReq)) } return interceptor(ctx, in, info, handler) } var _Comet_serviceDesc = grpc.ServiceDesc{ ServiceName: "goim.comet.Comet", HandlerType: (*CometServer)(nil), Methods: []grpc.MethodDesc{ { MethodName: "PushMsg", Handler: _Comet_PushMsg_Handler, }, { MethodName: "Broadcast", Handler: _Comet_Broadcast_Handler, }, { MethodName: "BroadcastRoom", Handler: _Comet_BroadcastRoom_Handler, }, { MethodName: "Rooms", Handler: _Comet_Rooms_Handler, }, }, Streams: []grpc.StreamDesc{}, Metadata: "comet/comet.proto", } ================================================ FILE: api/comet/comet.proto ================================================ syntax = "proto3"; package goim.comet; option go_package = "github.com/Terry-Mao/goim/api/comet;comet"; import "github.com/Terry-Mao/goim/api/protocol/protocol.proto"; message PushMsgReq { repeated string keys = 1; int32 protoOp = 3; goim.protocol.Proto proto = 2; } message PushMsgReply {} message BroadcastReq{ int32 protoOp = 1; goim.protocol.Proto proto = 2; int32 speed = 3; } message BroadcastReply{} message BroadcastRoomReq { string roomID = 1; goim.protocol.Proto proto = 2; } message BroadcastRoomReply{} message RoomsReq{} message RoomsReply { map rooms = 1; } service Comet { // PushMsg push by key or mid rpc PushMsg(PushMsgReq) returns (PushMsgReply); // Broadcast send to every enrity rpc Broadcast(BroadcastReq) returns (BroadcastReply); // BroadcastRoom broadcast to one room rpc BroadcastRoom(BroadcastRoomReq) returns (BroadcastRoomReply); // Rooms get all rooms rpc Rooms(RoomsReq) returns (RoomsReply); } ================================================ FILE: api/generate.go ================================================ package api //go:generate protoc -I. -I$GOPATH/src --go_out=plugins=grpc:. --go_opt=paths=source_relative protocol/protocol.proto //go:generate protoc -I. -I$GOPATH/src --go_out=plugins=grpc:. --go_opt=paths=source_relative comet/comet.proto //go:generate protoc -I. -I$GOPATH/src --go_out=plugins=grpc:. --go_opt=paths=source_relative logic/logic.proto ================================================ FILE: api/logic/logic.pb.go ================================================ // Code generated by protoc-gen-go. DO NOT EDIT. // source: logic/logic.proto package logic import ( context "context" fmt "fmt" protocol "github.com/Terry-Mao/goim/api/protocol" proto "github.com/golang/protobuf/proto" grpc "google.golang.org/grpc" codes "google.golang.org/grpc/codes" status "google.golang.org/grpc/status" math "math" ) // Reference imports to suppress errors if they are not otherwise used. var _ = proto.Marshal var _ = fmt.Errorf var _ = math.Inf // This is a compile-time assertion to ensure that this generated file // is compatible with the proto package it is being compiled against. // A compilation error at this line likely means your copy of the // proto package needs to be updated. const _ = proto.ProtoPackageIsVersion3 // please upgrade the proto package type PushMsg_Type int32 const ( PushMsg_PUSH PushMsg_Type = 0 PushMsg_ROOM PushMsg_Type = 1 PushMsg_BROADCAST PushMsg_Type = 2 ) var PushMsg_Type_name = map[int32]string{ 0: "PUSH", 1: "ROOM", 2: "BROADCAST", } var PushMsg_Type_value = map[string]int32{ "PUSH": 0, "ROOM": 1, "BROADCAST": 2, } func (x PushMsg_Type) String() string { return proto.EnumName(PushMsg_Type_name, int32(x)) } func (PushMsg_Type) EnumDescriptor() ([]byte, []int) { return fileDescriptor_2dfb3aef05fe3328, []int{0, 0} } type PushMsg struct { Type PushMsg_Type `protobuf:"varint,1,opt,name=type,proto3,enum=goim.logic.PushMsg_Type" json:"type,omitempty"` Operation int32 `protobuf:"varint,2,opt,name=operation,proto3" json:"operation,omitempty"` Speed int32 `protobuf:"varint,3,opt,name=speed,proto3" json:"speed,omitempty"` Server string `protobuf:"bytes,4,opt,name=server,proto3" json:"server,omitempty"` Room string `protobuf:"bytes,5,opt,name=room,proto3" json:"room,omitempty"` Keys []string `protobuf:"bytes,6,rep,name=keys,proto3" json:"keys,omitempty"` Msg []byte `protobuf:"bytes,7,opt,name=msg,proto3" json:"msg,omitempty"` XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` } func (m *PushMsg) Reset() { *m = PushMsg{} } func (m *PushMsg) String() string { return proto.CompactTextString(m) } func (*PushMsg) ProtoMessage() {} func (*PushMsg) Descriptor() ([]byte, []int) { return fileDescriptor_2dfb3aef05fe3328, []int{0} } func (m *PushMsg) XXX_Unmarshal(b []byte) error { return xxx_messageInfo_PushMsg.Unmarshal(m, b) } func (m *PushMsg) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { return xxx_messageInfo_PushMsg.Marshal(b, m, deterministic) } func (m *PushMsg) XXX_Merge(src proto.Message) { xxx_messageInfo_PushMsg.Merge(m, src) } func (m *PushMsg) XXX_Size() int { return xxx_messageInfo_PushMsg.Size(m) } func (m *PushMsg) XXX_DiscardUnknown() { xxx_messageInfo_PushMsg.DiscardUnknown(m) } var xxx_messageInfo_PushMsg proto.InternalMessageInfo func (m *PushMsg) GetType() PushMsg_Type { if m != nil { return m.Type } return PushMsg_PUSH } func (m *PushMsg) GetOperation() int32 { if m != nil { return m.Operation } return 0 } func (m *PushMsg) GetSpeed() int32 { if m != nil { return m.Speed } return 0 } func (m *PushMsg) GetServer() string { if m != nil { return m.Server } return "" } func (m *PushMsg) GetRoom() string { if m != nil { return m.Room } return "" } func (m *PushMsg) GetKeys() []string { if m != nil { return m.Keys } return nil } func (m *PushMsg) GetMsg() []byte { if m != nil { return m.Msg } return nil } type ConnectReq struct { Server string `protobuf:"bytes,1,opt,name=server,proto3" json:"server,omitempty"` Cookie string `protobuf:"bytes,2,opt,name=cookie,proto3" json:"cookie,omitempty"` Token []byte `protobuf:"bytes,3,opt,name=token,proto3" json:"token,omitempty"` XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` } func (m *ConnectReq) Reset() { *m = ConnectReq{} } func (m *ConnectReq) String() string { return proto.CompactTextString(m) } func (*ConnectReq) ProtoMessage() {} func (*ConnectReq) Descriptor() ([]byte, []int) { return fileDescriptor_2dfb3aef05fe3328, []int{1} } func (m *ConnectReq) XXX_Unmarshal(b []byte) error { return xxx_messageInfo_ConnectReq.Unmarshal(m, b) } func (m *ConnectReq) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { return xxx_messageInfo_ConnectReq.Marshal(b, m, deterministic) } func (m *ConnectReq) XXX_Merge(src proto.Message) { xxx_messageInfo_ConnectReq.Merge(m, src) } func (m *ConnectReq) XXX_Size() int { return xxx_messageInfo_ConnectReq.Size(m) } func (m *ConnectReq) XXX_DiscardUnknown() { xxx_messageInfo_ConnectReq.DiscardUnknown(m) } var xxx_messageInfo_ConnectReq proto.InternalMessageInfo func (m *ConnectReq) GetServer() string { if m != nil { return m.Server } return "" } func (m *ConnectReq) GetCookie() string { if m != nil { return m.Cookie } return "" } func (m *ConnectReq) GetToken() []byte { if m != nil { return m.Token } return nil } type ConnectReply struct { Mid int64 `protobuf:"varint,1,opt,name=mid,proto3" json:"mid,omitempty"` Key string `protobuf:"bytes,2,opt,name=key,proto3" json:"key,omitempty"` RoomID string `protobuf:"bytes,3,opt,name=roomID,proto3" json:"roomID,omitempty"` Accepts []int32 `protobuf:"varint,4,rep,packed,name=accepts,proto3" json:"accepts,omitempty"` Heartbeat int64 `protobuf:"varint,5,opt,name=heartbeat,proto3" json:"heartbeat,omitempty"` XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` } func (m *ConnectReply) Reset() { *m = ConnectReply{} } func (m *ConnectReply) String() string { return proto.CompactTextString(m) } func (*ConnectReply) ProtoMessage() {} func (*ConnectReply) Descriptor() ([]byte, []int) { return fileDescriptor_2dfb3aef05fe3328, []int{2} } func (m *ConnectReply) XXX_Unmarshal(b []byte) error { return xxx_messageInfo_ConnectReply.Unmarshal(m, b) } func (m *ConnectReply) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { return xxx_messageInfo_ConnectReply.Marshal(b, m, deterministic) } func (m *ConnectReply) XXX_Merge(src proto.Message) { xxx_messageInfo_ConnectReply.Merge(m, src) } func (m *ConnectReply) XXX_Size() int { return xxx_messageInfo_ConnectReply.Size(m) } func (m *ConnectReply) XXX_DiscardUnknown() { xxx_messageInfo_ConnectReply.DiscardUnknown(m) } var xxx_messageInfo_ConnectReply proto.InternalMessageInfo func (m *ConnectReply) GetMid() int64 { if m != nil { return m.Mid } return 0 } func (m *ConnectReply) GetKey() string { if m != nil { return m.Key } return "" } func (m *ConnectReply) GetRoomID() string { if m != nil { return m.RoomID } return "" } func (m *ConnectReply) GetAccepts() []int32 { if m != nil { return m.Accepts } return nil } func (m *ConnectReply) GetHeartbeat() int64 { if m != nil { return m.Heartbeat } return 0 } type DisconnectReq struct { Mid int64 `protobuf:"varint,1,opt,name=mid,proto3" json:"mid,omitempty"` Key string `protobuf:"bytes,2,opt,name=key,proto3" json:"key,omitempty"` Server string `protobuf:"bytes,3,opt,name=server,proto3" json:"server,omitempty"` XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` } func (m *DisconnectReq) Reset() { *m = DisconnectReq{} } func (m *DisconnectReq) String() string { return proto.CompactTextString(m) } func (*DisconnectReq) ProtoMessage() {} func (*DisconnectReq) Descriptor() ([]byte, []int) { return fileDescriptor_2dfb3aef05fe3328, []int{3} } func (m *DisconnectReq) XXX_Unmarshal(b []byte) error { return xxx_messageInfo_DisconnectReq.Unmarshal(m, b) } func (m *DisconnectReq) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { return xxx_messageInfo_DisconnectReq.Marshal(b, m, deterministic) } func (m *DisconnectReq) XXX_Merge(src proto.Message) { xxx_messageInfo_DisconnectReq.Merge(m, src) } func (m *DisconnectReq) XXX_Size() int { return xxx_messageInfo_DisconnectReq.Size(m) } func (m *DisconnectReq) XXX_DiscardUnknown() { xxx_messageInfo_DisconnectReq.DiscardUnknown(m) } var xxx_messageInfo_DisconnectReq proto.InternalMessageInfo func (m *DisconnectReq) GetMid() int64 { if m != nil { return m.Mid } return 0 } func (m *DisconnectReq) GetKey() string { if m != nil { return m.Key } return "" } func (m *DisconnectReq) GetServer() string { if m != nil { return m.Server } return "" } type DisconnectReply struct { Has bool `protobuf:"varint,1,opt,name=has,proto3" json:"has,omitempty"` XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` } func (m *DisconnectReply) Reset() { *m = DisconnectReply{} } func (m *DisconnectReply) String() string { return proto.CompactTextString(m) } func (*DisconnectReply) ProtoMessage() {} func (*DisconnectReply) Descriptor() ([]byte, []int) { return fileDescriptor_2dfb3aef05fe3328, []int{4} } func (m *DisconnectReply) XXX_Unmarshal(b []byte) error { return xxx_messageInfo_DisconnectReply.Unmarshal(m, b) } func (m *DisconnectReply) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { return xxx_messageInfo_DisconnectReply.Marshal(b, m, deterministic) } func (m *DisconnectReply) XXX_Merge(src proto.Message) { xxx_messageInfo_DisconnectReply.Merge(m, src) } func (m *DisconnectReply) XXX_Size() int { return xxx_messageInfo_DisconnectReply.Size(m) } func (m *DisconnectReply) XXX_DiscardUnknown() { xxx_messageInfo_DisconnectReply.DiscardUnknown(m) } var xxx_messageInfo_DisconnectReply proto.InternalMessageInfo func (m *DisconnectReply) GetHas() bool { if m != nil { return m.Has } return false } type HeartbeatReq struct { Mid int64 `protobuf:"varint,1,opt,name=mid,proto3" json:"mid,omitempty"` Key string `protobuf:"bytes,2,opt,name=key,proto3" json:"key,omitempty"` Server string `protobuf:"bytes,3,opt,name=server,proto3" json:"server,omitempty"` XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` } func (m *HeartbeatReq) Reset() { *m = HeartbeatReq{} } func (m *HeartbeatReq) String() string { return proto.CompactTextString(m) } func (*HeartbeatReq) ProtoMessage() {} func (*HeartbeatReq) Descriptor() ([]byte, []int) { return fileDescriptor_2dfb3aef05fe3328, []int{5} } func (m *HeartbeatReq) XXX_Unmarshal(b []byte) error { return xxx_messageInfo_HeartbeatReq.Unmarshal(m, b) } func (m *HeartbeatReq) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { return xxx_messageInfo_HeartbeatReq.Marshal(b, m, deterministic) } func (m *HeartbeatReq) XXX_Merge(src proto.Message) { xxx_messageInfo_HeartbeatReq.Merge(m, src) } func (m *HeartbeatReq) XXX_Size() int { return xxx_messageInfo_HeartbeatReq.Size(m) } func (m *HeartbeatReq) XXX_DiscardUnknown() { xxx_messageInfo_HeartbeatReq.DiscardUnknown(m) } var xxx_messageInfo_HeartbeatReq proto.InternalMessageInfo func (m *HeartbeatReq) GetMid() int64 { if m != nil { return m.Mid } return 0 } func (m *HeartbeatReq) GetKey() string { if m != nil { return m.Key } return "" } func (m *HeartbeatReq) GetServer() string { if m != nil { return m.Server } return "" } type HeartbeatReply struct { XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` } func (m *HeartbeatReply) Reset() { *m = HeartbeatReply{} } func (m *HeartbeatReply) String() string { return proto.CompactTextString(m) } func (*HeartbeatReply) ProtoMessage() {} func (*HeartbeatReply) Descriptor() ([]byte, []int) { return fileDescriptor_2dfb3aef05fe3328, []int{6} } func (m *HeartbeatReply) XXX_Unmarshal(b []byte) error { return xxx_messageInfo_HeartbeatReply.Unmarshal(m, b) } func (m *HeartbeatReply) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { return xxx_messageInfo_HeartbeatReply.Marshal(b, m, deterministic) } func (m *HeartbeatReply) XXX_Merge(src proto.Message) { xxx_messageInfo_HeartbeatReply.Merge(m, src) } func (m *HeartbeatReply) XXX_Size() int { return xxx_messageInfo_HeartbeatReply.Size(m) } func (m *HeartbeatReply) XXX_DiscardUnknown() { xxx_messageInfo_HeartbeatReply.DiscardUnknown(m) } var xxx_messageInfo_HeartbeatReply proto.InternalMessageInfo type OnlineReq struct { Server string `protobuf:"bytes,1,opt,name=server,proto3" json:"server,omitempty"` RoomCount map[string]int32 `protobuf:"bytes,2,rep,name=roomCount,proto3" json:"roomCount,omitempty" protobuf_key:"bytes,1,opt,name=key,proto3" protobuf_val:"varint,2,opt,name=value,proto3"` XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` } func (m *OnlineReq) Reset() { *m = OnlineReq{} } func (m *OnlineReq) String() string { return proto.CompactTextString(m) } func (*OnlineReq) ProtoMessage() {} func (*OnlineReq) Descriptor() ([]byte, []int) { return fileDescriptor_2dfb3aef05fe3328, []int{7} } func (m *OnlineReq) XXX_Unmarshal(b []byte) error { return xxx_messageInfo_OnlineReq.Unmarshal(m, b) } func (m *OnlineReq) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { return xxx_messageInfo_OnlineReq.Marshal(b, m, deterministic) } func (m *OnlineReq) XXX_Merge(src proto.Message) { xxx_messageInfo_OnlineReq.Merge(m, src) } func (m *OnlineReq) XXX_Size() int { return xxx_messageInfo_OnlineReq.Size(m) } func (m *OnlineReq) XXX_DiscardUnknown() { xxx_messageInfo_OnlineReq.DiscardUnknown(m) } var xxx_messageInfo_OnlineReq proto.InternalMessageInfo func (m *OnlineReq) GetServer() string { if m != nil { return m.Server } return "" } func (m *OnlineReq) GetRoomCount() map[string]int32 { if m != nil { return m.RoomCount } return nil } type OnlineReply struct { AllRoomCount map[string]int32 `protobuf:"bytes,1,rep,name=allRoomCount,proto3" json:"allRoomCount,omitempty" protobuf_key:"bytes,1,opt,name=key,proto3" protobuf_val:"varint,2,opt,name=value,proto3"` XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` } func (m *OnlineReply) Reset() { *m = OnlineReply{} } func (m *OnlineReply) String() string { return proto.CompactTextString(m) } func (*OnlineReply) ProtoMessage() {} func (*OnlineReply) Descriptor() ([]byte, []int) { return fileDescriptor_2dfb3aef05fe3328, []int{8} } func (m *OnlineReply) XXX_Unmarshal(b []byte) error { return xxx_messageInfo_OnlineReply.Unmarshal(m, b) } func (m *OnlineReply) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { return xxx_messageInfo_OnlineReply.Marshal(b, m, deterministic) } func (m *OnlineReply) XXX_Merge(src proto.Message) { xxx_messageInfo_OnlineReply.Merge(m, src) } func (m *OnlineReply) XXX_Size() int { return xxx_messageInfo_OnlineReply.Size(m) } func (m *OnlineReply) XXX_DiscardUnknown() { xxx_messageInfo_OnlineReply.DiscardUnknown(m) } var xxx_messageInfo_OnlineReply proto.InternalMessageInfo func (m *OnlineReply) GetAllRoomCount() map[string]int32 { if m != nil { return m.AllRoomCount } return nil } type ReceiveReq struct { Mid int64 `protobuf:"varint,1,opt,name=mid,proto3" json:"mid,omitempty"` Proto *protocol.Proto `protobuf:"bytes,2,opt,name=proto,proto3" json:"proto,omitempty"` XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` } func (m *ReceiveReq) Reset() { *m = ReceiveReq{} } func (m *ReceiveReq) String() string { return proto.CompactTextString(m) } func (*ReceiveReq) ProtoMessage() {} func (*ReceiveReq) Descriptor() ([]byte, []int) { return fileDescriptor_2dfb3aef05fe3328, []int{9} } func (m *ReceiveReq) XXX_Unmarshal(b []byte) error { return xxx_messageInfo_ReceiveReq.Unmarshal(m, b) } func (m *ReceiveReq) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { return xxx_messageInfo_ReceiveReq.Marshal(b, m, deterministic) } func (m *ReceiveReq) XXX_Merge(src proto.Message) { xxx_messageInfo_ReceiveReq.Merge(m, src) } func (m *ReceiveReq) XXX_Size() int { return xxx_messageInfo_ReceiveReq.Size(m) } func (m *ReceiveReq) XXX_DiscardUnknown() { xxx_messageInfo_ReceiveReq.DiscardUnknown(m) } var xxx_messageInfo_ReceiveReq proto.InternalMessageInfo func (m *ReceiveReq) GetMid() int64 { if m != nil { return m.Mid } return 0 } func (m *ReceiveReq) GetProto() *protocol.Proto { if m != nil { return m.Proto } return nil } type ReceiveReply struct { XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` } func (m *ReceiveReply) Reset() { *m = ReceiveReply{} } func (m *ReceiveReply) String() string { return proto.CompactTextString(m) } func (*ReceiveReply) ProtoMessage() {} func (*ReceiveReply) Descriptor() ([]byte, []int) { return fileDescriptor_2dfb3aef05fe3328, []int{10} } func (m *ReceiveReply) XXX_Unmarshal(b []byte) error { return xxx_messageInfo_ReceiveReply.Unmarshal(m, b) } func (m *ReceiveReply) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { return xxx_messageInfo_ReceiveReply.Marshal(b, m, deterministic) } func (m *ReceiveReply) XXX_Merge(src proto.Message) { xxx_messageInfo_ReceiveReply.Merge(m, src) } func (m *ReceiveReply) XXX_Size() int { return xxx_messageInfo_ReceiveReply.Size(m) } func (m *ReceiveReply) XXX_DiscardUnknown() { xxx_messageInfo_ReceiveReply.DiscardUnknown(m) } var xxx_messageInfo_ReceiveReply proto.InternalMessageInfo type NodesReq struct { Platform string `protobuf:"bytes,1,opt,name=platform,proto3" json:"platform,omitempty"` ClientIP string `protobuf:"bytes,2,opt,name=clientIP,proto3" json:"clientIP,omitempty"` XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` } func (m *NodesReq) Reset() { *m = NodesReq{} } func (m *NodesReq) String() string { return proto.CompactTextString(m) } func (*NodesReq) ProtoMessage() {} func (*NodesReq) Descriptor() ([]byte, []int) { return fileDescriptor_2dfb3aef05fe3328, []int{11} } func (m *NodesReq) XXX_Unmarshal(b []byte) error { return xxx_messageInfo_NodesReq.Unmarshal(m, b) } func (m *NodesReq) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { return xxx_messageInfo_NodesReq.Marshal(b, m, deterministic) } func (m *NodesReq) XXX_Merge(src proto.Message) { xxx_messageInfo_NodesReq.Merge(m, src) } func (m *NodesReq) XXX_Size() int { return xxx_messageInfo_NodesReq.Size(m) } func (m *NodesReq) XXX_DiscardUnknown() { xxx_messageInfo_NodesReq.DiscardUnknown(m) } var xxx_messageInfo_NodesReq proto.InternalMessageInfo func (m *NodesReq) GetPlatform() string { if m != nil { return m.Platform } return "" } func (m *NodesReq) GetClientIP() string { if m != nil { return m.ClientIP } return "" } type NodesReply struct { Domain string `protobuf:"bytes,1,opt,name=domain,proto3" json:"domain,omitempty"` TcpPort int32 `protobuf:"varint,2,opt,name=tcp_port,json=tcpPort,proto3" json:"tcp_port,omitempty"` WsPort int32 `protobuf:"varint,3,opt,name=ws_port,json=wsPort,proto3" json:"ws_port,omitempty"` WssPort int32 `protobuf:"varint,4,opt,name=wss_port,json=wssPort,proto3" json:"wss_port,omitempty"` Heartbeat int32 `protobuf:"varint,5,opt,name=heartbeat,proto3" json:"heartbeat,omitempty"` Nodes []string `protobuf:"bytes,6,rep,name=nodes,proto3" json:"nodes,omitempty"` Backoff *Backoff `protobuf:"bytes,7,opt,name=backoff,proto3" json:"backoff,omitempty"` HeartbeatMax int32 `protobuf:"varint,8,opt,name=heartbeat_max,json=heartbeatMax,proto3" json:"heartbeat_max,omitempty"` XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` } func (m *NodesReply) Reset() { *m = NodesReply{} } func (m *NodesReply) String() string { return proto.CompactTextString(m) } func (*NodesReply) ProtoMessage() {} func (*NodesReply) Descriptor() ([]byte, []int) { return fileDescriptor_2dfb3aef05fe3328, []int{12} } func (m *NodesReply) XXX_Unmarshal(b []byte) error { return xxx_messageInfo_NodesReply.Unmarshal(m, b) } func (m *NodesReply) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { return xxx_messageInfo_NodesReply.Marshal(b, m, deterministic) } func (m *NodesReply) XXX_Merge(src proto.Message) { xxx_messageInfo_NodesReply.Merge(m, src) } func (m *NodesReply) XXX_Size() int { return xxx_messageInfo_NodesReply.Size(m) } func (m *NodesReply) XXX_DiscardUnknown() { xxx_messageInfo_NodesReply.DiscardUnknown(m) } var xxx_messageInfo_NodesReply proto.InternalMessageInfo func (m *NodesReply) GetDomain() string { if m != nil { return m.Domain } return "" } func (m *NodesReply) GetTcpPort() int32 { if m != nil { return m.TcpPort } return 0 } func (m *NodesReply) GetWsPort() int32 { if m != nil { return m.WsPort } return 0 } func (m *NodesReply) GetWssPort() int32 { if m != nil { return m.WssPort } return 0 } func (m *NodesReply) GetHeartbeat() int32 { if m != nil { return m.Heartbeat } return 0 } func (m *NodesReply) GetNodes() []string { if m != nil { return m.Nodes } return nil } func (m *NodesReply) GetBackoff() *Backoff { if m != nil { return m.Backoff } return nil } func (m *NodesReply) GetHeartbeatMax() int32 { if m != nil { return m.HeartbeatMax } return 0 } type Backoff struct { MaxDelay int32 `protobuf:"varint,1,opt,name=max_delay,json=maxDelay,proto3" json:"max_delay,omitempty"` BaseDelay int32 `protobuf:"varint,2,opt,name=base_delay,json=baseDelay,proto3" json:"base_delay,omitempty"` Factor float32 `protobuf:"fixed32,3,opt,name=factor,proto3" json:"factor,omitempty"` Jitter float32 `protobuf:"fixed32,4,opt,name=jitter,proto3" json:"jitter,omitempty"` XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` } func (m *Backoff) Reset() { *m = Backoff{} } func (m *Backoff) String() string { return proto.CompactTextString(m) } func (*Backoff) ProtoMessage() {} func (*Backoff) Descriptor() ([]byte, []int) { return fileDescriptor_2dfb3aef05fe3328, []int{13} } func (m *Backoff) XXX_Unmarshal(b []byte) error { return xxx_messageInfo_Backoff.Unmarshal(m, b) } func (m *Backoff) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { return xxx_messageInfo_Backoff.Marshal(b, m, deterministic) } func (m *Backoff) XXX_Merge(src proto.Message) { xxx_messageInfo_Backoff.Merge(m, src) } func (m *Backoff) XXX_Size() int { return xxx_messageInfo_Backoff.Size(m) } func (m *Backoff) XXX_DiscardUnknown() { xxx_messageInfo_Backoff.DiscardUnknown(m) } var xxx_messageInfo_Backoff proto.InternalMessageInfo func (m *Backoff) GetMaxDelay() int32 { if m != nil { return m.MaxDelay } return 0 } func (m *Backoff) GetBaseDelay() int32 { if m != nil { return m.BaseDelay } return 0 } func (m *Backoff) GetFactor() float32 { if m != nil { return m.Factor } return 0 } func (m *Backoff) GetJitter() float32 { if m != nil { return m.Jitter } return 0 } func init() { proto.RegisterEnum("goim.logic.PushMsg_Type", PushMsg_Type_name, PushMsg_Type_value) proto.RegisterType((*PushMsg)(nil), "goim.logic.PushMsg") proto.RegisterType((*ConnectReq)(nil), "goim.logic.ConnectReq") proto.RegisterType((*ConnectReply)(nil), "goim.logic.ConnectReply") proto.RegisterType((*DisconnectReq)(nil), "goim.logic.DisconnectReq") proto.RegisterType((*DisconnectReply)(nil), "goim.logic.DisconnectReply") proto.RegisterType((*HeartbeatReq)(nil), "goim.logic.HeartbeatReq") proto.RegisterType((*HeartbeatReply)(nil), "goim.logic.HeartbeatReply") proto.RegisterType((*OnlineReq)(nil), "goim.logic.OnlineReq") proto.RegisterMapType((map[string]int32)(nil), "goim.logic.OnlineReq.RoomCountEntry") proto.RegisterType((*OnlineReply)(nil), "goim.logic.OnlineReply") proto.RegisterMapType((map[string]int32)(nil), "goim.logic.OnlineReply.AllRoomCountEntry") proto.RegisterType((*ReceiveReq)(nil), "goim.logic.ReceiveReq") proto.RegisterType((*ReceiveReply)(nil), "goim.logic.ReceiveReply") proto.RegisterType((*NodesReq)(nil), "goim.logic.NodesReq") proto.RegisterType((*NodesReply)(nil), "goim.logic.NodesReply") proto.RegisterType((*Backoff)(nil), "goim.logic.Backoff") } func init() { proto.RegisterFile("logic/logic.proto", fileDescriptor_2dfb3aef05fe3328) } var fileDescriptor_2dfb3aef05fe3328 = []byte{ // 893 bytes of a gzipped FileDescriptorProto 0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xa4, 0x55, 0x4f, 0x6f, 0xe3, 0x44, 0x14, 0xc7, 0x71, 0x9c, 0xc4, 0x2f, 0x69, 0xc9, 0x0e, 0x25, 0xeb, 0x7a, 0x41, 0x8a, 0xbc, 0x1c, 0x52, 0x60, 0x13, 0x29, 0x68, 0x25, 0xc4, 0x82, 0x50, 0xd3, 0x20, 0xed, 0x2e, 0x84, 0x46, 0xb3, 0xe5, 0xc2, 0xa5, 0x9a, 0x38, 0xd3, 0xd4, 0xc4, 0xf6, 0x18, 0x7b, 0xd2, 0xd4, 0x37, 0xc4, 0xf7, 0xe0, 0xc8, 0x77, 0xe3, 0x3b, 0x70, 0x41, 0xf3, 0x27, 0xb6, 0xa3, 0x4d, 0x57, 0x20, 0x2e, 0xd6, 0xfb, 0x37, 0xbf, 0xf9, 0xbd, 0xf7, 0xe6, 0x3d, 0xc3, 0xa3, 0x90, 0xad, 0x02, 0x7f, 0x24, 0xbf, 0xc3, 0x24, 0x65, 0x9c, 0x21, 0x58, 0xb1, 0x20, 0x1a, 0x4a, 0x8b, 0xfb, 0x7c, 0x15, 0xf0, 0xdb, 0xcd, 0x62, 0xe8, 0xb3, 0x68, 0x74, 0x45, 0xd3, 0x34, 0x7f, 0x36, 0x23, 0x6c, 0x24, 0x02, 0x46, 0x24, 0x09, 0x46, 0xf2, 0x80, 0xcf, 0xc2, 0x42, 0x50, 0x10, 0xde, 0x5f, 0x06, 0x34, 0xe7, 0x9b, 0xec, 0x76, 0x96, 0xad, 0xd0, 0xe7, 0x50, 0xe7, 0x79, 0x42, 0x1d, 0xa3, 0x6f, 0x0c, 0x8e, 0xc7, 0xce, 0xb0, 0x44, 0x1f, 0xea, 0x90, 0xe1, 0x55, 0x9e, 0x50, 0x2c, 0xa3, 0xd0, 0x47, 0x60, 0xb3, 0x84, 0xa6, 0x84, 0x07, 0x2c, 0x76, 0x6a, 0x7d, 0x63, 0x60, 0xe1, 0xd2, 0x80, 0x4e, 0xc0, 0xca, 0x12, 0x4a, 0x97, 0x8e, 0x29, 0x3d, 0x4a, 0x41, 0x3d, 0x68, 0x64, 0x34, 0xbd, 0xa3, 0xa9, 0x53, 0xef, 0x1b, 0x03, 0x1b, 0x6b, 0x0d, 0x21, 0xa8, 0xa7, 0x8c, 0x45, 0x8e, 0x25, 0xad, 0x52, 0x16, 0xb6, 0x35, 0xcd, 0x33, 0xa7, 0xd1, 0x37, 0x85, 0x4d, 0xc8, 0xa8, 0x0b, 0x66, 0x94, 0xad, 0x9c, 0x66, 0xdf, 0x18, 0x74, 0xb0, 0x10, 0xbd, 0x33, 0xa8, 0x0b, 0x4e, 0xa8, 0x05, 0xf5, 0xf9, 0x4f, 0x6f, 0x5e, 0x76, 0xdf, 0x13, 0x12, 0xbe, 0xbc, 0x9c, 0x75, 0x0d, 0x74, 0x04, 0xf6, 0x04, 0x5f, 0x9e, 0x4f, 0x2f, 0xce, 0xdf, 0x5c, 0x75, 0x6b, 0x1e, 0x06, 0xb8, 0x60, 0x71, 0x4c, 0x7d, 0x8e, 0xe9, 0xaf, 0x15, 0x2a, 0xc6, 0x1e, 0x95, 0x1e, 0x34, 0x7c, 0xc6, 0xd6, 0x01, 0x95, 0x39, 0xd9, 0x58, 0x6b, 0x22, 0x21, 0xce, 0xd6, 0x34, 0x96, 0x09, 0x75, 0xb0, 0x52, 0xbc, 0xdf, 0x0d, 0xe8, 0x14, 0xa0, 0x49, 0x98, 0x4b, 0x86, 0xc1, 0x52, 0x62, 0x9a, 0x58, 0x88, 0xc2, 0xb2, 0xa6, 0xb9, 0x46, 0x13, 0xa2, 0xb8, 0x42, 0x64, 0xf8, 0x6a, 0x2a, 0xb1, 0x6c, 0xac, 0x35, 0xe4, 0x40, 0x93, 0xf8, 0x3e, 0x4d, 0x78, 0xe6, 0xd4, 0xfb, 0xe6, 0xc0, 0xc2, 0x3b, 0x55, 0xd4, 0xfa, 0x96, 0x92, 0x94, 0x2f, 0x28, 0xe1, 0xb2, 0x48, 0x26, 0x2e, 0x0d, 0xde, 0xf7, 0x70, 0x34, 0x0d, 0x32, 0xbf, 0xcc, 0xed, 0x5f, 0x92, 0xd0, 0xf9, 0x9b, 0xd5, 0xfc, 0xbd, 0xa7, 0xf0, 0x7e, 0x15, 0x4c, 0xe7, 0x74, 0x4b, 0x32, 0x09, 0xd7, 0xc2, 0x42, 0xf4, 0x5e, 0x43, 0xe7, 0xe5, 0xee, 0xfa, 0xff, 0x7b, 0x61, 0x17, 0x8e, 0x2b, 0x58, 0x49, 0x98, 0x7b, 0x7f, 0x1a, 0x60, 0x5f, 0xc6, 0x61, 0x10, 0xd3, 0x77, 0x35, 0x6a, 0x02, 0xb6, 0xa8, 0xdb, 0x05, 0xdb, 0xc4, 0xdc, 0xa9, 0xf5, 0xcd, 0x41, 0x7b, 0xfc, 0x49, 0xf5, 0xc9, 0x16, 0x08, 0x43, 0xbc, 0x0b, 0xfb, 0x2e, 0xe6, 0x69, 0x8e, 0xcb, 0x63, 0xee, 0xd7, 0x70, 0xbc, 0xef, 0xdc, 0xf1, 0x36, 0x4a, 0xde, 0x27, 0x60, 0xdd, 0x91, 0x70, 0x43, 0xf5, 0x1b, 0x57, 0xca, 0x57, 0xb5, 0x2f, 0x0d, 0xef, 0x0f, 0x03, 0xda, 0xbb, 0x5b, 0x44, 0x9d, 0x66, 0xd0, 0x21, 0x61, 0x58, 0x00, 0x3a, 0x86, 0x24, 0x75, 0x76, 0x88, 0x54, 0x12, 0xe6, 0xc3, 0xf3, 0x4a, 0xac, 0x62, 0xb6, 0x77, 0xdc, 0xfd, 0x16, 0x1e, 0xbd, 0x15, 0xf2, 0x9f, 0xf8, 0xbd, 0x06, 0xc0, 0xd4, 0xa7, 0xc1, 0x1d, 0x3d, 0xdc, 0xa3, 0x4f, 0xc1, 0x92, 0x4b, 0x40, 0x9e, 0x6c, 0x8f, 0x4f, 0x14, 0xd1, 0x62, 0x41, 0xcc, 0x85, 0x80, 0x55, 0x88, 0x77, 0x0c, 0x9d, 0x02, 0x4b, 0xf4, 0x68, 0x02, 0xad, 0x1f, 0xd9, 0x92, 0x66, 0x02, 0xd9, 0x85, 0x56, 0x12, 0x12, 0x7e, 0xc3, 0xd2, 0x48, 0x13, 0x2b, 0x74, 0xe1, 0xf3, 0xc3, 0x80, 0xc6, 0xfc, 0xd5, 0x5c, 0x3f, 0x86, 0x42, 0xf7, 0xfe, 0x36, 0x00, 0x34, 0x88, 0x28, 0x5f, 0x0f, 0x1a, 0x4b, 0x16, 0x91, 0x20, 0xde, 0x35, 0x5a, 0x69, 0xe8, 0x14, 0x5a, 0xdc, 0x4f, 0xae, 0x13, 0x96, 0x72, 0x9d, 0x63, 0x93, 0xfb, 0xc9, 0x9c, 0xa5, 0x1c, 0x3d, 0x86, 0xe6, 0x36, 0x53, 0x1e, 0xb5, 0x67, 0x1a, 0xdb, 0x4c, 0x3a, 0x4e, 0xa1, 0xb5, 0xcd, 0xb4, 0xa7, 0xae, 0xce, 0x6c, 0x33, 0xe5, 0x7a, 0x6b, 0x96, 0xac, 0xca, 0x2c, 0x89, 0x6a, 0xc6, 0x82, 0x92, 0x5e, 0x3b, 0x4a, 0x41, 0xcf, 0xa0, 0xb9, 0x20, 0xfe, 0x9a, 0xdd, 0xdc, 0xc8, 0xdd, 0xd3, 0x1e, 0x7f, 0x50, 0x6d, 0xea, 0x44, 0xb9, 0xf0, 0x2e, 0x06, 0x3d, 0x85, 0xa3, 0x02, 0xf1, 0x3a, 0x22, 0xf7, 0x4e, 0x4b, 0x5e, 0xd3, 0x29, 0x8c, 0x33, 0x72, 0xef, 0x6d, 0xa0, 0xa9, 0x0f, 0xa2, 0x27, 0x60, 0x47, 0xe4, 0xfe, 0x7a, 0x49, 0x43, 0xa2, 0x5a, 0x6b, 0xe1, 0x56, 0x44, 0xee, 0xa7, 0x42, 0x47, 0x1f, 0x03, 0x2c, 0x48, 0x46, 0xb5, 0x57, 0x2f, 0x5a, 0x61, 0x51, 0xee, 0x1e, 0x34, 0x6e, 0x88, 0xcf, 0x99, 0x1a, 0xab, 0x1a, 0xd6, 0x9a, 0xb0, 0xff, 0x12, 0x70, 0xae, 0x57, 0x6d, 0x0d, 0x6b, 0x6d, 0xfc, 0x9b, 0x09, 0xd6, 0x0f, 0x82, 0x36, 0x7a, 0x01, 0x4d, 0xbd, 0xba, 0x50, 0xaf, 0x9a, 0x4e, 0xb9, 0x24, 0x5d, 0xe7, 0xa0, 0x5d, 0x34, 0x6b, 0x0a, 0x50, 0xae, 0x09, 0x74, 0x5a, 0x8d, 0xdb, 0xdb, 0x45, 0xee, 0x93, 0x87, 0x5c, 0x02, 0xe5, 0x1c, 0xec, 0x62, 0xf6, 0xd1, 0xde, 0x65, 0xd5, 0xf5, 0xe2, 0xba, 0x0f, 0x78, 0x04, 0xc4, 0x37, 0xd0, 0xc6, 0x34, 0xa6, 0x5b, 0x35, 0x59, 0xe8, 0xc3, 0x83, 0x2b, 0xc0, 0x7d, 0xfc, 0xc0, 0x10, 0x8a, 0x22, 0xe8, 0x77, 0xbd, 0x5f, 0x84, 0x72, 0x70, 0xf6, 0x8b, 0x50, 0x1d, 0x02, 0xf4, 0x1c, 0x2c, 0xf9, 0x7e, 0xd1, 0x49, 0x35, 0x64, 0x37, 0x17, 0x6e, 0xef, 0x80, 0x35, 0x09, 0xf3, 0xc9, 0x67, 0x3f, 0x9f, 0xbd, 0xfb, 0x67, 0x2d, 0x4f, 0xbc, 0x90, 0xdf, 0x45, 0x43, 0xce, 0xdf, 0x17, 0xff, 0x04, 0x00, 0x00, 0xff, 0xff, 0x6f, 0x34, 0x54, 0x90, 0xff, 0x07, 0x00, 0x00, } // Reference imports to suppress errors if they are not otherwise used. var _ context.Context var _ grpc.ClientConn // This is a compile-time assertion to ensure that this generated file // is compatible with the grpc package it is being compiled against. const _ = grpc.SupportPackageIsVersion4 // LogicClient is the client API for Logic service. // // For semantics around ctx use and closing/ending streaming RPCs, please refer to https://godoc.org/google.golang.org/grpc#ClientConn.NewStream. type LogicClient interface { // Connect Connect(ctx context.Context, in *ConnectReq, opts ...grpc.CallOption) (*ConnectReply, error) // Disconnect Disconnect(ctx context.Context, in *DisconnectReq, opts ...grpc.CallOption) (*DisconnectReply, error) // Heartbeat Heartbeat(ctx context.Context, in *HeartbeatReq, opts ...grpc.CallOption) (*HeartbeatReply, error) // RenewOnline RenewOnline(ctx context.Context, in *OnlineReq, opts ...grpc.CallOption) (*OnlineReply, error) // Receive Receive(ctx context.Context, in *ReceiveReq, opts ...grpc.CallOption) (*ReceiveReply, error) //ServerList Nodes(ctx context.Context, in *NodesReq, opts ...grpc.CallOption) (*NodesReply, error) } type logicClient struct { cc *grpc.ClientConn } func NewLogicClient(cc *grpc.ClientConn) LogicClient { return &logicClient{cc} } func (c *logicClient) Connect(ctx context.Context, in *ConnectReq, opts ...grpc.CallOption) (*ConnectReply, error) { out := new(ConnectReply) err := c.cc.Invoke(ctx, "/goim.logic.Logic/Connect", in, out, opts...) if err != nil { return nil, err } return out, nil } func (c *logicClient) Disconnect(ctx context.Context, in *DisconnectReq, opts ...grpc.CallOption) (*DisconnectReply, error) { out := new(DisconnectReply) err := c.cc.Invoke(ctx, "/goim.logic.Logic/Disconnect", in, out, opts...) if err != nil { return nil, err } return out, nil } func (c *logicClient) Heartbeat(ctx context.Context, in *HeartbeatReq, opts ...grpc.CallOption) (*HeartbeatReply, error) { out := new(HeartbeatReply) err := c.cc.Invoke(ctx, "/goim.logic.Logic/Heartbeat", in, out, opts...) if err != nil { return nil, err } return out, nil } func (c *logicClient) RenewOnline(ctx context.Context, in *OnlineReq, opts ...grpc.CallOption) (*OnlineReply, error) { out := new(OnlineReply) err := c.cc.Invoke(ctx, "/goim.logic.Logic/RenewOnline", in, out, opts...) if err != nil { return nil, err } return out, nil } func (c *logicClient) Receive(ctx context.Context, in *ReceiveReq, opts ...grpc.CallOption) (*ReceiveReply, error) { out := new(ReceiveReply) err := c.cc.Invoke(ctx, "/goim.logic.Logic/Receive", in, out, opts...) if err != nil { return nil, err } return out, nil } func (c *logicClient) Nodes(ctx context.Context, in *NodesReq, opts ...grpc.CallOption) (*NodesReply, error) { out := new(NodesReply) err := c.cc.Invoke(ctx, "/goim.logic.Logic/Nodes", in, out, opts...) if err != nil { return nil, err } return out, nil } // LogicServer is the server API for Logic service. type LogicServer interface { // Connect Connect(context.Context, *ConnectReq) (*ConnectReply, error) // Disconnect Disconnect(context.Context, *DisconnectReq) (*DisconnectReply, error) // Heartbeat Heartbeat(context.Context, *HeartbeatReq) (*HeartbeatReply, error) // RenewOnline RenewOnline(context.Context, *OnlineReq) (*OnlineReply, error) // Receive Receive(context.Context, *ReceiveReq) (*ReceiveReply, error) //ServerList Nodes(context.Context, *NodesReq) (*NodesReply, error) } // UnimplementedLogicServer can be embedded to have forward compatible implementations. type UnimplementedLogicServer struct { } func (*UnimplementedLogicServer) Connect(ctx context.Context, req *ConnectReq) (*ConnectReply, error) { return nil, status.Errorf(codes.Unimplemented, "method Connect not implemented") } func (*UnimplementedLogicServer) Disconnect(ctx context.Context, req *DisconnectReq) (*DisconnectReply, error) { return nil, status.Errorf(codes.Unimplemented, "method Disconnect not implemented") } func (*UnimplementedLogicServer) Heartbeat(ctx context.Context, req *HeartbeatReq) (*HeartbeatReply, error) { return nil, status.Errorf(codes.Unimplemented, "method Heartbeat not implemented") } func (*UnimplementedLogicServer) RenewOnline(ctx context.Context, req *OnlineReq) (*OnlineReply, error) { return nil, status.Errorf(codes.Unimplemented, "method RenewOnline not implemented") } func (*UnimplementedLogicServer) Receive(ctx context.Context, req *ReceiveReq) (*ReceiveReply, error) { return nil, status.Errorf(codes.Unimplemented, "method Receive not implemented") } func (*UnimplementedLogicServer) Nodes(ctx context.Context, req *NodesReq) (*NodesReply, error) { return nil, status.Errorf(codes.Unimplemented, "method Nodes not implemented") } func RegisterLogicServer(s *grpc.Server, srv LogicServer) { s.RegisterService(&_Logic_serviceDesc, srv) } func _Logic_Connect_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { in := new(ConnectReq) if err := dec(in); err != nil { return nil, err } if interceptor == nil { return srv.(LogicServer).Connect(ctx, in) } info := &grpc.UnaryServerInfo{ Server: srv, FullMethod: "/goim.logic.Logic/Connect", } handler := func(ctx context.Context, req interface{}) (interface{}, error) { return srv.(LogicServer).Connect(ctx, req.(*ConnectReq)) } return interceptor(ctx, in, info, handler) } func _Logic_Disconnect_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { in := new(DisconnectReq) if err := dec(in); err != nil { return nil, err } if interceptor == nil { return srv.(LogicServer).Disconnect(ctx, in) } info := &grpc.UnaryServerInfo{ Server: srv, FullMethod: "/goim.logic.Logic/Disconnect", } handler := func(ctx context.Context, req interface{}) (interface{}, error) { return srv.(LogicServer).Disconnect(ctx, req.(*DisconnectReq)) } return interceptor(ctx, in, info, handler) } func _Logic_Heartbeat_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { in := new(HeartbeatReq) if err := dec(in); err != nil { return nil, err } if interceptor == nil { return srv.(LogicServer).Heartbeat(ctx, in) } info := &grpc.UnaryServerInfo{ Server: srv, FullMethod: "/goim.logic.Logic/Heartbeat", } handler := func(ctx context.Context, req interface{}) (interface{}, error) { return srv.(LogicServer).Heartbeat(ctx, req.(*HeartbeatReq)) } return interceptor(ctx, in, info, handler) } func _Logic_RenewOnline_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { in := new(OnlineReq) if err := dec(in); err != nil { return nil, err } if interceptor == nil { return srv.(LogicServer).RenewOnline(ctx, in) } info := &grpc.UnaryServerInfo{ Server: srv, FullMethod: "/goim.logic.Logic/RenewOnline", } handler := func(ctx context.Context, req interface{}) (interface{}, error) { return srv.(LogicServer).RenewOnline(ctx, req.(*OnlineReq)) } return interceptor(ctx, in, info, handler) } func _Logic_Receive_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { in := new(ReceiveReq) if err := dec(in); err != nil { return nil, err } if interceptor == nil { return srv.(LogicServer).Receive(ctx, in) } info := &grpc.UnaryServerInfo{ Server: srv, FullMethod: "/goim.logic.Logic/Receive", } handler := func(ctx context.Context, req interface{}) (interface{}, error) { return srv.(LogicServer).Receive(ctx, req.(*ReceiveReq)) } return interceptor(ctx, in, info, handler) } func _Logic_Nodes_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { in := new(NodesReq) if err := dec(in); err != nil { return nil, err } if interceptor == nil { return srv.(LogicServer).Nodes(ctx, in) } info := &grpc.UnaryServerInfo{ Server: srv, FullMethod: "/goim.logic.Logic/Nodes", } handler := func(ctx context.Context, req interface{}) (interface{}, error) { return srv.(LogicServer).Nodes(ctx, req.(*NodesReq)) } return interceptor(ctx, in, info, handler) } var _Logic_serviceDesc = grpc.ServiceDesc{ ServiceName: "goim.logic.Logic", HandlerType: (*LogicServer)(nil), Methods: []grpc.MethodDesc{ { MethodName: "Connect", Handler: _Logic_Connect_Handler, }, { MethodName: "Disconnect", Handler: _Logic_Disconnect_Handler, }, { MethodName: "Heartbeat", Handler: _Logic_Heartbeat_Handler, }, { MethodName: "RenewOnline", Handler: _Logic_RenewOnline_Handler, }, { MethodName: "Receive", Handler: _Logic_Receive_Handler, }, { MethodName: "Nodes", Handler: _Logic_Nodes_Handler, }, }, Streams: []grpc.StreamDesc{}, Metadata: "logic/logic.proto", } ================================================ FILE: api/logic/logic.proto ================================================ syntax = "proto3"; package goim.logic; option go_package = "github.com/Terry-Mao/goim/api/logic;logic"; import "github.com/Terry-Mao/goim/api/protocol/protocol.proto"; message PushMsg { enum Type { PUSH = 0; ROOM = 1; BROADCAST = 2; } Type type = 1; int32 operation = 2; int32 speed = 3; string server = 4; string room = 5; repeated string keys = 6; bytes msg = 7; } message ConnectReq { string server = 1; string cookie = 2; bytes token = 3; } message ConnectReply { int64 mid = 1; string key = 2; string roomID = 3; repeated int32 accepts = 4; int64 heartbeat = 5; } message DisconnectReq { int64 mid = 1; string key = 2; string server = 3; } message DisconnectReply { bool has = 1; } message HeartbeatReq { int64 mid = 1; string key = 2; string server = 3; } message HeartbeatReply { } message OnlineReq { string server = 1; map roomCount = 2; } message OnlineReply { map allRoomCount = 1; } message ReceiveReq { int64 mid = 1; goim.protocol.Proto proto = 2; } message ReceiveReply { } message NodesReq { string platform = 1; string clientIP = 2; } message NodesReply { string domain = 1; int32 tcp_port = 2; int32 ws_port = 3; int32 wss_port = 4; int32 heartbeat = 5; repeated string nodes = 6; Backoff backoff = 7; int32 heartbeat_max = 8; } message Backoff { int32 max_delay = 1; int32 base_delay = 2; float factor = 3; float jitter = 4; } service Logic { // Connect rpc Connect(ConnectReq) returns (ConnectReply); // Disconnect rpc Disconnect(DisconnectReq) returns (DisconnectReply); // Heartbeat rpc Heartbeat(HeartbeatReq) returns (HeartbeatReply); // RenewOnline rpc RenewOnline(OnlineReq) returns (OnlineReply); // Receive rpc Receive(ReceiveReq) returns (ReceiveReply); //ServerList rpc Nodes(NodesReq) returns (NodesReply); } ================================================ FILE: api/protocol/operation.go ================================================ package protocol const ( // OpHandshake handshake OpHandshake = int32(0) // OpHandshakeReply handshake reply OpHandshakeReply = int32(1) // OpHeartbeat heartbeat OpHeartbeat = int32(2) // OpHeartbeatReply heartbeat reply OpHeartbeatReply = int32(3) // OpSendMsg send message. OpSendMsg = int32(4) // OpSendMsgReply send message reply OpSendMsgReply = int32(5) // OpDisconnectReply disconnect reply OpDisconnectReply = int32(6) // OpAuth auth connnect OpAuth = int32(7) // OpAuthReply auth connect reply OpAuthReply = int32(8) // OpRaw raw message OpRaw = int32(9) // OpProtoReady proto ready OpProtoReady = int32(10) // OpProtoFinish proto finish OpProtoFinish = int32(11) // OpChangeRoom change room OpChangeRoom = int32(12) // OpChangeRoomReply change room reply OpChangeRoomReply = int32(13) // OpSub subscribe operation OpSub = int32(14) // OpSubReply subscribe operation OpSubReply = int32(15) // OpUnsub unsubscribe operation OpUnsub = int32(16) // OpUnsubReply unsubscribe operation reply OpUnsubReply = int32(17) ) ================================================ FILE: api/protocol/protocol.go ================================================ package protocol import ( "errors" "github.com/Terry-Mao/goim/pkg/bufio" "github.com/Terry-Mao/goim/pkg/bytes" "github.com/Terry-Mao/goim/pkg/encoding/binary" "github.com/Terry-Mao/goim/pkg/websocket" ) const ( // MaxBodySize max proto body size MaxBodySize = int32(1 << 12) ) const ( // size _packSize = 4 _headerSize = 2 _verSize = 2 _opSize = 4 _seqSize = 4 _heartSize = 4 _rawHeaderSize = _packSize + _headerSize + _verSize + _opSize + _seqSize _maxPackSize = MaxBodySize + int32(_rawHeaderSize) // offset _packOffset = 0 _headerOffset = _packOffset + _packSize _verOffset = _headerOffset + _headerSize _opOffset = _verOffset + _verSize _seqOffset = _opOffset + _opSize _heartOffset = _seqOffset + _seqSize ) var ( // ErrProtoPackLen proto packet len error ErrProtoPackLen = errors.New("default server codec pack length error") // ErrProtoHeaderLen proto header len error ErrProtoHeaderLen = errors.New("default server codec header length error") ) var ( // ProtoReady proto ready ProtoReady = &Proto{Op: OpProtoReady} // ProtoFinish proto finish ProtoFinish = &Proto{Op: OpProtoFinish} ) // WriteTo write a proto to bytes writer. func (p *Proto) WriteTo(b *bytes.Writer) { var ( packLen = _rawHeaderSize + int32(len(p.Body)) buf = b.Peek(_rawHeaderSize) ) binary.BigEndian.PutInt32(buf[_packOffset:], packLen) binary.BigEndian.PutInt16(buf[_headerOffset:], int16(_rawHeaderSize)) binary.BigEndian.PutInt16(buf[_verOffset:], int16(p.Ver)) binary.BigEndian.PutInt32(buf[_opOffset:], p.Op) binary.BigEndian.PutInt32(buf[_seqOffset:], p.Seq) if p.Body != nil { b.Write(p.Body) } } // ReadTCP read a proto from TCP reader. func (p *Proto) ReadTCP(rr *bufio.Reader) (err error) { var ( bodyLen int headerLen int16 packLen int32 buf []byte ) if buf, err = rr.Pop(_rawHeaderSize); err != nil { return } packLen = binary.BigEndian.Int32(buf[_packOffset:_headerOffset]) headerLen = binary.BigEndian.Int16(buf[_headerOffset:_verOffset]) p.Ver = int32(binary.BigEndian.Int16(buf[_verOffset:_opOffset])) p.Op = binary.BigEndian.Int32(buf[_opOffset:_seqOffset]) p.Seq = binary.BigEndian.Int32(buf[_seqOffset:]) if packLen > _maxPackSize { return ErrProtoPackLen } if headerLen != _rawHeaderSize { return ErrProtoHeaderLen } if bodyLen = int(packLen - int32(headerLen)); bodyLen > 0 { p.Body, err = rr.Pop(bodyLen) } else { p.Body = nil } return } // WriteTCP write a proto to TCP writer. func (p *Proto) WriteTCP(wr *bufio.Writer) (err error) { var ( buf []byte packLen int32 ) if p.Op == OpRaw { // write without buffer, job concact proto into raw buffer _, err = wr.WriteRaw(p.Body) return } packLen = _rawHeaderSize + int32(len(p.Body)) if buf, err = wr.Peek(_rawHeaderSize); err != nil { return } binary.BigEndian.PutInt32(buf[_packOffset:], packLen) binary.BigEndian.PutInt16(buf[_headerOffset:], int16(_rawHeaderSize)) binary.BigEndian.PutInt16(buf[_verOffset:], int16(p.Ver)) binary.BigEndian.PutInt32(buf[_opOffset:], p.Op) binary.BigEndian.PutInt32(buf[_seqOffset:], p.Seq) if p.Body != nil { _, err = wr.Write(p.Body) } return } // WriteTCPHeart write TCP heartbeat with room online. func (p *Proto) WriteTCPHeart(wr *bufio.Writer, online int32) (err error) { var ( buf []byte packLen int ) packLen = _rawHeaderSize + _heartSize if buf, err = wr.Peek(packLen); err != nil { return } // header binary.BigEndian.PutInt32(buf[_packOffset:], int32(packLen)) binary.BigEndian.PutInt16(buf[_headerOffset:], int16(_rawHeaderSize)) binary.BigEndian.PutInt16(buf[_verOffset:], int16(p.Ver)) binary.BigEndian.PutInt32(buf[_opOffset:], p.Op) binary.BigEndian.PutInt32(buf[_seqOffset:], p.Seq) // body binary.BigEndian.PutInt32(buf[_heartOffset:], online) return } // ReadWebsocket read a proto from websocket connection. func (p *Proto) ReadWebsocket(ws *websocket.Conn) (err error) { var ( bodyLen int headerLen int16 packLen int32 buf []byte ) if _, buf, err = ws.ReadMessage(); err != nil { return } if len(buf) < _rawHeaderSize { return ErrProtoPackLen } packLen = binary.BigEndian.Int32(buf[_packOffset:_headerOffset]) headerLen = binary.BigEndian.Int16(buf[_headerOffset:_verOffset]) p.Ver = int32(binary.BigEndian.Int16(buf[_verOffset:_opOffset])) p.Op = binary.BigEndian.Int32(buf[_opOffset:_seqOffset]) p.Seq = binary.BigEndian.Int32(buf[_seqOffset:]) if packLen < 0 || packLen > _maxPackSize { return ErrProtoPackLen } if headerLen != _rawHeaderSize { return ErrProtoHeaderLen } if bodyLen = int(packLen - int32(headerLen)); bodyLen > 0 { p.Body = buf[headerLen:packLen] } else { p.Body = nil } return } // WriteWebsocket write a proto to websocket connection. func (p *Proto) WriteWebsocket(ws *websocket.Conn) (err error) { var ( buf []byte packLen int ) packLen = _rawHeaderSize + len(p.Body) if err = ws.WriteHeader(websocket.BinaryMessage, packLen); err != nil { return } if buf, err = ws.Peek(_rawHeaderSize); err != nil { return } binary.BigEndian.PutInt32(buf[_packOffset:], int32(packLen)) binary.BigEndian.PutInt16(buf[_headerOffset:], int16(_rawHeaderSize)) binary.BigEndian.PutInt16(buf[_verOffset:], int16(p.Ver)) binary.BigEndian.PutInt32(buf[_opOffset:], p.Op) binary.BigEndian.PutInt32(buf[_seqOffset:], p.Seq) if p.Body != nil { err = ws.WriteBody(p.Body) } return } // WriteWebsocketHeart write websocket heartbeat with room online. func (p *Proto) WriteWebsocketHeart(wr *websocket.Conn, online int32) (err error) { var ( buf []byte packLen int ) packLen = _rawHeaderSize + _heartSize // websocket header if err = wr.WriteHeader(websocket.BinaryMessage, packLen); err != nil { return } if buf, err = wr.Peek(packLen); err != nil { return } // proto header binary.BigEndian.PutInt32(buf[_packOffset:], int32(packLen)) binary.BigEndian.PutInt16(buf[_headerOffset:], int16(_rawHeaderSize)) binary.BigEndian.PutInt16(buf[_verOffset:], int16(p.Ver)) binary.BigEndian.PutInt32(buf[_opOffset:], p.Op) binary.BigEndian.PutInt32(buf[_seqOffset:], p.Seq) // proto body binary.BigEndian.PutInt32(buf[_heartOffset:], online) return } ================================================ FILE: api/protocol/protocol.pb.go ================================================ // Code generated by protoc-gen-go. DO NOT EDIT. // source: protocol/protocol.proto package protocol import ( fmt "fmt" proto "github.com/golang/protobuf/proto" math "math" ) // Reference imports to suppress errors if they are not otherwise used. var _ = proto.Marshal var _ = fmt.Errorf var _ = math.Inf // This is a compile-time assertion to ensure that this generated file // is compatible with the proto package it is being compiled against. // A compilation error at this line likely means your copy of the // proto package needs to be updated. const _ = proto.ProtoPackageIsVersion3 // please upgrade the proto package // // v1.0.0 // protocol type Proto struct { Ver int32 `protobuf:"varint,1,opt,name=ver,proto3" json:"ver,omitempty"` Op int32 `protobuf:"varint,2,opt,name=op,proto3" json:"op,omitempty"` Seq int32 `protobuf:"varint,3,opt,name=seq,proto3" json:"seq,omitempty"` Body []byte `protobuf:"bytes,4,opt,name=body,proto3" json:"body,omitempty"` XXX_NoUnkeyedLiteral struct{} `json:"-"` XXX_unrecognized []byte `json:"-"` XXX_sizecache int32 `json:"-"` } func (m *Proto) Reset() { *m = Proto{} } func (m *Proto) String() string { return proto.CompactTextString(m) } func (*Proto) ProtoMessage() {} func (*Proto) Descriptor() ([]byte, []int) { return fileDescriptor_87968d26f3046c60, []int{0} } func (m *Proto) XXX_Unmarshal(b []byte) error { return xxx_messageInfo_Proto.Unmarshal(m, b) } func (m *Proto) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { return xxx_messageInfo_Proto.Marshal(b, m, deterministic) } func (m *Proto) XXX_Merge(src proto.Message) { xxx_messageInfo_Proto.Merge(m, src) } func (m *Proto) XXX_Size() int { return xxx_messageInfo_Proto.Size(m) } func (m *Proto) XXX_DiscardUnknown() { xxx_messageInfo_Proto.DiscardUnknown(m) } var xxx_messageInfo_Proto proto.InternalMessageInfo func (m *Proto) GetVer() int32 { if m != nil { return m.Ver } return 0 } func (m *Proto) GetOp() int32 { if m != nil { return m.Op } return 0 } func (m *Proto) GetSeq() int32 { if m != nil { return m.Seq } return 0 } func (m *Proto) GetBody() []byte { if m != nil { return m.Body } return nil } func init() { proto.RegisterType((*Proto)(nil), "goim.protocol.Proto") } func init() { proto.RegisterFile("protocol/protocol.proto", fileDescriptor_87968d26f3046c60) } var fileDescriptor_87968d26f3046c60 = []byte{ // 153 bytes of a gzipped FileDescriptorProto 0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xe2, 0x12, 0x2f, 0x28, 0xca, 0x2f, 0xc9, 0x4f, 0xce, 0xcf, 0xd1, 0x87, 0x31, 0xf4, 0xc0, 0x0c, 0x21, 0xde, 0xf4, 0xfc, 0xcc, 0x5c, 0x3d, 0x98, 0xa0, 0x92, 0x3f, 0x17, 0x6b, 0x00, 0x58, 0x5c, 0x80, 0x8b, 0xb9, 0x2c, 0xb5, 0x48, 0x82, 0x51, 0x81, 0x51, 0x83, 0x35, 0x08, 0xc4, 0x14, 0xe2, 0xe3, 0x62, 0xca, 0x2f, 0x90, 0x60, 0x02, 0x0b, 0x30, 0xe5, 0x17, 0x80, 0x54, 0x14, 0xa7, 0x16, 0x4a, 0x30, 0x43, 0x54, 0x14, 0xa7, 0x16, 0x0a, 0x09, 0x71, 0xb1, 0x24, 0xe5, 0xa7, 0x54, 0x4a, 0xb0, 0x28, 0x30, 0x6a, 0xf0, 0x04, 0x81, 0xd9, 0x4e, 0x86, 0x51, 0xfa, 0xe9, 0x99, 0x25, 0x19, 0xa5, 0x49, 0x7a, 0xc9, 0xf9, 0xb9, 0xfa, 0x21, 0xa9, 0x45, 0x45, 0x95, 0xba, 0xbe, 0x89, 0xf9, 0xfa, 0x20, 0x6b, 0xf5, 0x13, 0x0b, 0x32, 0xe1, 0xee, 0xb1, 0x86, 0x31, 0x92, 0xd8, 0xc0, 0x2c, 0x63, 0x40, 0x00, 0x00, 0x00, 0xff, 0xff, 0x8d, 0xc7, 0xbf, 0x64, 0xb4, 0x00, 0x00, 0x00, } ================================================ FILE: api/protocol/protocol.proto ================================================ syntax = "proto3"; package goim.protocol; option go_package = "github.com/Terry-Mao/goim/api/protocol;protocol"; /* * v1.0.0 * protocol */ message Proto { int32 ver = 1; int32 op = 2; int32 seq = 3; bytes body = 4; } ================================================ FILE: benchmarks/client/main.go ================================================ package main // Start Commond eg: ./client 1 1000 localhost:3101 // first parameter：beginning userId // second parameter: amount of clients // third parameter: comet server ip import ( "bufio" "encoding/binary" "encoding/json" "flag" "fmt" "math/rand" "net" "os" "runtime" "strconv" "sync/atomic" "time" log "github.com/golang/glog" ) const ( opHeartbeat = int32(2) opHeartbeatReply = int32(3) opAuth = int32(7) opAuthReply = int32(8) ) const ( rawHeaderLen = uint16(16) heart = 240 * time.Second ) // Proto proto. type Proto struct { PackLen int32 // package length HeaderLen int16 // header length Ver int16 // protocol version Operation int32 // operation for request Seq int32 // sequence number chosen by client Body []byte // body } // AuthToken auth token. type AuthToken struct { Mid int64 `json:"mid"` Key string `json:"key"` RoomID string `json:"room_id"` Platform string `json:"platform"` Accepts []int32 `json:"accepts"` } var ( countDown int64 aliveCount int64 ) func main() { runtime.GOMAXPROCS(runtime.NumCPU()) flag.Parse() begin, err := strconv.Atoi(os.Args[1]) if err != nil { panic(err) } num, err := strconv.Atoi(os.Args[2]) if err != nil { panic(err) } go result() for i := begin; i < begin+num; i++ { go client(int64(i)) } // signal var exit chan bool <-exit } func result() { var ( lastTimes int64 interval = int64(5) ) for { nowCount := atomic.LoadInt64(&countDown) nowAlive := atomic.LoadInt64(&aliveCount) diff := nowCount - lastTimes lastTimes = nowCount fmt.Println(fmt.Sprintf("%s alive:%d down:%d down/s:%d", time.Now().Format("2006-01-02 15:04:05"), nowAlive, nowCount, diff/interval)) time.Sleep(time.Second * time.Duration(interval)) } } func client(mid int64) { for { startClient(mid) time.Sleep(time.Duration(rand.Intn(10)) * time.Second) } } func startClient(key int64) { time.Sleep(time.Duration(rand.Intn(120)) * time.Second) atomic.AddInt64(&aliveCount, 1) quit := make(chan bool, 1) defer func() { close(quit) atomic.AddInt64(&aliveCount, -1) }() // connnect to server conn, err := net.Dial("tcp", os.Args[3]) if err != nil { log.Errorf("net.Dial(%s) error(%v)", os.Args[3], err) return } seq := int32(0) wr := bufio.NewWriter(conn) rd := bufio.NewReader(conn) authToken := &AuthToken{ key, "", "test://1", "ios", []int32{1000, 1001, 1002}, } proto := new(Proto) proto.Ver = 1 proto.Operation = opAuth proto.Seq = seq proto.Body, _ = json.Marshal(authToken) if err = tcpWriteProto(wr, proto); err != nil { log.Errorf("tcpWriteProto() error(%v)", err) return } if err = tcpReadProto(rd, proto); err != nil { log.Errorf("tcpReadProto() error(%v)", err) return } log.Infof("key:%d auth ok, proto: %v", key, proto) seq++ // writer go func() { hbProto := new(Proto) for { // heartbeat hbProto.Operation = opHeartbeat hbProto.Seq = seq hbProto.Body = nil if err = tcpWriteProto(wr, hbProto); err != nil { log.Errorf("key:%d tcpWriteProto() error(%v)", key, err) return } log.Infof("key:%d Write heartbeat", key) time.Sleep(heart) seq++ select { case <-quit: return default: } } }() // reader for { if err = tcpReadProto(rd, proto); err != nil { log.Errorf("key:%d tcpReadProto() error(%v)", key, err) quit <- true return } if proto.Operation == opAuthReply { log.Infof("key:%d auth success", key) } else if proto.Operation == opHeartbeatReply { log.Infof("key:%d receive heartbeat", key) if err = conn.SetReadDeadline(time.Now().Add(heart + 60*time.Second)); err != nil { log.Errorf("conn.SetReadDeadline() error(%v)", err) quit <- true return } } else { log.Infof("key:%d op:%d msg: %s", key, proto.Operation, string(proto.Body)) atomic.AddInt64(&countDown, 1) } } } func tcpWriteProto(wr *bufio.Writer, proto *Proto) (err error) { // write if err = binary.Write(wr, binary.BigEndian, uint32(rawHeaderLen)+uint32(len(proto.Body))); err != nil { return } if err = binary.Write(wr, binary.BigEndian, rawHeaderLen); err != nil { return } if err = binary.Write(wr, binary.BigEndian, proto.Ver); err != nil { return } if err = binary.Write(wr, binary.BigEndian, proto.Operation); err != nil { return } if err = binary.Write(wr, binary.BigEndian, proto.Seq); err != nil { return } if proto.Body != nil { if err = binary.Write(wr, binary.BigEndian, proto.Body); err != nil { return } } err = wr.Flush() return } func tcpReadProto(rd *bufio.Reader, proto *Proto) (err error) { var ( packLen int32 headerLen int16 ) // read if err = binary.Read(rd, binary.BigEndian, &packLen); err != nil { return } if err = binary.Read(rd, binary.BigEndian, &headerLen); err != nil { return } if err = binary.Read(rd, binary.BigEndian, &proto.Ver); err != nil { return } if err = binary.Read(rd, binary.BigEndian, &proto.Operation); err != nil { return } if err = binary.Read(rd, binary.BigEndian, &proto.Seq); err != nil { return } var ( n, t int bodyLen = int(packLen - int32(headerLen)) ) if bodyLen > 0 { proto.Body = make([]byte, bodyLen) for { if t, err = rd.Read(proto.Body[n:]); err != nil { return } if n += t; n == bodyLen { break } } } else { proto.Body = nil } return } ================================================ FILE: benchmarks/multi_push/main.go ================================================ package main // Start Command eg : ./multi_push 0 20000 localhost:7172 60 import ( "bytes" "encoding/json" "fmt" "io" "io/ioutil" "log" "net" "net/http" "os" "runtime" "strconv" "time" ) var ( lg *log.Logger httpClient *http.Client t int ) const testContent = "{\"test\":1}" type pushsBodyMsg struct { Msg json.RawMessage `json:"m"` UserIds []int64 `json:"u"` } func init() { httpTransport := &http.Transport{ Dial: func(netw, addr string) (net.Conn, error) { deadline := time.Now().Add(30 * time.Second) c, err := net.DialTimeout(netw, addr, 20*time.Second) if err != nil { return nil, err } _ = c.SetDeadline(deadline) return c, nil }, DisableKeepAlives: false, } httpClient = &http.Client{ Transport: httpTransport, } } func main() { runtime.GOMAXPROCS(runtime.NumCPU()) infoLogfi, err := os.OpenFile("./multi_push.log", os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0644) if err != nil { panic(err) } lg = log.New(infoLogfi, "", log.LstdFlags|log.Lshortfile) begin, err := strconv.Atoi(os.Args[1]) if err != nil { panic(err) } length, err := strconv.Atoi(os.Args[2]) if err != nil { panic(err) } t, err = strconv.Atoi(os.Args[4]) if err != nil { panic(err) } num := runtime.NumCPU() * 8 l := length / num b, e := begin, begin+l time.AfterFunc(time.Duration(t)*time.Second, stop) for i := 0; i < num; i++ { go startPush(b, e) b += l e += l } if b < begin+length { go startPush(b, begin+length) } time.Sleep(9999 * time.Hour) } func stop() { os.Exit(-1) } func startPush(b, e int) { l := make([]int64, 0, e-b) for i := b; i < e; i++ { l = append(l, int64(i)) } msg := &pushsBodyMsg{Msg: json.RawMessage(testContent), UserIds: l} body, err := json.Marshal(msg) if err != nil { panic(err) } for { resp, err := httpPost(fmt.Sprintf("http://%s/goim/push/mids=%d", os.Args[3], b), "application/x-www-form-urlencoded", bytes.NewBuffer(body)) if err != nil { lg.Printf("post error (%v)", err) continue } body, err := ioutil.ReadAll(resp.Body) if err != nil { lg.Printf("post error (%v)", err) return } resp.Body.Close() lg.Printf("response %s", string(body)) } } func httpPost(url string, contentType string, body io.Reader) (*http.Response, error) { req, err := http.NewRequest("POST", url, body) if err != nil { return nil, err } req.Header.Set("Content-Type", contentType) resp, err := httpClient.Do(req) if err != nil { return nil, err } return resp, nil } ================================================ FILE: benchmarks/push/main.go ================================================ package main // Start Command eg : ./push 0 20000 localhost:7172 60 import ( "bytes" "encoding/json" "fmt" "io" "io/ioutil" "log" "net" "net/http" "os" "runtime" "strconv" "time" ) var ( httpClient *http.Client t int ) const testContent = "{\"test\":1}" type pushBodyMsg struct { Msg json.RawMessage `json:"m"` UserID int64 `json:"u"` } func init() { httpTransport := &http.Transport{ Dial: func(netw, addr string) (net.Conn, error) { deadline := time.Now().Add(30 * time.Second) c, err := net.DialTimeout(netw, addr, 20*time.Second) if err != nil { return nil, err } _ = c.SetDeadline(deadline) return c, nil }, DisableKeepAlives: false, } httpClient = &http.Client{ Transport: httpTransport, } } func main() { runtime.GOMAXPROCS(runtime.NumCPU()) begin, err := strconv.Atoi(os.Args[1]) if err != nil { panic(err) } length, err := strconv.Atoi(os.Args[2]) if err != nil { panic(err) } t, err = strconv.Atoi(os.Args[4]) if err != nil { panic(err) } num := runtime.NumCPU() * 2 log.Printf("start routine num:%d", num) l := length / num b, e := begin, begin+l time.AfterFunc(time.Duration(t)*time.Second, stop) for i := 0; i < num; i++ { go startPush(b, e) b += l e += l } if b < begin+length { go startPush(b, begin+length) } time.Sleep(9999 * time.Hour) } func stop() { os.Exit(-1) } func startPush(b, e int) { log.Printf("start Push from %d to %d", b, e) bodys := make([][]byte, e-b) for i := 0; i < e-b; i++ { msg := &pushBodyMsg{Msg: json.RawMessage(testContent), UserID: int64(b)} body, err := json.Marshal(msg) if err != nil { panic(err) } bodys[i] = body } for { for i := 0; i < len(bodys); i++ { resp, err := httpPost(fmt.Sprintf("http://%s/goim/push/mids?operation=1000&mids=%d", os.Args[3], b), "application/x-www-form-urlencoded", bytes.NewBuffer(bodys[i])) if err != nil { log.Printf("post error (%v)", err) continue } body, err := ioutil.ReadAll(resp.Body) if err != nil { log.Printf("post error (%v)", err) return } resp.Body.Close() log.Printf("response %s", string(body)) //time.Sleep(50 * time.Millisecond) } } } func httpPost(url string, contentType string, body io.Reader) (*http.Response, error) { req, err := http.NewRequest("POST", url, body) if err != nil { return nil, err } req.Header.Set("Content-Type", contentType) resp, err := httpClient.Do(req) if err != nil { return nil, err } return resp, nil } ================================================ FILE: benchmarks/push_room/main.go ================================================ package main // Start Commond eg: ./push_room 1 20 localhost:3111 // first parameter: room id // second parameter: num per seconds // third parameter: logic server ip import ( "bytes" "fmt" "io/ioutil" "net/http" "os" "strconv" "time" ) func main() { rountineNum, err := strconv.Atoi(os.Args[2]) if err != nil { panic(err) } addr := os.Args[3] gap := time.Second / time.Duration(rountineNum) delay := time.Duration(0) go run(addr, time.Duration(0)*time.Second) for i := 0; i < rountineNum-1; i++ { go run(addr, delay) delay += gap fmt.Println("delay:", delay) } time.Sleep(9999 * time.Hour) } func run(addr string, delay time.Duration) { time.Sleep(delay) i := int64(0) for { go post(addr, i) time.Sleep(time.Second) i++ } } func post(addr string, i int64) { resp, err := http.Post("http://"+addr+"/goim/push/room?operation=1000&type=test&room="+os.Args[1], "application/json", bytes.NewBufferString(fmt.Sprintf("{\"test\":%d}", i))) if err != nil { fmt.Printf("Error: http.post() error(%v)\n", err) return } defer resp.Body.Close() body, err := ioutil.ReadAll(resp.Body) if err != nil { fmt.Printf("Error: http.post() error(%v)\n", err) return } fmt.Printf("%s postId:%d, response:%s\n", time.Now().Format("2006-01-02 15:04:05"), i, string(body)) } ================================================ FILE: benchmarks/push_rooms/main.go ================================================ package main // Start Command eg : ./push_rooms 0 20000 localhost:7172 40 // param 1 : the start of room number // param 2 : the end of room number // param 3 : comet server tcp address // param 4 : push amount each goroutines per second import ( "bytes" "fmt" "io/ioutil" "log" "net" "net/http" "os" "runtime" "strconv" "time" ) var ( httpClient *http.Client ) const testContent = "{\"test\":1}" func init() { httpTransport := &http.Transport{ Dial: func(netw, addr string) (net.Conn, error) { deadline := time.Now().Add(30 * time.Second) c, err := net.DialTimeout(netw, addr, 20*time.Second) if err != nil { return nil, err } _ = c.SetDeadline(deadline) return c, nil }, DisableKeepAlives: false, } httpClient = &http.Client{ Transport: httpTransport, } } func main() { runtime.GOMAXPROCS(runtime.NumCPU()) begin, err := strconv.Atoi(os.Args[1]) if err != nil { panic(err) } length, err := strconv.Atoi(os.Args[2]) if err != nil { panic(err) } num, err := strconv.Atoi(os.Args[4]) if err != nil { panic(err) } delay := (1000 * time.Millisecond) / time.Duration(num) routines := runtime.NumCPU() * 2 log.Printf("start routine num:%d", routines) l := length / routines b, e := begin, begin+l for i := 0; i < routines; i++ { go startPush(b, e, delay) b += l e += l } if b < begin+length { go startPush(b, begin+length, delay) } time.Sleep(9999 * time.Hour) } func startPush(b, e int, delay time.Duration) { log.Printf("start Push from %d to %d", b, e) for { for i := b; i < e; i++ { resp, err := http.Post(fmt.Sprintf("http://%s/goim/push/room?operation=1000&type=test&room=%d", os.Args[3], i), "application/json", bytes.NewBufferString(testContent)) if err != nil { log.Printf("post error (%v)", err) continue } body, err := ioutil.ReadAll(resp.Body) if err != nil { log.Printf("post error (%v)", err) return } resp.Body.Close() log.Printf("push room:%d response %s", i, string(body)) time.Sleep(delay) } } } ================================================ FILE: cmd/comet/comet-example.toml ================================================ # This is a TOML document. Boom [discovery] nodes = ["127.0.0.1:7171"] [rpcServer] addr = ":3109" timeout = "1s" [rpcClient] dial = "1s" timeout = "1s" [tcp] bind = [":3101"] sndbuf = 4096 rcvbuf = 4096 keepalive = false reader = 32 readBuf = 1024 readBufSize = 8192 writer = 32 writeBuf = 1024 writeBufSize = 8192 [websocket] bind = [":3102"] tlsOpen = false tlsBind = [":3103"] certFile = "../../cert.pem" privateFile = "../../private.pem" [protocol] timer = 32 timerSize = 2048 svrProto = 10 cliProto = 5 handshakeTimeout = "8s" [whitelist] Whitelist = [123] WhiteLog = "/tmp/white_list.log" [bucket] size = 32 channel = 1024 room = 1024 routineAmount = 32 routineSize = 1024 ================================================ FILE: cmd/comet/main.go ================================================ package main import ( "context" "flag" "fmt" "math/rand" "net" "os" "os/signal" "runtime" "strconv" "strings" "syscall" "time" "github.com/bilibili/discovery/naming" resolver "github.com/bilibili/discovery/naming/grpc" "github.com/Terry-Mao/goim/internal/comet" "github.com/Terry-Mao/goim/internal/comet/conf" "github.com/Terry-Mao/goim/internal/comet/grpc" md "github.com/Terry-Mao/goim/internal/logic/model" "github.com/Terry-Mao/goim/pkg/ip" log "github.com/golang/glog" ) const ( ver = "2.0.0" appid = "goim.comet" ) func main() { flag.Parse() if err := conf.Init(); err != nil { panic(err) } rand.Seed(time.Now().UTC().UnixNano()) runtime.GOMAXPROCS(runtime.NumCPU()) println(conf.Conf.Debug) log.Infof("goim-comet [version: %s env: %+v] start", ver, conf.Conf.Env) // register discovery dis := naming.New(conf.Conf.Discovery) resolver.Register(dis) // new comet server srv := comet.NewServer(conf.Conf) if err := comet.InitWhitelist(conf.Conf.Whitelist); err != nil { panic(err) } if err := comet.InitTCP(srv, conf.Conf.TCP.Bind, runtime.NumCPU()); err != nil { panic(err) } if err := comet.InitWebsocket(srv, conf.Conf.Websocket.Bind, runtime.NumCPU()); err != nil { panic(err) } if conf.Conf.Websocket.TLSOpen { if err := comet.InitWebsocketWithTLS(srv, conf.Conf.Websocket.TLSBind, conf.Conf.Websocket.CertFile, conf.Conf.Websocket.PrivateFile, runtime.NumCPU()); err != nil { panic(err) } } // new grpc server rpcSrv := grpc.New(conf.Conf.RPCServer, srv) cancel := register(dis, srv) // signal c := make(chan os.Signal, 1) signal.Notify(c, syscall.SIGHUP, syscall.SIGQUIT, syscall.SIGTERM, syscall.SIGINT) for { s := <-c log.Infof("goim-comet get a signal %s", s.String()) switch s { case syscall.SIGQUIT, syscall.SIGTERM, syscall.SIGINT: if cancel != nil { cancel() } rpcSrv.GracefulStop() srv.Close() log.Infof("goim-comet [version: %s] exit", ver) log.Flush() return case syscall.SIGHUP: default: return } } } func register(dis *naming.Discovery, srv *comet.Server) context.CancelFunc { env := conf.Conf.Env addr := ip.InternalIP() _, port, _ := net.SplitHostPort(conf.Conf.RPCServer.Addr) ins := &naming.Instance{ Region: env.Region, Zone: env.Zone, Env: env.DeployEnv, Hostname: env.Host, AppID: appid, Addrs: []string{ "grpc://" + addr + ":" + port, }, Metadata: map[string]string{ md.MetaWeight: strconv.FormatInt(env.Weight, 10), md.MetaOffline: strconv.FormatBool(env.Offline), md.MetaAddrs: strings.Join(env.Addrs, ","), }, } cancel, err := dis.Register(ins) if err != nil { panic(err) } // renew discovery metadata go func() { for { var ( err error conns int ips = make(map[string]struct{}) ) for _, bucket := range srv.Buckets() { for ip := range bucket.IPCount() { ips[ip] = struct{}{} } conns += bucket.ChannelCount() } ins.Metadata[md.MetaConnCount] = fmt.Sprint(conns) ins.Metadata[md.MetaIPCount] = fmt.Sprint(len(ips)) if err = dis.Set(ins); err != nil { log.Errorf("dis.Set(%+v) error(%v)", ins, err) time.Sleep(time.Second) continue } time.Sleep(time.Second * 10) } }() return cancel } ================================================ FILE: cmd/job/job-example.toml ================================================ # This is a TOML document. Boom [discovery] nodes = ["127.0.0.1:7171"] [kafka] topic = "goim-push-topic" group = "goim-push-group-job" brokers = ["127.0.0.1:9092"] ================================================ FILE: cmd/job/main.go ================================================ package main import ( "flag" "os" "os/signal" "syscall" "github.com/bilibili/discovery/naming" "github.com/Terry-Mao/goim/internal/job" "github.com/Terry-Mao/goim/internal/job/conf" resolver "github.com/bilibili/discovery/naming/grpc" log "github.com/golang/glog" ) var ( ver = "2.0.0" ) func main() { flag.Parse() if err := conf.Init(); err != nil { panic(err) } log.Infof("goim-job [version: %s env: %+v] start", ver, conf.Conf.Env) // grpc register naming dis := naming.New(conf.Conf.Discovery) resolver.Register(dis) // job j := job.New(conf.Conf) go j.Consume() // signal c := make(chan os.Signal, 1) signal.Notify(c, syscall.SIGHUP, syscall.SIGQUIT, syscall.SIGTERM, syscall.SIGINT) for { s := <-c log.Infof("goim-job get a signal %s", s.String()) switch s { case syscall.SIGQUIT, syscall.SIGTERM, syscall.SIGINT: j.Close() log.Infof("goim-job [version: %s] exit", ver) log.Flush() return case syscall.SIGHUP: default: return } } } ================================================ FILE: cmd/logic/logic-example.toml ================================================ # This is a TOML document. Boom [discovery] nodes = ["127.0.0.1:7171"] [regions] "bj" = ["北京","天津","河北","山东","山西","内蒙古","辽宁","吉林","黑龙江","甘肃","宁夏","新疆"] "sh" = ["上海","江苏","浙江","安徽","江西","湖北","重庆","陕西","青海","河南","台湾"] "gz" = ["广东","福建","广西","海南","湖南","四川","贵州","云南","西藏","香港","澳门"] [node] defaultDomain = "conn.goim.io" hostDomain = ".goim.io" heartbeat = "4m" heartbeatMax = 2 tcpPort = 3101 wsPort = 3102 wssPort = 3103 regionWeight = 1.6 [backoff] maxDelay = 300 baseDelay = 3 factor = 1.8 jitter = 0.3 [rpcServer] network = "tcp" addr = ":3119" timeout = "1s" [rpcClient] dial = "1s" timeout = "1s" [httpServer] network = "tcp" addr = ":3111" readTimeout = "1s" writeTimeout = "1s" [kafka] topic = "goim-push-topic" brokers = ["127.0.0.1:9092"] [redis] network = "tcp" addr = "127.0.0.1:6379" active = 60000 idle = 1024 dialTimeout = "200ms" readTimeout = "500ms" writeTimeout = "500ms" idleTimeout = "120s" expire = "30m" ================================================ FILE: cmd/logic/main.go ================================================ package main import ( "context" "flag" "net" "os" "os/signal" "strconv" "syscall" "github.com/bilibili/discovery/naming" resolver "github.com/bilibili/discovery/naming/grpc" "github.com/Terry-Mao/goim/internal/logic" "github.com/Terry-Mao/goim/internal/logic/conf" "github.com/Terry-Mao/goim/internal/logic/grpc" "github.com/Terry-Mao/goim/internal/logic/http" "github.com/Terry-Mao/goim/internal/logic/model" "github.com/Terry-Mao/goim/pkg/ip" log "github.com/golang/glog" ) const ( ver = "2.0.0" appid = "goim.logic" ) func main() { flag.Parse() if err := conf.Init(); err != nil { panic(err) } log.Infof("goim-logic [version: %s env: %+v] start", ver, conf.Conf.Env) // grpc register naming dis := naming.New(conf.Conf.Discovery) resolver.Register(dis) // logic srv := logic.New(conf.Conf) httpSrv := http.New(conf.Conf.HTTPServer, srv) rpcSrv := grpc.New(conf.Conf.RPCServer, srv) cancel := register(dis, srv) // signal c := make(chan os.Signal, 1) signal.Notify(c, syscall.SIGHUP, syscall.SIGQUIT, syscall.SIGTERM, syscall.SIGINT) for { s := <-c log.Infof("goim-logic get a signal %s", s.String()) switch s { case syscall.SIGQUIT, syscall.SIGTERM, syscall.SIGINT: if cancel != nil { cancel() } srv.Close() httpSrv.Close() rpcSrv.GracefulStop() log.Infof("goim-logic [version: %s] exit", ver) log.Flush() return case syscall.SIGHUP: default: return } } } func register(dis *naming.Discovery, srv *logic.Logic) context.CancelFunc { env := conf.Conf.Env addr := ip.InternalIP() _, port, _ := net.SplitHostPort(conf.Conf.RPCServer.Addr) ins := &naming.Instance{ Region: env.Region, Zone: env.Zone, Env: env.DeployEnv, Hostname: env.Host, AppID: appid, Addrs: []string{ "grpc://" + addr + ":" + port, }, Metadata: map[string]string{ model.MetaWeight: strconv.FormatInt(env.Weight, 10), }, } cancel, err := dis.Register(ins) if err != nil { panic(err) } return cancel } ================================================ FILE: codecov.sh ================================================ #!/usr/bin/env bash set -e echo "" > coverage.txt for d in $(go list ./... | grep -v cmd | grep -v docs | grep -v srcipts | grep -v benchmarks | grep -v examples); do echo "testing for $d ..." go test -coverprofile=profile.out -covermode=atomic $d if [ -f profile.out ]; then cat profile.out >> coverage.txt rm profile.out fi done ================================================ FILE: docs/benchmark_cn.md ================================================ ## 压测图表 ![benchmark](benchmark.jpg) ### 服务端配置 | CPU | 内存 | 操作系统 | 数量 | | :---- | :---- | :---- | :---- | | Intel(R) Xeon(R) CPU E5-2630 v2 @ 2.60GHz | DDR3 32GB | Debian GNU/Linux 8 | 1 | ### 压测参数 * 不同UID同房间在线人数: 1,000,000 * 持续推送时长: 15分钟 * 持续推送数量: 40条/秒 * 推送内容: {"test":1} * 推送类型: 单房间推送 * 到达计算方式: 1秒统计一次,共30次 ### 资源使用 * 每台服务端CPU使用: 2000%~2300%(刚好满负载) * 每台服务端内存使用: 14GB左右 * GC耗时: 504毫秒左右 * 流量使用: Incoming(450MBit/s), Outgoing(4.39GBit/s) ### 压测结果 * 推送到达: 3590万/秒左右; ## comet模块 ![benchmark-comet](benchmark-comet.jpg) ## 流量 ![benchmark-flow](benchmark-flow.jpg) ## heap信息(包含GC) ![benchmark-flow](benchmark-heap.jpg) ================================================ FILE: docs/benchmark_en.md ================================================ ## Benchmark Chart ![benchmark](benchmark.jpg) ### Benchmark Server | CPU | Memory | OS | Instance | | :---- | :---- | :---- | :---- | | Intel(R) Xeon(R) CPU E5-2630 v2 @ 2.60GHz | DDR3 32GB | Debian GNU/Linux 8 | 1 | ### Benchmark Case * Online: 1,000,000 * Duration: 15min * Push Speed: 40/s (broadcast room) * Push Message: {"test":1} * Received calc mode: 1s per times, total 30 times ### Benchmark Resource * CPU: 2000%~2300% * Memory: 14GB * GC Pause: 504ms * Network: Incoming(450MBit/s), Outgoing(4.39GBit/s) ### Benchmark Result * Received: 35,900,000/s ## Comet ![benchmark-comet](benchmark-comet.jpg) ## Network traffic ![benchmark-flow](benchmark-flow.jpg) ## Heap (include GC) ![benchmark-flow](benchmark-heap.jpg) ================================================ FILE: docs/en/proto.md ================================================ # comet and clients protocols comet supports two protocols to communicate with client: WebSocket, TCP ## websocket **Request URL** ws://DOMAIN/sub **HTTP Request Method** WebSocket (JSON Frame). Response is same as the request. **Response Result** ```json { "ver": 102, "op": 10, "seq": 10, "body": {"data": "xxx"} } ``` **Request and Response Parameters** | parameter | is required | type | comment| | :----- | :--- | :--- | :--- | | ver | true | int | Protocol version | | op | true | int | Operation | | seq | true | int | Sequence number (Server returned number maps to client sent) | | body | json | The JSON message pushed | ## tcp **Request URL** tcp://DOMAIN **Protocol** Binary. Response is same as the request. **Request and Response Parameters** | parameter | is required | type | comment| | :----- | :--- | :--- | :--- | | package length | true | int32 bigendian | package length | | header Length | true | int16 bigendian | header length | | ver | true | int16 bigendian | Protocol version | | operation | true | int32 bigendian | Operation | | seq | true | int32 bigendian | jsonp callback | | body | false | binary | $(package lenth) - $(header length) | ## Operations | operation | comment | | :----- | :--- | | 2 | Client send heartbeat| | 3 | Server reply heartbeat| | 7 | authentication request | | 8 | authentication response | ================================================ FILE: docs/en/push.md ================================================

Terry-Mao/goim push HTTP protocols

push HTTP interface protocols for pusher

Interfaces

| Name | URL | HTTP method | | :---- | :---- | :---- | | [single push](#single push) | /1/push | POST | | [multiple push](#multiple push) | /1/pushs | POST | | [room push](#room push) | /1/push/room | POST | | [broadcasting](#broadcasting) | /1/push/all | POST |

Public response body

| response code | description | | :---- | :---- | | 1 | success | | 65535 | internal error |

Response structure

{
    "ret": 1  //response code
}
##### single push * Example request ```sh # uid is the user id pushing to?uid=0 curl -d "{\"test\":1}" http://127.0.0.1:7172/1/push?uid=0 ``` * Response 
{
    "ret": 1
}
##### Multiple push * Example request ```sh curl -d "{\"u\":[1,2,3,4,5],\"m\":{\"test\":1}}" http://127.0.0.1:7172/1/pushs ``` * Response 
{
    "ret": 1
}
##### room push * Example request ```sh curl -d "{\"test\": 1}" http://127.0.0.1:7172/1/push/room?rid=1 ``` * Response 
{
    "ret": 1
}
##### Broadcasting * Example request ```sh curl -d "{\"test\": 1}" http://127.0.0.1:7172/1/push/all ``` * Response 
{
    "ret": 1
}
================================================ FILE: docs/proto.md ================================================ # comet 客户端通讯协议文档 comet支持两种协议和客户端通讯 websocket， tcp。 ## websocket **请求URL** ws://DOMAIN/sub **HTTP请求方式** Websocket（JSON Frame），请求和返回协议一致 **请求和返回json** ```json { "ver": 102, "op": 10, "seq": 10, "body": {"data": "xxx"} } ``` **请求和返回参数说明** | 参数名 | 必选 | 类型 | 说明 | | :----- | :--- | :--- | :--- | | ver | true | int | 协议版本号 | | op | true | int | 指令 | | seq | true | int | 序列号（服务端返回和客户端发送一一对应） | | body | true | string | 授权令牌，用于检验获取用户真实用户Id | ## tcp **请求URL** tcp://DOMAIN **协议格式** 二进制，请求和返回协议一致 **请求&返回参数** | 参数名 | 必选 | 类型 | 说明 | | :----- | :--- | :--- | :--- | | package length | true | int32 bigendian | 包长度 | | header Length | true | int16 bigendian | 包头长度 | | ver | true | int16 bigendian | 协议版本 | | operation | true | int32 bigendian | 协议指令 | | seq | true | int32 bigendian | 序列号 | | body | false | binary | $(package lenth) - $(header length) | ## 指令 | 指令 | 说明 | | :----- | :--- | | 2 | 客户端请求心跳 | | 3 | 服务端心跳答复 | | 5 | 下行消息 | | 7 | auth认证 | | 8 | auth认证返回 | ================================================ FILE: docs/push.md ================================================ ## goim push API ### error codes ``` // ok OK = 0 // request error RequestErr = -400 // server error ServerErr = -500 ``` ### push keys [POST] /goim/push/keys | Name | Type | Remork | |:----------------|:--------:|:-----------------------| | [url]:operation | int32 | operation for response | | [url]:keys | []string | multiple client keys | | [Body] | []byte | http request body | response: ``` { "code": 0 } ``` ### push mids [POST] /goim/push/mids | Name | Type | Remork | |:----------------|:--------:|:-----------------------| | [url]:operation | int32 | operation for response | | [url]:mids | []int64 | multiple user mids | | [Body] | []byte | http request body | response: ``` { "code": 0 } ``` ### push room [POST] /goim/push/room | Name | Type | Remork | |:----------------|:--------:|:-----------------------| | [url]:operation | int32 | operation for response | | [url]:type | string | room type | | [url]:room | string | room id | | [Body] | []byte | http request body | response: ``` { "code": 0 } ``` ### push all [POST] /goim/push/all | Name | Type | Remork | |:----------------|:--------:|:-----------------------| | [url]:operation | int32 | operation for response | | [url]:speed | int32 | push speed | | [Body] | []byte | http request body | response: ``` { "code": 0 } ``` ### online top [GET] /goim/online/top | Name | Type | Remork | |:--------|:--------:|:-----------------------| | type | string | room type | | limit | string | online limit | response: ``` { "code": 0, "message": "", "data": [ { "room_id": "1000", "count": 100 }, { "room_id": "2000", "count": 200 }, { "room_id": "3000", "count": 300 } ] } ``` ### online room [GET] /goim/online/room | Name | Type | Remork | |:--------|:--------:|:-----------------------| | type | string | room type | | rooms | []string | room ids | response: ``` { "code": 0, "message": "", "data": { "1000": 100, "2000": 200, "3000": 300 } } ``` ### online total [GET] /goim/online/total response: ``` { "code": 0, "message": "", "data": { "conn_count": 1, "ip_count": 1 } } ``` ### nodes weighted [GET] /goim/nodes/weighted | Name | Type | Remork | |:---------|:--------:|:-----------------------| | platform | string | web/android/ios | response: ``` { "code": 0, "message": "", "data": { "domain": "conn.goim.io", "tcp_port": 3101, "ws_port": 3102, "wss_port": 3103, "heartbeat": 30, // heartbeat seconds "heartbeat_max": 3 // heartbeat tries "nodes": [ "47.89.10.97" ], "backoff": { "max_delay": 300, "base_delay": 3, "factor": 1.8, "jitter": 0.3 }, } } ``` ### nodes instances [GET] /nodes/instances response: ``` { "code": 0, "message": "", "data": [ { "region": "sh", "zone": "sh001", "env": "dev", "appid": "goim.comet", "hostname": "test", "addrs": [ "grpc://192.168.1.30:3109" ], "version": "", "latest_timestamp": 1545750122311688676, "metadata": { "addrs": "47.89.10.97", "conn_count": "1", "ip_count": "1", "offline": "false", "weight": "10" } } ] } ` ================================================ FILE: examples/cert.pem ================================================ -----BEGIN CERTIFICATE----- MIID3TCCAsWgAwIBAgIJAKWU8wETRh4fMA0GCSqGSIb3DQEBBQUAMIGEMQswCQYD VQQGEwJjbjERMA8GA1UECAwIc2hhbmdoYWkxETAPBgNVBAcMCHNoYW5naGFpMQ0w CwYDVQQKDARiaWxpMQ0wCwYDVQQLDARiaWxpMREwDwYDVQQDDAhiaWxpLmNvbTEe MBwGCSqGSIb3DQEJARYPMjI0MzAzNTRAcXEuY29tMB4XDTE1MDkwMTEyMDQxMloX DTI1MDgyOTEyMDQxMlowgYQxCzAJBgNVBAYTAmNuMREwDwYDVQQIDAhzaGFuZ2hh aTERMA8GA1UEBwwIc2hhbmdoYWkxDTALBgNVBAoMBGJpbGkxDTALBgNVBAsMBGJp bGkxETAPBgNVBAMMCGJpbGkuY29tMR4wHAYJKoZIhvcNAQkBFg8yMjQzMDM1NEBx cS5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDKFwSxJqYPxzMe m5PeYA4YmcUsDCqS9Z7PsszOMZ1YsWZIHMB74D49ad2R+9PoqlfNH1L9C4NFSBrF rhSkaLmFYxw9yeJ2EAPijASBgfxMFVrEJhu7SW86OPTVnHblU8UqQdnMFOqF49C9 mCdbiGu/99BZVCL1WmlSQCWVEIzOgX+goxqHuwXUF58YUwr6WLtF0DuBcLUai1vB Pg+PJ2fLjSR2o0KJkPOd6+y90cgoxfyJFUHuUKyV8EU4VwHEIA9rVizprziqPx6c 9A9Ng0FpA2leSPLGYCjnDtKIOvbSOS8DMkRT55ujqoVrj0yiNWsuJlc/NbD6bS16 fJjuLOtJAgMBAAGjUDBOMB0GA1UdDgQWBBQzxdSIYIkDABh98Cj6VeYasC7/STAf BgNVHSMEGDAWgBQzxdSIYIkDABh98Cj6VeYasC7/STAMBgNVHRMEBTADAQH/MA0G CSqGSIb3DQEBBQUAA4IBAQA1Fhr+SU62xHWlPOBhTbjod49+mNfXn2TZz/vBp/Jl pHZgDLAEcrhXHmi2A0G9K9+qOIEn4BvTd70jSYvYlaeUSzZ/nEpeM0oE0f2Qaxov PhxDpsqPsSQm6pE64/los1doaiElfMVFaP56UGV01kFdI013wxwd2WCuj51Hmvi9 thsS027aqxjHMJnKXPvBm2E6EDkPfc/e+AEmwBzry+aamRizaMrk/SfSGTy9/rvd +VbBfHiJ50kMld51SLIc6qkVaTXess7mIfcsk7kyjP4eFA0y+3wmXfRZeWadND3I O9XNNwsDVFXlhW40GUVriy95qa1Sq3sLUfUQcCH4VFFK -----END CERTIFICATE----- ================================================ FILE: examples/javascript/client.js ================================================ (function(win) { const rawHeaderLen = 16; const packetOffset = 0; const headerOffset = 4; const verOffset = 6; const opOffset = 8; const seqOffset = 12; var Client = function(options) { var MAX_CONNECT_TIMES = 10; var DELAY = 15000; this.options = options || {}; this.createConnect(MAX_CONNECT_TIMES, DELAY); } var appendMsg = function(text) { var span = document.createElement("SPAN"); var text = document.createTextNode(text); span.appendChild(text); document.getElementById("box").appendChild(span); } Client.prototype.createConnect = function(max, delay) { var self = this; if (max === 0) { return; } connect(); var textDecoder = new TextDecoder(); var textEncoder = new TextEncoder(); var heartbeatInterval; function connect() { var ws = new WebSocket('ws://sh.tony.wiki:3102/sub'); //var ws = new WebSocket('ws://127.0.0.1:3102/sub'); ws.binaryType = 'arraybuffer'; ws.onopen = function() { auth(); } ws.onmessage = function(evt) { var data = evt.data; var dataView = new DataView(data, 0); var packetLen = dataView.getInt32(packetOffset); var headerLen = dataView.getInt16(headerOffset); var ver = dataView.getInt16(verOffset); var op = dataView.getInt32(opOffset); var seq = dataView.getInt32(seqOffset); console.log("receiveHeader: packetLen=" + packetLen, "headerLen=" + headerLen, "ver=" + ver, "op=" + op, "seq=" + seq); switch(op) { case 8: // auth reply ok document.getElementById("status").innerHTML = "ok"; appendMsg("receive: auth reply"); // send a heartbeat to server heartbeat(); heartbeatInterval = setInterval(heartbeat, 30 * 1000); break; case 3: // receive a heartbeat from server console.log("receive: heartbeat"); appendMsg("receive: heartbeat reply"); break; case 9: // batch message for (var offset=rawHeaderLen; offset client demo

websocket

status:

push

curl -d 'mid message' 'http://api.goim.io:3111/goim/push/mids?operation=1000&mids=123'

curl -d 'room message' 'http://api.goim.io:3111/goim/push/room?operation=1000&type=live&room=1000'

curl -d 'broadcast message' 'http://api.goim.io:3111/goim/push/all?operation=1000'

message:

================================================ FILE: examples/javascript/main.go ================================================ package main import ( "log" "net/http" ) func main() { // Simple static webserver: log.Fatal(http.ListenAndServe(":1999", http.FileServer(http.Dir("./")))) } ================================================ FILE: examples/private.pem ================================================ -----BEGIN RSA PRIVATE KEY----- MIIEpQIBAAKCAQEAyhcEsSamD8czHpuT3mAOGJnFLAwqkvWez7LMzjGdWLFmSBzA e+A+PWndkfvT6KpXzR9S/QuDRUgaxa4UpGi5hWMcPcnidhAD4owEgYH8TBVaxCYb u0lvOjj01Zx25VPFKkHZzBTqhePQvZgnW4hrv/fQWVQi9VppUkAllRCMzoF/oKMa h7sF1BefGFMK+li7RdA7gXC1GotbwT4Pjydny40kdqNCiZDznevsvdHIKMX8iRVB 7lCslfBFOFcBxCAPa1Ys6a84qj8enPQPTYNBaQNpXkjyxmAo5w7SiDr20jkvAzJE U+ebo6qFa49MojVrLiZXPzWw+m0tenyY7izrSQIDAQABAoIBAQCNasAwy2/nmKjw IUS/l44lru1oXncocdMZWvCw1c1a9IEzs1MLHKfRSBTyBDyNEy7v7pyfUQAiaku5 y5DMYDB65BkuL+lWXuypCvxYOEL6ZvMmUdiUHdZE8vh5xsz4u788S+qCQpy+5uX6 1s+r4PIt2tekuxjfgs4y7YqfHn66PmxAWXTV+jnWpUWYjpRTgDWeU6ikbk2fHjvG ytF4AV4lEKNnGBC5F+5lk3QJKjb1IW/o5q42TCeERk1k5Ruc5kfzq72hdFYrQAF1 PmstEXMExMS/K6AerYtPfiWWFWH2gCTZMf/C8Jwr0zPVpHM1PF+Ien3IkgiOTfZE 1+/wrx4BAoGBAOnpnxFnnqhZdDRi+DALfY3mXPwOizFYnAhuuuzO87zoZsps529E j5pEZQoYPTiww5rptjFhNjoV0gsh2GO5QHCiMxM8A76aWVc4YmK0GiHVudpAu2t+ aK8+0Xr03SA27cKJjp12NWijzEdTjSQYaFwD/8/dMrU3MCu30kdEJ6iJAoGBAN0s JXLv4CtB/DLTvifTZ/OAGSc1/Z+X0w2EqIugw7IV9YuAm2721IFhsFFXeANRRHY6 zonLUEgsuQUc30NCz+susChs2GMEFypLem8D1c5ZZY+9sIsR1WFB97o7bTMvp5lH 2REpEDZKXVOB7UmrclLgTKRQgG3O73rMP6QXYPzBAoGBAJm8Gvi0crlQuagol9fz 5Vwa2GgtItyW0U5VgHNdfSJeWBiYxO8DT6Jja0jcL3iP7K9nBYCk1KAOcVMxtmes fKbKY+kzW36tMSS7ASbAGiC8uH6yZru6hBERp1o5jw+6Kj/eaqYg5+9TIFKMnknn 5Mb9NecnCUnC8Nz63rBKIgqJAoGBAJHUpeyfFaPwIiYxT1RbJFN9xxf/lXdBWDu1 mJxYKDCoIfsVlWcZAQ0+KE+56LvnPcjnBX/9urWcJ3KjkuJ6jzV211gQTK0c6VlN 4zCHytYAQ+L/JATOgW9bW8hDnsD9TvjWUt3pwXLKnbaOGLNWhE747g/5tHSy2VyS h/PeJmkBAoGABMfEaiLHXXQhUK0BPYxT3T8i9IjAYwXlrgSLlnvOGslZef/45kP5 CM1UbMSwAn+HvAziOFt2WmynFCysy/lCyTud+Fd/IZFcMThp7wvi7fSZo0NDM7ES 9JfgTmCY4Kwv6kT85poIka9bp4Nh47EVB9kDoqm/lSMkfYqcWH66DJA= -----END RSA PRIVATE KEY----- ================================================ FILE: go.mod ================================================ module github.com/Terry-Mao/goim go 1.13 require ( github.com/BurntSushi/toml v0.3.1 github.com/Shopify/sarama v1.19.0 // indirect github.com/Shopify/toxiproxy v2.1.4+incompatible // indirect github.com/bilibili/discovery v1.0.1 github.com/bsm/sarama-cluster v2.1.15+incompatible github.com/eapache/go-resiliency v1.1.0 // indirect github.com/eapache/go-xerial-snappy v0.0.0-20180814174437-776d5712da21 // indirect github.com/eapache/queue v1.1.0 // indirect github.com/gin-gonic/gin v1.7.0 github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b github.com/golang/protobuf v1.4.3 github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db // indirect github.com/gomodule/redigo v2.0.0+incompatible github.com/google/uuid v1.0.0 github.com/onsi/ginkgo v1.16.0 // indirect github.com/onsi/gomega v1.11.0 // indirect github.com/pierrec/lz4 v2.0.5+incompatible // indirect github.com/rcrowley/go-metrics v0.0.0-20181016184325-3113b8401b8a // indirect github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d // indirect github.com/stretchr/testify v1.5.1 github.com/zhenjl/cityhash v0.0.0-20131128155616-cdd6a94144ab golang.org/x/net v0.0.0-20201202161906-c7110b5ffcbb google.golang.org/grpc v1.22.3 gopkg.in/Shopify/sarama.v1 v1.19.0 ) ================================================ FILE: go.sum ================================================ cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= github.com/BurntSushi/toml v0.0.0-20170626110600-a368813c5e64/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= github.com/BurntSushi/toml v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ= github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= github.com/Shopify/sarama v1.19.0 h1:9oksLxC6uxVPHPVYUmq6xhr1BOF/hHobWH2UzO67z1s= github.com/Shopify/sarama v1.19.0/go.mod h1:FVkBWblsNy7DGZRfXLU0O9RCGt5g3g3yEuWXgklEdEo= github.com/Shopify/toxiproxy v2.1.4+incompatible h1:TKdv8HiTLgE5wdJuEML90aBgNWsokNbMijUGhmcoBJc= github.com/Shopify/toxiproxy v2.1.4+incompatible/go.mod h1:OXgGpZ6Cli1/URJOF1DMxUHB2q5Ap20/P/eIdh4G0pI= github.com/bilibili/discovery v1.0.1 h1:6W9B2caxOdfBEKCMawwXU3dJ0W1TCONuprbXbkGe+s4= github.com/bilibili/discovery v1.0.1/go.mod h1:daS5nEYEBt0scrrmuoNCxWXDHFK6gtEpjhVKG6MUxUg= github.com/bsm/sarama-cluster v2.1.15+incompatible h1:RkV6WiNRnqEEbp81druK8zYhmnIgdOjqSVi0+9Cnl2A= github.com/bsm/sarama-cluster v2.1.15+incompatible/go.mod h1:r7ao+4tTNXvWm+VRpRJchr2kQhqxgmAp2iEX5W96gMM= github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/eapache/go-resiliency v1.1.0 h1:1NtRmCAqadE2FN4ZcN6g90TP3uk8cg9rn9eNK2197aU= github.com/eapache/go-resiliency v1.1.0/go.mod h1:kFI+JgMyC7bLPUVY133qvEBtVayf5mFgVsvEsIPBvNs= github.com/eapache/go-xerial-snappy v0.0.0-20180814174437-776d5712da21 h1:YEetp8/yCZMuEPMUDHG0CW/brkkEp8mzqk2+ODEitlw= github.com/eapache/go-xerial-snappy v0.0.0-20180814174437-776d5712da21/go.mod h1:+020luEh2TKB4/GOp8oxxtq0Daoen/Cii55CzbTV6DU= github.com/eapache/queue v1.1.0 h1:YOEu7KNc61ntiQlcEeUIoDTJ2o8mQznoNvUhiigpIqc= github.com/eapache/queue v1.1.0/go.mod h1:6eCeP0CKFpHLu8blIFXhExK/dRa7WDZfr6jVFPTqq+I= github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo= github.com/fsnotify/fsnotify v1.4.9 h1:hsms1Qyu0jgnwNXIxa+/V/PDsU6CfLf6CNO8H7IWoS4= github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ= github.com/gin-contrib/sse v0.0.0-20170109093832-22d885f9ecc7/go.mod h1:VJ0WA2NBN22VlZ2dKZQPAPnyWw5XTlK1KymzLKsr59s= github.com/gin-contrib/sse v0.1.0 h1:Y/yl/+YNO8GZSjAhjMsSuLt29uWRFHdHYUb5lYOV9qE= github.com/gin-contrib/sse v0.1.0/go.mod h1:RHrZQHXnP2xjPF+u1gW/2HnVO7nvIa9PG3Gm+fLHvGI= github.com/gin-gonic/gin v0.0.0-20180512030042-bf7803815b0b/go.mod h1:7cKuhb5qV2ggCFctp2fJQ+ErvciLZrIeoOSOm6mUr7Y= github.com/gin-gonic/gin v1.7.0 h1:jGB9xAJQ12AIGNB4HguylppmDK1Am9ppF7XnGXXJuoU= github.com/gin-gonic/gin v1.7.0/go.mod h1:jD2toBW3GZUr5UMcdrwQA10I7RuaFOl/SGeDjXkfUtY= github.com/go-playground/assert/v2 v2.0.1 h1:MsBgLAaY856+nPRTKrp3/OZK38U/wa0CcBYNjji3q3A= github.com/go-playground/assert/v2 v2.0.1/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4= github.com/go-playground/locales v0.13.0 h1:HyWk6mgj5qFqCT5fjGBuRArbVDfE4hi8+e8ceBS/t7Q= github.com/go-playground/locales v0.13.0/go.mod h1:taPMhCMXrRLJO55olJkUXHZBHCxTMfnGwq/HNwmWNS8= github.com/go-playground/universal-translator v0.17.0 h1:icxd5fm+REJzpZx7ZfpaD876Lmtgy7VtROAbHHXk8no= github.com/go-playground/universal-translator v0.17.0/go.mod h1:UkSxE5sNxxRwHyU+Scu5vgOQjsIJAF8j9muTVoKLVtA= github.com/go-playground/validator/v10 v10.4.1 h1:pH2c5ADXtd66mxoE0Zm9SUhxE20r7aM3F26W0hOn+GE= github.com/go-playground/validator/v10 v10.4.1/go.mod h1:nlOn6nFhuKACm19sB/8EGNn9GlaMV7XkbRSipzJ0Ii4= github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0/go.mod h1:fyg7847qk6SyHyPtNmDHnmrv/HOrqktSC+C9fM+CJOE= github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b h1:VKtxabqXZkF25pY9ekfRL6a582T4P37/31XEstQ5p58= github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q= github.com/golang/lint v0.0.0-20180702182130-06c8688daad7/go.mod h1:tluoj9z5200jBnyusfRPU2LqT6J+DAorxEvtC7LHB+E= github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw= github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8= github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA= github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs= github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w= github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0= github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= github.com/golang/protobuf v1.4.3 h1:JjCZWpVbqXDqFVmTfYWEVTMIYrL/NPdPSCHPJ0T/raM= github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db h1:woRePGFeVFfLKN/pOkfl+p/TAqKOfFu+7KPlMVpok/w= github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= github.com/gomodule/redigo v2.0.0+incompatible h1:K/R+8tc58AaqLkqG2Ol3Qk+DR/TlNuhuh457pBFPtt0= github.com/gomodule/redigo v2.0.0+incompatible/go.mod h1:B4C85qUVwatsJoIUNIfCRsp7qO0iAmpGFZ4EELWSbC4= github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= github.com/google/go-cmp v0.4.0 h1:xsAVV57WRhGj6kEIi8ReJzQlHHqcBYCElAvkovg3B/4= github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/uuid v1.0.0 h1:b4Gk+7WdP/d3HZH8EJsZpvV7EtDOgaZLtnaNGIu1adA= github.com/google/uuid v1.0.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/gopherjs/gopherjs v0.0.0-20180424202546-8dffc02ea1cb h1:g0omhilXoAZ+6sFcF6puAzT+/MoKK3ZBQ9e7nVIRjrc= github.com/gopherjs/gopherjs v0.0.0-20180424202546-8dffc02ea1cb/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY= github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU= github.com/json-iterator/go v0.0.0-20180526014329-8744d7c5c7b4/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU= github.com/json-iterator/go v1.1.9 h1:9yzud/Ht36ygwatGx56VwCZtlI/2AD15T1X2sjSuGns= github.com/json-iterator/go v1.1.9/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= github.com/jtolds/gls v4.2.1+incompatible h1:fSuqC+Gmlu6l/ZYAoZzx2pyucC8Xza35fpRVWLVmUEE= github.com/jtolds/gls v4.2.1+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU= github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= github.com/leodido/go-urn v1.2.0 h1:hpXL4XnriNwQ/ABnpepYM/1vCLWNDfUNts8dX3xTG6Y= github.com/leodido/go-urn v1.2.0/go.mod h1:+8+nEpDfqqsY+g338gtMEUOtuK+4dEMhiQEgxpxOKII= github.com/mattn/go-isatty v0.0.4/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4= github.com/mattn/go-isatty v0.0.12 h1:wuysRhFDzyxgEmMf5xjvJ2M9dZoWAXNNr5LSBS7uHXY= github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU= github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/reflect2 v0.0.0-20180511053014-58118c1ea916/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742 h1:Esafd1046DLDQ0W1YjYsBW+p8U2u7vzgW2SQVmlNazg= github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= github.com/nbio/st v0.0.0-20140626010706-e9e8d9816f32 h1:W6apQkHrMkS0Muv8G/TipAy/FJl/rCYT0+EuS8+Z0z4= github.com/nbio/st v0.0.0-20140626010706-e9e8d9816f32/go.mod h1:9wM+0iRr9ahx58uYLpLIr5fm8diHn0JbqRycJi6w0Ms= github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A= github.com/nxadm/tail v1.4.8 h1:nPr65rt6Y5JFSKQO7qToXr7pePgD6Gwiw05lkbyAQTE= github.com/nxadm/tail v1.4.8/go.mod h1:+ncqLTQzXmGhMZNUePPaPqPvBxHAIsmXswZKocGu+AU= github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= github.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk= github.com/onsi/ginkgo v1.16.0 h1:NBrNLB37exjJLxXtFOktx6CISBdS1aF8+7MwKlTV8U4= github.com/onsi/ginkgo v1.16.0/go.mod h1:CObGmKUOKaSC0RjmoAK7tKyn4Azo5P2IWuoMnvwxz1E= github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY= github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo= github.com/onsi/gomega v1.11.0 h1:+CqWgvj0OZycCaqclBD1pxKHAU+tOkHmQIWvDHq2aug= github.com/onsi/gomega v1.11.0/go.mod h1:azGKhqFUon9Vuj0YmTfLSmx0FUwqXYSTl5re8lQLTUg= github.com/pierrec/lz4 v2.0.5+incompatible h1:2xWsjqPFWcplujydGg4WmhC/6fZqK42wMM8aXeqhl0I= github.com/pierrec/lz4 v2.0.5+incompatible/go.mod h1:pdkljMzZIN41W+lC3N2tnIh5sFi+IEE17M5jbnwPHcY= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/rcrowley/go-metrics v0.0.0-20181016184325-3113b8401b8a h1:9ZKAASQSHhDYGoxY8uLVpewe1GDZ2vu2Tr/vTdVAkFQ= github.com/rcrowley/go-metrics v0.0.0-20181016184325-3113b8401b8a/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4= github.com/smartystreets/assertions v0.0.0-20180301161246-7678a5452ebe/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc= github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d h1:zE9ykElWQ6/NYmHa3jpm/yHnI4xSofP+UP6SpjHcSeM= github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc= github.com/smartystreets/goconvey v0.0.0-20180222194500-ef6db91d284a h1:JSvGDIbmil4Ui/dDdFBExb7/cmkNjyX5F97oglmvCDo= github.com/smartystreets/goconvey v0.0.0-20180222194500-ef6db91d284a/go.mod h1:XDJAKZRPZ1CvBcN2aX5YOUTYGHki24fSF0Iv48Ibg0s= github.com/smartystreets/gunit v0.0.0-20180314194857-6f0d6275bdcd/go.mod h1:XUKj4gbqj2QvJk/OdLWzyZ3FYli0f+MdpngyryX0gcw= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= github.com/stretchr/testify v1.5.1 h1:nOGnQDM7FYENwehXlg/kFVnos3rEvtKTjRvOWSzb6H4= github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA= github.com/ugorji/go v0.0.0-20180407103000-f3cacc17c85e/go.mod h1:hnLbHMwcvSihnDhEfx2/BzKp2xb0Y+ErdfYcrs9tkJQ= github.com/ugorji/go v1.1.7 h1:/68gy2h+1mWMrwZFeD1kQialdSzAb432dtpeJ42ovdo= github.com/ugorji/go v1.1.7/go.mod h1:kZn38zHttfInRq0xu/PH0az30d+z6vm202qpg1oXVMw= github.com/ugorji/go/codec v1.1.7 h1:2SvQaVZ1ouYrrKKwoSk2pzd4A9evlKJb9oTL+OaLUSs= github.com/ugorji/go/codec v1.1.7/go.mod h1:Ax+UKWsSmolVDwsd+7N3ZtXu+yMGCf907BLYF3GoBXY= github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/zhenjl/cityhash v0.0.0-20131128155616-cdd6a94144ab h1:BWHvAOZz0pBILkGl/ebPQKZDrqbaWj/iN9RE8AvaTvg= github.com/zhenjl/cityhash v0.0.0-20131128155616-cdd6a94144ab/go.mod h1:P6L88wrqK99Njntah9SB7AyzFpUXsXYq06LkjixxQmY= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9 h1:psW17arqaxU48Z5kZ0CQnkZWQJsqcURM6tKiBApRjXI= golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/lint v0.0.0-20180702182130-06c8688daad7/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200520004742-59133d7f0dd7/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= golang.org/x/net v0.0.0-20201202161906-c7110b5ffcbb h1:eBmm0M9fYhWpKZLjQUUKka/LtIxf46G4fxeEz5KJr9U= golang.org/x/net v0.0.0-20201202161906-c7110b5ffcbb/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9 h1:SQFwaSi55rU7vdNs9Yr0Z324VNlrF+0wMqRXT4St8ck= golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190904154756-749cb33beabd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210112080510-489259a85091 h1:DMyOG0U+gKfu8JZzg2UQe9MeaC1X+xQWlAKcRnjxjCw= golang.org/x/sys v0.0.0-20210112080510-489259a85091/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= golang.org/x/text v0.3.3 h1:cokOdA+Jmi5PJGXLlLllQSgYigAEfHXJAERHVMaCc2k= golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/tools v0.0.0-20180828015842-6cd1fcedba52/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20201224043029-2b0845dc783e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE= golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM= google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8 h1:Nw54tB0rB7hY/N0NQvRW8DG4Yk3Q6T9cu9RcFQDu1tc= google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc= google.golang.org/grpc v0.0.0-20181030232906-a88340f3c899/go.mod h1:0JHn/cJsOMiMfNA9+DeHDlAU7KAAB5GDlYFpa9MZMio= google.golang.org/grpc v1.22.3 h1:U4Oh9xJixJwAFqa1c5uLhNAh8ERM/lc3hNhPEJiAEhs= google.golang.org/grpc v1.22.3/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg= google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8= google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0= google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM= google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE= google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo= google.golang.org/protobuf v1.23.0 h1:4MY060fB1DLGMB/7MBTLnwQUY6+F09GEiz6SsrNqyzM= google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= gopkg.in/Shopify/sarama.v1 v1.19.0 h1:yvI/R1jfMpKvvwmX4r/AQjaI5oszWEOlvKxUdaj53OM= gopkg.in/Shopify/sarama.v1 v1.19.0/go.mod h1:AxnvoaevB2nBjNK17cG61A3LleFcWFwVBHBt+cot4Oc= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys= gopkg.in/go-playground/assert.v1 v1.2.1/go.mod h1:9RXL0bg/zibRAgZUYszZSwO/z8Y/a8bDuhia5mkpMnE= gopkg.in/go-playground/validator.v8 v8.18.2/go.mod h1:RX2a/7Ha8BgOhfk7j780h4/u/RRjR0eouCJSH80/M2Y= gopkg.in/h2non/gock.v1 v1.0.8 h1:P8Ul3tXxL84suEhp+a7Uu6f9rBszP+gLkae2D6U1gS0= gopkg.in/h2non/gock.v1 v1.0.8/go.mod h1:KHI4Z1sxDW6P4N3DfTWSEza07YpkQP7KJBfglRMEjKY= gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ= gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw= gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= honnef.co/go/tools v0.0.0-20180728063816-88497007e858/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= ================================================ FILE: internal/comet/bucket.go ================================================ package comet import ( "sync" "sync/atomic" pb "github.com/Terry-Mao/goim/api/comet" "github.com/Terry-Mao/goim/api/protocol" "github.com/Terry-Mao/goim/internal/comet/conf" ) // Bucket is a channel holder. type Bucket struct { c *conf.Bucket cLock sync.RWMutex // protect the channels for chs chs map[string]*Channel // map sub key to a channel // room rooms map[string]*Room // bucket room channels routines []chan *pb.BroadcastRoomReq routinesNum uint64 ipCnts map[string]int32 } // NewBucket new a bucket struct. store the key with im channel. func NewBucket(c *conf.Bucket) (b *Bucket) { b = new(Bucket) b.chs = make(map[string]*Channel, c.Channel) b.ipCnts = make(map[string]int32) b.c = c b.rooms = make(map[string]*Room, c.Room) b.routines = make([]chan *pb.BroadcastRoomReq, c.RoutineAmount) for i := uint64(0); i < c.RoutineAmount; i++ { c := make(chan *pb.BroadcastRoomReq, c.RoutineSize) b.routines[i] = c go b.roomproc(c) } return } // ChannelCount channel count in the bucket func (b *Bucket) ChannelCount() int { return len(b.chs) } // RoomCount room count in the bucket func (b *Bucket) RoomCount() int { return len(b.rooms) } // RoomsCount get all room id where online number > 0. func (b *Bucket) RoomsCount() (res map[string]int32) { var ( roomID string room *Room ) b.cLock.RLock() res = make(map[string]int32) for roomID, room = range b.rooms { if room.Online > 0 { res[roomID] = room.Online } } b.cLock.RUnlock() return } // ChangeRoom change ro room func (b *Bucket) ChangeRoom(nrid string, ch *Channel) (err error) { var ( nroom *Room ok bool oroom = ch.Room ) // change to no room if nrid == "" { if oroom != nil && oroom.Del(ch) { b.DelRoom(oroom) } ch.Room = nil return } b.cLock.Lock() if nroom, ok = b.rooms[nrid]; !ok { nroom = NewRoom(nrid) b.rooms[nrid] = nroom } b.cLock.Unlock() if oroom != nil && oroom.Del(ch) { b.DelRoom(oroom) } if err = nroom.Put(ch); err != nil { return } ch.Room = nroom return } // Put put a channel according with sub key. func (b *Bucket) Put(rid string, ch *Channel) (err error) { var ( room *Room ok bool ) b.cLock.Lock() // close old channel if dch := b.chs[ch.Key]; dch != nil { dch.Close() } b.chs[ch.Key] = ch if rid != "" { if room, ok = b.rooms[rid]; !ok { room = NewRoom(rid) b.rooms[rid] = room } ch.Room = room } b.ipCnts[ch.IP]++ b.cLock.Unlock() if room != nil { err = room.Put(ch) } return } // Del delete the channel by sub key. func (b *Bucket) Del(dch *Channel) { room := dch.Room b.cLock.Lock() if ch, ok := b.chs[dch.Key]; ok { if ch == dch { delete(b.chs, ch.Key) } // ip counter if b.ipCnts[ch.IP] > 1 { b.ipCnts[ch.IP]-- } else { delete(b.ipCnts, ch.IP) } } b.cLock.Unlock() if room != nil && room.Del(dch) { // if empty room, must delete from bucket b.DelRoom(room) } } // Channel get a channel by sub key. func (b *Bucket) Channel(key string) (ch *Channel) { b.cLock.RLock() ch = b.chs[key] b.cLock.RUnlock() return } // Broadcast push msgs to all channels in the bucket. func (b *Bucket) Broadcast(p *protocol.Proto, op int32) { var ch *Channel b.cLock.RLock() for _, ch = range b.chs { if !ch.NeedPush(op) { continue } _ = ch.Push(p) } b.cLock.RUnlock() } // Room get a room by roomid. func (b *Bucket) Room(rid string) (room *Room) { b.cLock.RLock() room = b.rooms[rid] b.cLock.RUnlock() return } // DelRoom delete a room by roomid. func (b *Bucket) DelRoom(room *Room) { b.cLock.Lock() delete(b.rooms, room.ID) b.cLock.Unlock() room.Close() } // BroadcastRoom broadcast a message to specified room func (b *Bucket) BroadcastRoom(arg *pb.BroadcastRoomReq) { num := atomic.AddUint64(&b.routinesNum, 1) % b.c.RoutineAmount b.routines[num] <- arg } // Rooms get all room id where online number > 0. func (b *Bucket) Rooms() (res map[string]struct{}) { var ( roomID string room *Room ) res = make(map[string]struct{}) b.cLock.RLock() for roomID, room = range b.rooms { if room.Online > 0 { res[roomID] = struct{}{} } } b.cLock.RUnlock() return } // IPCount get ip count. func (b *Bucket) IPCount() (res map[string]struct{}) { var ( ip string ) b.cLock.RLock() res = make(map[string]struct{}, len(b.ipCnts)) for ip = range b.ipCnts { res[ip] = struct{}{} } b.cLock.RUnlock() return } // UpRoomsCount update all room count func (b *Bucket) UpRoomsCount(roomCountMap map[string]int32) { var ( roomID string room *Room ) b.cLock.RLock() for roomID, room = range b.rooms { room.AllOnline = roomCountMap[roomID] } b.cLock.RUnlock() } // roomproc func (b *Bucket) roomproc(c chan *pb.BroadcastRoomReq) { for { arg := <-c if room := b.Room(arg.RoomID); room != nil { room.Push(arg.Proto) } } } ================================================ FILE: internal/comet/channel.go ================================================ package comet import ( "sync" "github.com/Terry-Mao/goim/api/protocol" "github.com/Terry-Mao/goim/internal/comet/errors" "github.com/Terry-Mao/goim/pkg/bufio" ) // Channel used by message pusher send msg to write goroutine. type Channel struct { Room *Room CliProto Ring signal chan *protocol.Proto Writer bufio.Writer Reader bufio.Reader Next *Channel Prev *Channel Mid int64 Key string IP string watchOps map[int32]struct{} mutex sync.RWMutex } // NewChannel new a channel. func NewChannel(cli, svr int) *Channel { c := new(Channel) c.CliProto.Init(cli) c.signal = make(chan *protocol.Proto, svr) c.watchOps = make(map[int32]struct{}) return c } // Watch watch a operation. func (c *Channel) Watch(accepts ...int32) { c.mutex.Lock() for _, op := range accepts { c.watchOps[op] = struct{}{} } c.mutex.Unlock() } // UnWatch unwatch an operation func (c *Channel) UnWatch(accepts ...int32) { c.mutex.Lock() for _, op := range accepts { delete(c.watchOps, op) } c.mutex.Unlock() } // NeedPush verify if in watch. func (c *Channel) NeedPush(op int32) bool { c.mutex.RLock() if _, ok := c.watchOps[op]; ok { c.mutex.RUnlock() return true } c.mutex.RUnlock() return false } // Push server push message. func (c *Channel) Push(p *protocol.Proto) (err error) { select { case c.signal <- p: default: err = errors.ErrSignalFullMsgDropped } return } // Ready check the channel ready or close? func (c *Channel) Ready() *protocol.Proto { return <-c.signal } // Signal send signal to the channel, protocol ready. func (c *Channel) Signal() { c.signal <- protocol.ProtoReady } // Close close the channel. func (c *Channel) Close() { c.signal <- protocol.ProtoFinish } ================================================ FILE: internal/comet/conf/conf.go ================================================ package conf import ( "flag" "os" "strconv" "strings" "time" "github.com/bilibili/discovery/naming" "github.com/BurntSushi/toml" xtime "github.com/Terry-Mao/goim/pkg/time" ) var ( confPath string region string zone string deployEnv string host string addrs string weight int64 offline bool debug bool // Conf config Conf *Config ) func init() { var ( defHost, _ = os.Hostname() defAddrs = os.Getenv("ADDRS") defWeight, _ = strconv.ParseInt(os.Getenv("WEIGHT"), 10, 32) defOffline, _ = strconv.ParseBool(os.Getenv("OFFLINE")) defDebug, _ = strconv.ParseBool(os.Getenv("DEBUG")) ) flag.StringVar(&confPath, "conf", "comet-example.toml", "default config path.") flag.StringVar(®ion, "region", os.Getenv("REGION"), "avaliable region. or use REGION env variable, value: sh etc.") flag.StringVar(&zone, "zone", os.Getenv("ZONE"), "avaliable zone. or use ZONE env variable, value: sh001/sh002 etc.") flag.StringVar(&deployEnv, "deploy.env", os.Getenv("DEPLOY_ENV"), "deploy env. or use DEPLOY_ENV env variable, value: dev/fat1/uat/pre/prod etc.") flag.StringVar(&host, "host", defHost, "machine hostname. or use default machine hostname.") flag.StringVar(&addrs, "addrs", defAddrs, "server public ip addrs. or use ADDRS env variable, value: 127.0.0.1 etc.") flag.Int64Var(&weight, "weight", defWeight, "load balancing weight, or use WEIGHT env variable, value: 10 etc.") flag.BoolVar(&offline, "offline", defOffline, "server offline. or use OFFLINE env variable, value: true/false etc.") flag.BoolVar(&debug, "debug", defDebug, "server debug. or use DEBUG env variable, value: true/false etc.") } // Init init config. func Init() (err error) { Conf = Default() _, err = toml.DecodeFile(confPath, &Conf) return } // Default new a config with specified defualt value. func Default() *Config { return &Config{ Debug: debug, Env: &Env{Region: region, Zone: zone, DeployEnv: deployEnv, Host: host, Weight: weight, Addrs: strings.Split(addrs, ","), Offline: offline}, Discovery: &naming.Config{Region: region, Zone: zone, Env: deployEnv, Host: host}, RPCClient: &RPCClient{ Dial: xtime.Duration(time.Second), Timeout: xtime.Duration(time.Second), }, RPCServer: &RPCServer{ Network: "tcp", Addr: ":3109", Timeout: xtime.Duration(time.Second), IdleTimeout: xtime.Duration(time.Second * 60), MaxLifeTime: xtime.Duration(time.Hour * 2), ForceCloseWait: xtime.Duration(time.Second * 20), KeepAliveInterval: xtime.Duration(time.Second * 60), KeepAliveTimeout: xtime.Duration(time.Second * 20), }, TCP: &TCP{ Bind: []string{":3101"}, Sndbuf: 4096, Rcvbuf: 4096, KeepAlive: false, Reader: 32, ReadBuf: 1024, ReadBufSize: 8192, Writer: 32, WriteBuf: 1024, WriteBufSize: 8192, }, Websocket: &Websocket{ Bind: []string{":3102"}, }, Protocol: &Protocol{ Timer: 32, TimerSize: 2048, CliProto: 5, SvrProto: 10, HandshakeTimeout: xtime.Duration(time.Second * 5), }, Bucket: &Bucket{ Size: 32, Channel: 1024, Room: 1024, RoutineAmount: 32, RoutineSize: 1024, }, } } // Config is comet config. type Config struct { Debug bool Env *Env Discovery *naming.Config TCP *TCP Websocket *Websocket Protocol *Protocol Bucket *Bucket RPCClient *RPCClient RPCServer *RPCServer Whitelist *Whitelist } // Env is env config. type Env struct { Region string Zone string DeployEnv string Host string Weight int64 Offline bool Addrs []string } // RPCClient is RPC client config. type RPCClient struct { Dial xtime.Duration Timeout xtime.Duration } // RPCServer is RPC server config. type RPCServer struct { Network string Addr string Timeout xtime.Duration IdleTimeout xtime.Duration MaxLifeTime xtime.Duration ForceCloseWait xtime.Duration KeepAliveInterval xtime.Duration KeepAliveTimeout xtime.Duration } // TCP is tcp config. type TCP struct { Bind []string Sndbuf int Rcvbuf int KeepAlive bool Reader int ReadBuf int ReadBufSize int Writer int WriteBuf int WriteBufSize int } // Websocket is websocket config. type Websocket struct { Bind []string TLSOpen bool TLSBind []string CertFile string PrivateFile string } // Protocol is protocol config. type Protocol struct { Timer int TimerSize int SvrProto int CliProto int HandshakeTimeout xtime.Duration } // Bucket is bucket config. type Bucket struct { Size int Channel int Room int RoutineAmount uint64 RoutineSize int } // Whitelist is white list config. type Whitelist struct { Whitelist []int64 WhiteLog string } ================================================ FILE: internal/comet/errors/errors.go ================================================ package errors import ( "errors" ) // . var ( // server ErrHandshake = errors.New("handshake failed") ErrOperation = errors.New("request operation not valid") // ring ErrRingEmpty = errors.New("ring buffer empty") ErrRingFull = errors.New("ring buffer full") // timer ErrTimerFull = errors.New("timer full") ErrTimerEmpty = errors.New("timer empty") ErrTimerNoItem = errors.New("timer item not exist") // channel ErrPushMsgArg = errors.New("rpc pushmsg arg error") ErrPushMsgsArg = errors.New("rpc pushmsgs arg error") ErrMPushMsgArg = errors.New("rpc mpushmsg arg error") ErrMPushMsgsArg = errors.New("rpc mpushmsgs arg error") ErrSignalFullMsgDropped = errors.New("signal channel full, msg dropped") // bucket ErrBroadCastArg = errors.New("rpc broadcast arg error") ErrBroadCastRoomArg = errors.New("rpc broadcast room arg error") // room ErrRoomDroped = errors.New("room droped") // rpc ErrLogic = errors.New("logic rpc is not available") ) ================================================ FILE: internal/comet/grpc/server.go ================================================ package grpc import ( "context" "net" "time" pb "github.com/Terry-Mao/goim/api/comet" "github.com/Terry-Mao/goim/internal/comet" "github.com/Terry-Mao/goim/internal/comet/conf" "github.com/Terry-Mao/goim/internal/comet/errors" "google.golang.org/grpc" "google.golang.org/grpc/keepalive" ) // New comet grpc server. func New(c *conf.RPCServer, s *comet.Server) *grpc.Server { keepParams := grpc.KeepaliveParams(keepalive.ServerParameters{ MaxConnectionIdle: time.Duration(c.IdleTimeout), MaxConnectionAgeGrace: time.Duration(c.ForceCloseWait), Time: time.Duration(c.KeepAliveInterval), Timeout: time.Duration(c.KeepAliveTimeout), MaxConnectionAge: time.Duration(c.MaxLifeTime), }) srv := grpc.NewServer(keepParams) pb.RegisterCometServer(srv, &server{s}) lis, err := net.Listen(c.Network, c.Addr) if err != nil { panic(err) } go func() { if err := srv.Serve(lis); err != nil { panic(err) } }() return srv } type server struct { srv *comet.Server } var _ pb.CometServer = &server{} // PushMsg push a message to specified sub keys. func (s *server) PushMsg(ctx context.Context, req *pb.PushMsgReq) (reply *pb.PushMsgReply, err error) { if len(req.Keys) == 0 || req.Proto == nil { return nil, errors.ErrPushMsgArg } for _, key := range req.Keys { bucket := s.srv.Bucket(key) if bucket == nil { continue } if channel := bucket.Channel(key); channel != nil { if !channel.NeedPush(req.ProtoOp) { continue } if err = channel.Push(req.Proto); err != nil { return } } } return &pb.PushMsgReply{}, nil } // Broadcast broadcast msg to all user. func (s *server) Broadcast(ctx context.Context, req *pb.BroadcastReq) (*pb.BroadcastReply, error) { if req.Proto == nil { return nil, errors.ErrBroadCastArg } // TODO use broadcast queue go func() { for _, bucket := range s.srv.Buckets() { bucket.Broadcast(req.GetProto(), req.ProtoOp) if req.Speed > 0 { t := bucket.ChannelCount() / int(req.Speed) time.Sleep(time.Duration(t) * time.Second) } } }() return &pb.BroadcastReply{}, nil } // BroadcastRoom broadcast msg to specified room. func (s *server) BroadcastRoom(ctx context.Context, req *pb.BroadcastRoomReq) (*pb.BroadcastRoomReply, error) { if req.Proto == nil || req.RoomID == "" { return nil, errors.ErrBroadCastRoomArg } for _, bucket := range s.srv.Buckets() { bucket.BroadcastRoom(req) } return &pb.BroadcastRoomReply{}, nil } // Rooms gets all the room ids for the server. func (s *server) Rooms(ctx context.Context, req *pb.RoomsReq) (*pb.RoomsReply, error) { var ( roomIds = make(map[string]bool) ) for _, bucket := range s.srv.Buckets() { for roomID := range bucket.Rooms() { roomIds[roomID] = true } } return &pb.RoomsReply{Rooms: roomIds}, nil } ================================================ FILE: internal/comet/operation.go ================================================ package comet import ( "context" "time" "github.com/Terry-Mao/goim/api/logic" "github.com/Terry-Mao/goim/api/protocol" "github.com/Terry-Mao/goim/pkg/strings" log "github.com/golang/glog" "google.golang.org/grpc" "google.golang.org/grpc/encoding/gzip" ) // Connect connected a connection. func (s *Server) Connect(c context.Context, p *protocol.Proto, cookie string) (mid int64, key, rid string, accepts []int32, heartbeat time.Duration, err error) { reply, err := s.rpcClient.Connect(c, &logic.ConnectReq{ Server: s.serverID, Cookie: cookie, Token: p.Body, }) if err != nil { return } return reply.Mid, reply.Key, reply.RoomID, reply.Accepts, time.Duration(reply.Heartbeat), nil } // Disconnect disconnected a connection. func (s *Server) Disconnect(c context.Context, mid int64, key string) (err error) { _, err = s.rpcClient.Disconnect(context.Background(), &logic.DisconnectReq{ Server: s.serverID, Mid: mid, Key: key, }) return } // Heartbeat heartbeat a connection session. func (s *Server) Heartbeat(ctx context.Context, mid int64, key string) (err error) { _, err = s.rpcClient.Heartbeat(ctx, &logic.HeartbeatReq{ Server: s.serverID, Mid: mid, Key: key, }) return } // RenewOnline renew room online. func (s *Server) RenewOnline(ctx context.Context, serverID string, roomCount map[string]int32) (allRoom map[string]int32, err error) { reply, err := s.rpcClient.RenewOnline(ctx, &logic.OnlineReq{ Server: s.serverID, RoomCount: roomCount, }, grpc.UseCompressor(gzip.Name)) if err != nil { return } return reply.AllRoomCount, nil } // Receive receive a message. func (s *Server) Receive(ctx context.Context, mid int64, p *protocol.Proto) (err error) { _, err = s.rpcClient.Receive(ctx, &logic.ReceiveReq{Mid: mid, Proto: p}) return } // Operate operate. func (s *Server) Operate(ctx context.Context, p *protocol.Proto, ch *Channel, b *Bucket) error { switch p.Op { case protocol.OpChangeRoom: if err := b.ChangeRoom(string(p.Body), ch); err != nil { log.Errorf("b.ChangeRoom(%s) error(%v)", p.Body, err) } p.Op = protocol.OpChangeRoomReply case protocol.OpSub: if ops, err := strings.SplitInt32s(string(p.Body), ","); err == nil { ch.Watch(ops...) } p.Op = protocol.OpSubReply case protocol.OpUnsub: if ops, err := strings.SplitInt32s(string(p.Body), ","); err == nil { ch.UnWatch(ops...) } p.Op = protocol.OpUnsubReply default: // TODO ack ok&failed if err := s.Receive(ctx, ch.Mid, p); err != nil { log.Errorf("s.Report(%d) op:%d error(%v)", ch.Mid, p.Op, err) } p.Body = nil } return nil } ================================================ FILE: internal/comet/ring.go ================================================ package comet import ( "github.com/Terry-Mao/goim/api/protocol" "github.com/Terry-Mao/goim/internal/comet/conf" "github.com/Terry-Mao/goim/internal/comet/errors" log "github.com/golang/glog" ) // Ring ring proto buffer. type Ring struct { // read rp uint64 num uint64 mask uint64 // TODO split cacheline, many cpu cache line size is 64 // pad [40]byte // write wp uint64 data []protocol.Proto } // NewRing new a ring buffer. func NewRing(num int) *Ring { r := new(Ring) r.init(uint64(num)) return r } // Init init ring. func (r *Ring) Init(num int) { r.init(uint64(num)) } func (r *Ring) init(num uint64) { // 2^N if num&(num-1) != 0 { for num&(num-1) != 0 { num &= num - 1 } num <<= 1 } r.data = make([]protocol.Proto, num) r.num = num r.mask = r.num - 1 } // Get get a proto from ring. func (r *Ring) Get() (proto *protocol.Proto, err error) { if r.rp == r.wp { return nil, errors.ErrRingEmpty } proto = &r.data[r.rp&r.mask] return } // GetAdv incr read index. func (r *Ring) GetAdv() { r.rp++ if conf.Conf.Debug { log.Infof("ring rp: %d, idx: %d", r.rp, r.rp&r.mask) } } // Set get a proto to write. func (r *Ring) Set() (proto *protocol.Proto, err error) { if r.wp-r.rp >= r.num { return nil, errors.ErrRingFull } proto = &r.data[r.wp&r.mask] return } // SetAdv incr write index. func (r *Ring) SetAdv() { r.wp++ if conf.Conf.Debug { log.Infof("ring wp: %d, idx: %d", r.wp, r.wp&r.mask) } } // Reset reset ring. func (r *Ring) Reset() { r.rp = 0 r.wp = 0 // prevent pad compiler optimization // r.pad = [40]byte{} } ================================================ FILE: internal/comet/room.go ================================================ package comet import ( "sync" "github.com/Terry-Mao/goim/api/protocol" "github.com/Terry-Mao/goim/internal/comet/errors" ) // Room is a room and store channel room info. type Room struct { ID string rLock sync.RWMutex next *Channel drop bool Online int32 // dirty read is ok AllOnline int32 } // NewRoom new a room struct, store channel room info. func NewRoom(id string) (r *Room) { r = new(Room) r.ID = id r.drop = false r.next = nil r.Online = 0 return } // Put put channel into the room. func (r *Room) Put(ch *Channel) (err error) { r.rLock.Lock() if !r.drop { if r.next != nil { r.next.Prev = ch } ch.Next = r.next ch.Prev = nil r.next = ch // insert to header r.Online++ } else { err = errors.ErrRoomDroped } r.rLock.Unlock() return } // Del delete channel from the room. func (r *Room) Del(ch *Channel) bool { r.rLock.Lock() if ch.Next != nil { // if not footer ch.Next.Prev = ch.Prev } if ch.Prev != nil { // if not header ch.Prev.Next = ch.Next } else { r.next = ch.Next } ch.Next = nil ch.Prev = nil r.Online-- r.drop = r.Online == 0 r.rLock.Unlock() return r.drop } // Push push msg to the room, if chan full discard it. func (r *Room) Push(p *protocol.Proto) { r.rLock.RLock() for ch := r.next; ch != nil; ch = ch.Next { _ = ch.Push(p) } r.rLock.RUnlock() } // Close close the room. func (r *Room) Close() { r.rLock.RLock() for ch := r.next; ch != nil; ch = ch.Next { ch.Close() } r.rLock.RUnlock() } // OnlineNum the room all online. func (r *Room) OnlineNum() int32 { if r.AllOnline > 0 { return r.AllOnline } return r.Online } ================================================ FILE: internal/comet/round.go ================================================ package comet import ( "github.com/Terry-Mao/goim/internal/comet/conf" "github.com/Terry-Mao/goim/pkg/bytes" "github.com/Terry-Mao/goim/pkg/time" ) // RoundOptions round options. type RoundOptions struct { Timer int TimerSize int Reader int ReadBuf int ReadBufSize int Writer int WriteBuf int WriteBufSize int } // Round used for connection round-robin get a reader/writer/timer for split big lock. type Round struct { readers []bytes.Pool writers []bytes.Pool timers []time.Timer options RoundOptions } // NewRound new a round struct. func NewRound(c *conf.Config) (r *Round) { var i int r = &Round{ options: RoundOptions{ Reader: c.TCP.Reader, ReadBuf: c.TCP.ReadBuf, ReadBufSize: c.TCP.ReadBufSize, Writer: c.TCP.Writer, WriteBuf: c.TCP.WriteBuf, WriteBufSize: c.TCP.WriteBufSize, Timer: c.Protocol.Timer, TimerSize: c.Protocol.TimerSize, }} // reader r.readers = make([]bytes.Pool, r.options.Reader) for i = 0; i < r.options.Reader; i++ { r.readers[i].Init(r.options.ReadBuf, r.options.ReadBufSize) } // writer r.writers = make([]bytes.Pool, r.options.Writer) for i = 0; i < r.options.Writer; i++ { r.writers[i].Init(r.options.WriteBuf, r.options.WriteBufSize) } // timer r.timers = make([]time.Timer, r.options.Timer) for i = 0; i < r.options.Timer; i++ { r.timers[i].Init(r.options.TimerSize) } return } // Timer get a timer. func (r *Round) Timer(rn int) *time.Timer { return &(r.timers[rn%r.options.Timer]) } // Reader get a reader memory buffer. func (r *Round) Reader(rn int) *bytes.Pool { return &(r.readers[rn%r.options.Reader]) } // Writer get a writer memory buffer pool. func (r *Round) Writer(rn int) *bytes.Pool { return &(r.writers[rn%r.options.Writer]) } ================================================ FILE: internal/comet/server.go ================================================ package comet import ( "context" "math/rand" "time" "github.com/Terry-Mao/goim/api/logic" "github.com/Terry-Mao/goim/internal/comet/conf" log "github.com/golang/glog" "github.com/zhenjl/cityhash" "google.golang.org/grpc" "google.golang.org/grpc/balancer/roundrobin" "google.golang.org/grpc/keepalive" ) const ( minServerHeartbeat = time.Minute * 10 maxServerHeartbeat = time.Minute * 30 // grpc options grpcInitialWindowSize = 1 << 24 grpcInitialConnWindowSize = 1 << 24 grpcMaxSendMsgSize = 1 << 24 grpcMaxCallMsgSize = 1 << 24 grpcKeepAliveTime = time.Second * 10 grpcKeepAliveTimeout = time.Second * 3 grpcBackoffMaxDelay = time.Second * 3 ) func newLogicClient(c *conf.RPCClient) logic.LogicClient { ctx, cancel := context.WithTimeout(context.Background(), time.Duration(c.Dial)) defer cancel() conn, err := grpc.DialContext(ctx, "discovery://default/goim.logic", []grpc.DialOption{ grpc.WithInsecure(), grpc.WithInitialWindowSize(grpcInitialWindowSize), grpc.WithInitialConnWindowSize(grpcInitialConnWindowSize), grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(grpcMaxCallMsgSize)), grpc.WithDefaultCallOptions(grpc.MaxCallSendMsgSize(grpcMaxSendMsgSize)), grpc.WithBackoffMaxDelay(grpcBackoffMaxDelay), grpc.WithKeepaliveParams(keepalive.ClientParameters{ Time: grpcKeepAliveTime, Timeout: grpcKeepAliveTimeout, PermitWithoutStream: true, }), grpc.WithBalancerName(roundrobin.Name), }...) if err != nil { panic(err) } return logic.NewLogicClient(conn) } // Server is comet server. type Server struct { c *conf.Config round *Round // accept round store buckets []*Bucket // subkey bucket bucketIdx uint32 serverID string rpcClient logic.LogicClient } // NewServer returns a new Server. func NewServer(c *conf.Config) *Server { s := &Server{ c: c, round: NewRound(c), rpcClient: newLogicClient(c.RPCClient), } // init bucket s.buckets = make([]*Bucket, c.Bucket.Size) s.bucketIdx = uint32(c.Bucket.Size) for i := 0; i < c.Bucket.Size; i++ { s.buckets[i] = NewBucket(c.Bucket) } s.serverID = c.Env.Host go s.onlineproc() return s } // Buckets return all buckets. func (s *Server) Buckets() []*Bucket { return s.buckets } // Bucket get the bucket by subkey. func (s *Server) Bucket(subKey string) *Bucket { idx := cityhash.CityHash32([]byte(subKey), uint32(len(subKey))) % s.bucketIdx if conf.Conf.Debug { log.Infof("%s hit channel bucket index: %d use cityhash", subKey, idx) } return s.buckets[idx] } // RandServerHearbeat rand server heartbeat. func (s *Server) RandServerHearbeat() time.Duration { return (minServerHeartbeat + time.Duration(rand.Int63n(int64(maxServerHeartbeat-minServerHeartbeat)))) } // Close close the server. func (s *Server) Close() (err error) { return } func (s *Server) onlineproc() { for { var ( allRoomsCount map[string]int32 err error ) roomCount := make(map[string]int32) for _, bucket := range s.buckets { for roomID, count := range bucket.RoomsCount() { roomCount[roomID] += count } } if allRoomsCount, err = s.RenewOnline(context.Background(), s.serverID, roomCount); err != nil { time.Sleep(time.Second) continue } for _, bucket := range s.buckets { bucket.UpRoomsCount(allRoomsCount) } time.Sleep(time.Second * 10) } } ================================================ FILE: internal/comet/server_tcp.go ================================================ package comet import ( "context" "io" "net" "strings" "time" "github.com/Terry-Mao/goim/api/protocol" "github.com/Terry-Mao/goim/internal/comet/conf" "github.com/Terry-Mao/goim/pkg/bufio" "github.com/Terry-Mao/goim/pkg/bytes" xtime "github.com/Terry-Mao/goim/pkg/time" log "github.com/golang/glog" ) const ( maxInt = 1<<31 - 1 ) // InitTCP listen all tcp.bind and start accept connections. func InitTCP(server *Server, addrs []string, accept int) (err error) { var ( bind string listener *net.TCPListener addr *net.TCPAddr ) for _, bind = range addrs { if addr, err = net.ResolveTCPAddr("tcp", bind); err != nil { log.Errorf("net.ResolveTCPAddr(tcp, %s) error(%v)", bind, err) return } if listener, err = net.ListenTCP("tcp", addr); err != nil { log.Errorf("net.ListenTCP(tcp, %s) error(%v)", bind, err) return } log.Infof("start tcp listen: %s", bind) // split N core accept for i := 0; i < accept; i++ { go acceptTCP(server, listener) } } return } // Accept accepts connections on the listener and serves requests // for each incoming connection. Accept blocks; the caller typically // invokes it in a go statement. func acceptTCP(server *Server, lis *net.TCPListener) { var ( conn *net.TCPConn err error r int ) for { if conn, err = lis.AcceptTCP(); err != nil { // if listener close then return log.Errorf("listener.Accept(\"%s\") error(%v)", lis.Addr().String(), err) return } if err = conn.SetKeepAlive(server.c.TCP.KeepAlive); err != nil { log.Errorf("conn.SetKeepAlive() error(%v)", err) return } if err = conn.SetReadBuffer(server.c.TCP.Rcvbuf); err != nil { log.Errorf("conn.SetReadBuffer() error(%v)", err) return } if err = conn.SetWriteBuffer(server.c.TCP.Sndbuf); err != nil { log.Errorf("conn.SetWriteBuffer() error(%v)", err) return } go serveTCP(server, conn, r) if r++; r == maxInt { r = 0 } } } func serveTCP(s *Server, conn *net.TCPConn, r int) { var ( // timer tr = s.round.Timer(r) rp = s.round.Reader(r) wp = s.round.Writer(r) // ip addr lAddr = conn.LocalAddr().String() rAddr = conn.RemoteAddr().String() ) if conf.Conf.Debug { log.Infof("start tcp serve \"%s\" with \"%s\"", lAddr, rAddr) } s.ServeTCP(conn, rp, wp, tr) } // ServeTCP serve a tcp connection. func (s *Server) ServeTCP(conn *net.TCPConn, rp, wp *bytes.Pool, tr *xtime.Timer) { var ( err error rid string accepts []int32 hb time.Duration white bool p *protocol.Proto b *Bucket trd *xtime.TimerData lastHb = time.Now() rb = rp.Get() wb = wp.Get() ch = NewChannel(s.c.Protocol.CliProto, s.c.Protocol.SvrProto) rr = &ch.Reader wr = &ch.Writer ) ch.Reader.ResetBuffer(conn, rb.Bytes()) ch.Writer.ResetBuffer(conn, wb.Bytes()) ctx, cancel := context.WithCancel(context.Background()) defer cancel() // handshake step := 0 trd = tr.Add(time.Duration(s.c.Protocol.HandshakeTimeout), func() { conn.Close() log.Errorf("key: %s remoteIP: %s step: %d tcp handshake timeout", ch.Key, conn.RemoteAddr().String(), step) }) ch.IP, _, _ = net.SplitHostPort(conn.RemoteAddr().String()) // must not setadv, only used in auth step = 1 if p, err = ch.CliProto.Set(); err == nil { if ch.Mid, ch.Key, rid, accepts, hb, err = s.authTCP(ctx, rr, wr, p); err == nil { ch.Watch(accepts...) b = s.Bucket(ch.Key) err = b.Put(rid, ch) if conf.Conf.Debug { log.Infof("tcp connnected key:%s mid:%d proto:%+v", ch.Key, ch.Mid, p) } } } step = 2 if err != nil { conn.Close() rp.Put(rb) wp.Put(wb) tr.Del(trd) log.Errorf("key: %s handshake failed error(%v)", ch.Key, err) return } trd.Key = ch.Key tr.Set(trd, hb) white = whitelist.Contains(ch.Mid) if white { whitelist.Printf("key: %s[%s] auth\n", ch.Key, rid) } step = 3 // hanshake ok start dispatch goroutine go s.dispatchTCP(conn, wr, wp, wb, ch) serverHeartbeat := s.RandServerHearbeat() for { if p, err = ch.CliProto.Set(); err != nil { break } if white { whitelist.Printf("key: %s start read proto\n", ch.Key) } if err = p.ReadTCP(rr); err != nil { break } if white { whitelist.Printf("key: %s read proto:%v\n", ch.Key, p) } if p.Op == protocol.OpHeartbeat { tr.Set(trd, hb) p.Op = protocol.OpHeartbeatReply p.Body = nil // NOTE: send server heartbeat for a long time if now := time.Now(); now.Sub(lastHb) > serverHeartbeat { if err1 := s.Heartbeat(ctx, ch.Mid, ch.Key); err1 == nil { lastHb = now } } if conf.Conf.Debug { log.Infof("tcp heartbeat receive key:%s, mid:%d", ch.Key, ch.Mid) } step++ } else { if err = s.Operate(ctx, p, ch, b); err != nil { break } } if white { whitelist.Printf("key: %s process proto:%v\n", ch.Key, p) } ch.CliProto.SetAdv() ch.Signal() if white { whitelist.Printf("key: %s signal\n", ch.Key) } } if white { whitelist.Printf("key: %s server tcp error(%v)\n", ch.Key, err) } if err != nil && err != io.EOF && !strings.Contains(err.Error(), "closed") { log.Errorf("key: %s server tcp failed error(%v)", ch.Key, err) } b.Del(ch) tr.Del(trd) rp.Put(rb) conn.Close() ch.Close() if err = s.Disconnect(ctx, ch.Mid, ch.Key); err != nil { log.Errorf("key: %s mid: %d operator do disconnect error(%v)", ch.Key, ch.Mid, err) } if white { whitelist.Printf("key: %s mid: %d disconnect error(%v)\n", ch.Key, ch.Mid, err) } if conf.Conf.Debug { log.Infof("tcp disconnected key: %s mid: %d", ch.Key, ch.Mid) } } // dispatch accepts connections on the listener and serves requests // for each incoming connection. dispatch blocks; the caller typically // invokes it in a go statement. func (s *Server) dispatchTCP(conn *net.TCPConn, wr *bufio.Writer, wp *bytes.Pool, wb *bytes.Buffer, ch *Channel) { var ( err error finish bool online int32 white = whitelist.Contains(ch.Mid) ) if conf.Conf.Debug { log.Infof("key: %s start dispatch tcp goroutine", ch.Key) } for { if white { whitelist.Printf("key: %s wait proto ready\n", ch.Key) } var p = ch.Ready() if white { whitelist.Printf("key: %s proto ready\n", ch.Key) } if conf.Conf.Debug { log.Infof("key:%s dispatch msg:%v", ch.Key, *p) } switch p { case protocol.ProtoFinish: if white { whitelist.Printf("key: %s receive proto finish\n", ch.Key) } if conf.Conf.Debug { log.Infof("key: %s wakeup exit dispatch goroutine", ch.Key) } finish = true goto failed case protocol.ProtoReady: // fetch message from svrbox(client send) for { if p, err = ch.CliProto.Get(); err != nil { break } if white { whitelist.Printf("key: %s start write client proto%v\n", ch.Key, p) } if p.Op == protocol.OpHeartbeatReply { if ch.Room != nil { online = ch.Room.OnlineNum() } if err = p.WriteTCPHeart(wr, online); err != nil { goto failed } } else { if err = p.WriteTCP(wr); err != nil { goto failed } } if white { whitelist.Printf("key: %s write client proto%v\n", ch.Key, p) } p.Body = nil // avoid memory leak ch.CliProto.GetAdv() } default: if white { whitelist.Printf("key: %s start write server proto%v\n", ch.Key, p) } // server send if err = p.WriteTCP(wr); err != nil { goto failed } if white { whitelist.Printf("key: %s write server proto%v\n", ch.Key, p) } if conf.Conf.Debug { log.Infof("tcp sent a message key:%s mid:%d proto:%+v", ch.Key, ch.Mid, p) } } if white { whitelist.Printf("key: %s start flush \n", ch.Key) } // only hungry flush response if err = wr.Flush(); err != nil { break } if white { whitelist.Printf("key: %s flush\n", ch.Key) } } failed: if white { whitelist.Printf("key: %s dispatch tcp error(%v)\n", ch.Key, err) } if err != nil { log.Errorf("key: %s dispatch tcp error(%v)", ch.Key, err) } conn.Close() wp.Put(wb) // must ensure all channel message discard, for reader won't blocking Signal for !finish { finish = (ch.Ready() == protocol.ProtoFinish) } if conf.Conf.Debug { log.Infof("key: %s dispatch goroutine exit", ch.Key) } } // auth for goim handshake with client, use rsa & aes. func (s *Server) authTCP(ctx context.Context, rr *bufio.Reader, wr *bufio.Writer, p *protocol.Proto) (mid int64, key, rid string, accepts []int32, hb time.Duration, err error) { for { if err = p.ReadTCP(rr); err != nil { return } if p.Op == protocol.OpAuth { break } else { log.Errorf("tcp request operation(%d) not auth", p.Op) } } if mid, key, rid, accepts, hb, err = s.Connect(ctx, p, ""); err != nil { log.Errorf("authTCP.Connect(key:%v).err(%v)", key, err) return } p.Op = protocol.OpAuthReply p.Body = nil if err = p.WriteTCP(wr); err != nil { log.Errorf("authTCP.WriteTCP(key:%v).err(%v)", key, err) return } err = wr.Flush() return } ================================================ FILE: internal/comet/server_websocket.go ================================================ package comet import ( "context" "crypto/tls" "io" "net" "strings" "time" "github.com/Terry-Mao/goim/api/protocol" "github.com/Terry-Mao/goim/internal/comet/conf" "github.com/Terry-Mao/goim/pkg/bytes" xtime "github.com/Terry-Mao/goim/pkg/time" "github.com/Terry-Mao/goim/pkg/websocket" log "github.com/golang/glog" ) // InitWebsocket listen all tcp.bind and start accept connections. func InitWebsocket(server *Server, addrs []string, accept int) (err error) { var ( bind string listener *net.TCPListener addr *net.TCPAddr ) for _, bind = range addrs { if addr, err = net.ResolveTCPAddr("tcp", bind); err != nil { log.Errorf("net.ResolveTCPAddr(tcp, %s) error(%v)", bind, err) return } if listener, err = net.ListenTCP("tcp", addr); err != nil { log.Errorf("net.ListenTCP(tcp, %s) error(%v)", bind, err) return } log.Infof("start ws listen: %s", bind) // split N core accept for i := 0; i < accept; i++ { go acceptWebsocket(server, listener) } } return } // InitWebsocketWithTLS init websocket with tls. func InitWebsocketWithTLS(server *Server, addrs []string, certFile, privateFile string, accept int) (err error) { var ( bind string listener net.Listener cert tls.Certificate certs []tls.Certificate ) certFiles := strings.Split(certFile, ",") privateFiles := strings.Split(privateFile, ",") for i := range certFiles { cert, err = tls.LoadX509KeyPair(certFiles[i], privateFiles[i]) if err != nil { log.Errorf("Error loading certificate. error(%v)", err) return } certs = append(certs, cert) } tlsCfg := &tls.Config{Certificates: certs} tlsCfg.BuildNameToCertificate() for _, bind = range addrs { if listener, err = tls.Listen("tcp", bind, tlsCfg); err != nil { log.Errorf("net.ListenTCP(tcp, %s) error(%v)", bind, err) return } log.Infof("start wss listen: %s", bind) // split N core accept for i := 0; i < accept; i++ { go acceptWebsocketWithTLS(server, listener) } } return } // Accept accepts connections on the listener and serves requests // for each incoming connection. Accept blocks; the caller typically // invokes it in a go statement. func acceptWebsocket(server *Server, lis *net.TCPListener) { var ( conn *net.TCPConn err error r int ) for { if conn, err = lis.AcceptTCP(); err != nil { // if listener close then return log.Errorf("listener.Accept(%s) error(%v)", lis.Addr().String(), err) return } if err = conn.SetKeepAlive(server.c.TCP.KeepAlive); err != nil { log.Errorf("conn.SetKeepAlive() error(%v)", err) return } if err = conn.SetReadBuffer(server.c.TCP.Rcvbuf); err != nil { log.Errorf("conn.SetReadBuffer() error(%v)", err) return } if err = conn.SetWriteBuffer(server.c.TCP.Sndbuf); err != nil { log.Errorf("conn.SetWriteBuffer() error(%v)", err) return } go serveWebsocket(server, conn, r) if r++; r == maxInt { r = 0 } } } // Accept accepts connections on the listener and serves requests // for each incoming connection. Accept blocks; the caller typically // invokes it in a go statement. func acceptWebsocketWithTLS(server *Server, lis net.Listener) { var ( conn net.Conn err error r int ) for { if conn, err = lis.Accept(); err != nil { // if listener close then return log.Errorf("listener.Accept(\"%s\") error(%v)", lis.Addr().String(), err) return } go serveWebsocket(server, conn, r) if r++; r == maxInt { r = 0 } } } func serveWebsocket(s *Server, conn net.Conn, r int) { var ( // timer tr = s.round.Timer(r) rp = s.round.Reader(r) wp = s.round.Writer(r) ) if conf.Conf.Debug { // ip addr lAddr := conn.LocalAddr().String() rAddr := conn.RemoteAddr().String() log.Infof("start tcp serve \"%s\" with \"%s\"", lAddr, rAddr) } s.ServeWebsocket(conn, rp, wp, tr) } // ServeWebsocket serve a websocket connection. func (s *Server) ServeWebsocket(conn net.Conn, rp, wp *bytes.Pool, tr *xtime.Timer) { var ( err error rid string accepts []int32 hb time.Duration white bool p *protocol.Proto b *Bucket trd *xtime.TimerData lastHB = time.Now() rb = rp.Get() ch = NewChannel(s.c.Protocol.CliProto, s.c.Protocol.SvrProto) rr = &ch.Reader wr = &ch.Writer ws *websocket.Conn // websocket req *websocket.Request ) // reader ch.Reader.ResetBuffer(conn, rb.Bytes()) ctx, cancel := context.WithCancel(context.Background()) defer cancel() // handshake step := 0 trd = tr.Add(time.Duration(s.c.Protocol.HandshakeTimeout), func() { // NOTE: fix close block for tls _ = conn.SetDeadline(time.Now().Add(time.Millisecond * 100)) _ = conn.Close() log.Errorf("key: %s remoteIP: %s step: %d ws handshake timeout", ch.Key, conn.RemoteAddr().String(), step) }) // websocket ch.IP, _, _ = net.SplitHostPort(conn.RemoteAddr().String()) step = 1 if req, err = websocket.ReadRequest(rr); err != nil || req.RequestURI != "/sub" { conn.Close() tr.Del(trd) rp.Put(rb) if err != io.EOF { log.Errorf("http.ReadRequest(rr) error(%v)", err) } return } // writer wb := wp.Get() ch.Writer.ResetBuffer(conn, wb.Bytes()) step = 2 if ws, err = websocket.Upgrade(conn, rr, wr, req); err != nil { conn.Close() tr.Del(trd) rp.Put(rb) wp.Put(wb) if err != io.EOF { log.Errorf("websocket.NewServerConn error(%v)", err) } return } // must not setadv, only used in auth step = 3 if p, err = ch.CliProto.Set(); err == nil { if ch.Mid, ch.Key, rid, accepts, hb, err = s.authWebsocket(ctx, ws, p, req.Header.Get("Cookie")); err == nil { ch.Watch(accepts...) b = s.Bucket(ch.Key) err = b.Put(rid, ch) if conf.Conf.Debug { log.Infof("websocket connected key:%s mid:%d proto:%+v", ch.Key, ch.Mid, p) } } } step = 4 if err != nil { ws.Close() rp.Put(rb) wp.Put(wb) tr.Del(trd) if err != io.EOF && err != websocket.ErrMessageClose { log.Errorf("key: %s remoteIP: %s step: %d ws handshake failed error(%v)", ch.Key, conn.RemoteAddr().String(), step, err) } return } trd.Key = ch.Key tr.Set(trd, hb) white = whitelist.Contains(ch.Mid) if white { whitelist.Printf("key: %s[%s] auth\n", ch.Key, rid) } // handshake ok start dispatch goroutine step = 5 go s.dispatchWebsocket(ws, wp, wb, ch) serverHeartbeat := s.RandServerHearbeat() for { if p, err = ch.CliProto.Set(); err != nil { break } if white { whitelist.Printf("key: %s start read proto\n", ch.Key) } if err = p.ReadWebsocket(ws); err != nil { break } if white { whitelist.Printf("key: %s read proto:%v\n", ch.Key, p) } if p.Op == protocol.OpHeartbeat { tr.Set(trd, hb) p.Op = protocol.OpHeartbeatReply p.Body = nil // NOTE: send server heartbeat for a long time if now := time.Now(); now.Sub(lastHB) > serverHeartbeat { if err1 := s.Heartbeat(ctx, ch.Mid, ch.Key); err1 == nil { lastHB = now } } if conf.Conf.Debug { log.Infof("websocket heartbeat receive key:%s, mid:%d", ch.Key, ch.Mid) } step++ } else { if err = s.Operate(ctx, p, ch, b); err != nil { break } } if white { whitelist.Printf("key: %s process proto:%v\n", ch.Key, p) } ch.CliProto.SetAdv() ch.Signal() if white { whitelist.Printf("key: %s signal\n", ch.Key) } } if white { whitelist.Printf("key: %s server tcp error(%v)\n", ch.Key, err) } if err != nil && err != io.EOF && err != websocket.ErrMessageClose && !strings.Contains(err.Error(), "closed") { log.Errorf("key: %s server ws failed error(%v)", ch.Key, err) } b.Del(ch) tr.Del(trd) ws.Close() ch.Close() rp.Put(rb) if err = s.Disconnect(ctx, ch.Mid, ch.Key); err != nil { log.Errorf("key: %s operator do disconnect error(%v)", ch.Key, err) } if white { whitelist.Printf("key: %s disconnect error(%v)\n", ch.Key, err) } if conf.Conf.Debug { log.Infof("websocket disconnected key: %s mid:%d", ch.Key, ch.Mid) } } // dispatch accepts connections on the listener and serves requests // for each incoming connection. dispatch blocks; the caller typically // invokes it in a go statement. func (s *Server) dispatchWebsocket(ws *websocket.Conn, wp *bytes.Pool, wb *bytes.Buffer, ch *Channel) { var ( err error finish bool online int32 white = whitelist.Contains(ch.Mid) ) if conf.Conf.Debug { log.Infof("key: %s start dispatch tcp goroutine", ch.Key) } for { if white { whitelist.Printf("key: %s wait proto ready\n", ch.Key) } var p = ch.Ready() if white { whitelist.Printf("key: %s proto ready\n", ch.Key) } if conf.Conf.Debug { log.Infof("key:%s dispatch msg:%s", ch.Key, p.Body) } switch p { case protocol.ProtoFinish: if white { whitelist.Printf("key: %s receive proto finish\n", ch.Key) } if conf.Conf.Debug { log.Infof("key: %s wakeup exit dispatch goroutine", ch.Key) } finish = true goto failed case protocol.ProtoReady: // fetch message from svrbox(client send) for { if p, err = ch.CliProto.Get(); err != nil { break } if white { whitelist.Printf("key: %s start write client proto%v\n", ch.Key, p) } if p.Op == protocol.OpHeartbeatReply { if ch.Room != nil { online = ch.Room.OnlineNum() } if err = p.WriteWebsocketHeart(ws, online); err != nil { goto failed } } else { if err = p.WriteWebsocket(ws); err != nil { goto failed } } if white { whitelist.Printf("key: %s write client proto%v\n", ch.Key, p) } p.Body = nil // avoid memory leak ch.CliProto.GetAdv() } default: if white { whitelist.Printf("key: %s start write server proto%v\n", ch.Key, p) } if err = p.WriteWebsocket(ws); err != nil { goto failed } if white { whitelist.Printf("key: %s write server proto%v\n", ch.Key, p) } if conf.Conf.Debug { log.Infof("websocket sent a message key:%s mid:%d proto:%+v", ch.Key, ch.Mid, p) } } if white { whitelist.Printf("key: %s start flush \n", ch.Key) } // only hungry flush response if err = ws.Flush(); err != nil { break } if white { whitelist.Printf("key: %s flush\n", ch.Key) } } failed: if white { whitelist.Printf("key: %s dispatch tcp error(%v)\n", ch.Key, err) } if err != nil && err != io.EOF && err != websocket.ErrMessageClose { log.Errorf("key: %s dispatch ws error(%v)", ch.Key, err) } ws.Close() wp.Put(wb) // must ensure all channel message discard, for reader won't blocking Signal for !finish { finish = (ch.Ready() == protocol.ProtoFinish) } if conf.Conf.Debug { log.Infof("key: %s dispatch goroutine exit", ch.Key) } } // auth for goim handshake with client, use rsa & aes. func (s *Server) authWebsocket(ctx context.Context, ws *websocket.Conn, p *protocol.Proto, cookie string) (mid int64, key, rid string, accepts []int32, hb time.Duration, err error) { for { if err = p.ReadWebsocket(ws); err != nil { return } if p.Op == protocol.OpAuth { break } else { log.Errorf("ws request operation(%d) not auth", p.Op) } } if mid, key, rid, accepts, hb, err = s.Connect(ctx, p, cookie); err != nil { return } p.Op = protocol.OpAuthReply p.Body = nil if err = p.WriteWebsocket(ws); err != nil { return } err = ws.Flush() return } ================================================ FILE: internal/comet/whitelist.go ================================================ package comet import ( "log" "os" "github.com/Terry-Mao/goim/internal/comet/conf" ) var whitelist *Whitelist // Whitelist . type Whitelist struct { log *log.Logger list map[int64]struct{} // whitelist for debug } // InitWhitelist a whitelist struct. func InitWhitelist(c *conf.Whitelist) (err error) { var ( mid int64 f *os.File ) if f, err = os.OpenFile(c.WhiteLog, os.O_RDWR|os.O_APPEND|os.O_CREATE, 0644); err == nil { whitelist = new(Whitelist) whitelist.log = log.New(f, "", log.LstdFlags) whitelist.list = make(map[int64]struct{}) for _, mid = range c.Whitelist { whitelist.list[mid] = struct{}{} } } return } // Contains whitelist contains a mid or not. func (w *Whitelist) Contains(mid int64) (ok bool) { if mid > 0 { _, ok = w.list[mid] } return } // Printf calls l.Output to print to the logger. func (w *Whitelist) Printf(format string, v ...interface{}) { w.log.Printf(format, v...) } ================================================ FILE: internal/job/comet.go ================================================ package job import ( "context" "fmt" "net/url" "sync/atomic" "time" "github.com/Terry-Mao/goim/api/comet" "github.com/Terry-Mao/goim/internal/job/conf" "github.com/bilibili/discovery/naming" log "github.com/golang/glog" "google.golang.org/grpc" "google.golang.org/grpc/keepalive" ) var ( // grpc options grpcKeepAliveTime = time.Duration(10) * time.Second grpcKeepAliveTimeout = time.Duration(3) * time.Second grpcBackoffMaxDelay = time.Duration(3) * time.Second grpcMaxSendMsgSize = 1 << 24 grpcMaxCallMsgSize = 1 << 24 ) const ( // grpc options grpcInitialWindowSize = 1 << 24 grpcInitialConnWindowSize = 1 << 24 ) func newCometClient(addr string) (comet.CometClient, error) { ctx, cancel := context.WithTimeout(context.Background(), time.Duration(time.Second)) defer cancel() conn, err := grpc.DialContext(ctx, addr, []grpc.DialOption{ grpc.WithInsecure(), grpc.WithInitialWindowSize(grpcInitialWindowSize), grpc.WithInitialConnWindowSize(grpcInitialConnWindowSize), grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(grpcMaxCallMsgSize)), grpc.WithDefaultCallOptions(grpc.MaxCallSendMsgSize(grpcMaxSendMsgSize)), grpc.WithBackoffMaxDelay(grpcBackoffMaxDelay), grpc.WithKeepaliveParams(keepalive.ClientParameters{ Time: grpcKeepAliveTime, Timeout: grpcKeepAliveTimeout, PermitWithoutStream: true, }), }..., ) if err != nil { return nil, err } return comet.NewCometClient(conn), err } // Comet is a comet. type Comet struct { serverID string client comet.CometClient pushChan []chan *comet.PushMsgReq roomChan []chan *comet.BroadcastRoomReq broadcastChan chan *comet.BroadcastReq pushChanNum uint64 roomChanNum uint64 routineSize uint64 ctx context.Context cancel context.CancelFunc } // NewComet new a comet. func NewComet(in *naming.Instance, c *conf.Comet) (*Comet, error) { cmt := &Comet{ serverID: in.Hostname, pushChan: make([]chan *comet.PushMsgReq, c.RoutineSize), roomChan: make([]chan *comet.BroadcastRoomReq, c.RoutineSize), broadcastChan: make(chan *comet.BroadcastReq, c.RoutineSize), routineSize: uint64(c.RoutineSize), } var grpcAddr string for _, addrs := range in.Addrs { u, err := url.Parse(addrs) if err == nil && u.Scheme == "grpc" { grpcAddr = u.Host } } if grpcAddr == "" { return nil, fmt.Errorf("invalid grpc address:%v", in.Addrs) } var err error if cmt.client, err = newCometClient(grpcAddr); err != nil { return nil, err } cmt.ctx, cmt.cancel = context.WithCancel(context.Background()) for i := 0; i < c.RoutineSize; i++ { cmt.pushChan[i] = make(chan *comet.PushMsgReq, c.RoutineChan) cmt.roomChan[i] = make(chan *comet.BroadcastRoomReq, c.RoutineChan) go cmt.process(cmt.pushChan[i], cmt.roomChan[i], cmt.broadcastChan) } return cmt, nil } // Push push a user message. func (c *Comet) Push(arg *comet.PushMsgReq) (err error) { idx := atomic.AddUint64(&c.pushChanNum, 1) % c.routineSize c.pushChan[idx] <- arg return } // BroadcastRoom broadcast a room message. func (c *Comet) BroadcastRoom(arg *comet.BroadcastRoomReq) (err error) { idx := atomic.AddUint64(&c.roomChanNum, 1) % c.routineSize c.roomChan[idx] <- arg return } // Broadcast broadcast a message. func (c *Comet) Broadcast(arg *comet.BroadcastReq) (err error) { c.broadcastChan <- arg return } func (c *Comet) process(pushChan chan *comet.PushMsgReq, roomChan chan *comet.BroadcastRoomReq, broadcastChan chan *comet.BroadcastReq) { for { select { case broadcastArg := <-broadcastChan: _, err := c.client.Broadcast(context.Background(), &comet.BroadcastReq{ Proto: broadcastArg.Proto, ProtoOp: broadcastArg.ProtoOp, Speed: broadcastArg.Speed, }) if err != nil { log.Errorf("c.client.Broadcast(%s, reply) serverId:%s error(%v)", broadcastArg, c.serverID, err) } case roomArg := <-roomChan: _, err := c.client.BroadcastRoom(context.Background(), &comet.BroadcastRoomReq{ RoomID: roomArg.RoomID, Proto: roomArg.Proto, }) if err != nil { log.Errorf("c.client.BroadcastRoom(%s, reply) serverId:%s error(%v)", roomArg, c.serverID, err) } case pushArg := <-pushChan: _, err := c.client.PushMsg(context.Background(), &comet.PushMsgReq{ Keys: pushArg.Keys, Proto: pushArg.Proto, ProtoOp: pushArg.ProtoOp, }) if err != nil { log.Errorf("c.client.PushMsg(%s, reply) serverId:%s error(%v)", pushArg, c.serverID, err) } case <-c.ctx.Done(): return } } } // Close close the resources. func (c *Comet) Close() (err error) { finish := make(chan bool) go func() { for { n := len(c.broadcastChan) for _, ch := range c.pushChan { n += len(ch) } for _, ch := range c.roomChan { n += len(ch) } if n == 0 { finish <- true return } time.Sleep(time.Second) } }() select { case <-finish: log.Info("close comet finish") case <-time.After(5 * time.Second): err = fmt.Errorf("close comet(server:%s push:%d room:%d broadcast:%d) timeout", c.serverID, len(c.pushChan), len(c.roomChan), len(c.broadcastChan)) } c.cancel() return } ================================================ FILE: internal/job/conf/conf.go ================================================ package conf import ( "flag" "os" "time" "github.com/bilibili/discovery/naming" "github.com/BurntSushi/toml" xtime "github.com/Terry-Mao/goim/pkg/time" ) var ( confPath string region string zone string deployEnv string host string // Conf config Conf *Config ) func init() { var ( defHost, _ = os.Hostname() ) flag.StringVar(&confPath, "conf", "job-example.toml", "default config path") flag.StringVar(®ion, "region", os.Getenv("REGION"), "avaliable region. or use REGION env variable, value: sh etc.") flag.StringVar(&zone, "zone", os.Getenv("ZONE"), "avaliable zone. or use ZONE env variable, value: sh001/sh002 etc.") flag.StringVar(&deployEnv, "deploy.env", os.Getenv("DEPLOY_ENV"), "deploy env. or use DEPLOY_ENV env variable, value: dev/fat1/uat/pre/prod etc.") flag.StringVar(&host, "host", defHost, "machine hostname. or use default machine hostname.") } // Init init config. func Init() (err error) { Conf = Default() _, err = toml.DecodeFile(confPath, &Conf) return } // Default new a config with specified defualt value. func Default() *Config { return &Config{ Env: &Env{Region: region, Zone: zone, DeployEnv: deployEnv, Host: host}, Discovery: &naming.Config{Region: region, Zone: zone, Env: deployEnv, Host: host}, Comet: &Comet{RoutineChan: 1024, RoutineSize: 32}, Room: &Room{ Batch: 20, Signal: xtime.Duration(time.Second), Idle: xtime.Duration(time.Minute * 15), }, } } // Config is job config. type Config struct { Env *Env Kafka *Kafka Discovery *naming.Config Comet *Comet Room *Room } // Room is room config. type Room struct { Batch int Signal xtime.Duration Idle xtime.Duration } // Comet is comet config. type Comet struct { RoutineChan int RoutineSize int } // Kafka is kafka config. type Kafka struct { Topic string Group string Brokers []string } // Env is env config. type Env struct { Region string Zone string DeployEnv string Host string } ================================================ FILE: internal/job/job.go ================================================ package job import ( "context" "fmt" "sync" "time" pb "github.com/Terry-Mao/goim/api/logic" "github.com/Terry-Mao/goim/internal/job/conf" "github.com/bilibili/discovery/naming" "github.com/golang/protobuf/proto" cluster "github.com/bsm/sarama-cluster" log "github.com/golang/glog" ) // Job is push job. type Job struct { c *conf.Config consumer *cluster.Consumer cometServers map[string]*Comet rooms map[string]*Room roomsMutex sync.RWMutex } // New new a push job. func New(c *conf.Config) *Job { j := &Job{ c: c, consumer: newKafkaSub(c.Kafka), rooms: make(map[string]*Room), } j.watchComet(c.Discovery) return j } func newKafkaSub(c *conf.Kafka) *cluster.Consumer { config := cluster.NewConfig() config.Consumer.Return.Errors = true config.Group.Return.Notifications = true consumer, err := cluster.NewConsumer(c.Brokers, c.Group, []string{c.Topic}, config) if err != nil { panic(err) } return consumer } // Close close resounces. func (j *Job) Close() error { if j.consumer != nil { return j.consumer.Close() } return nil } // Consume messages, watch signals func (j *Job) Consume() { for { select { case err := <-j.consumer.Errors(): log.Errorf("consumer error(%v)", err) case n := <-j.consumer.Notifications(): log.Infof("consumer rebalanced(%v)", n) case msg, ok := <-j.consumer.Messages(): if !ok { return } j.consumer.MarkOffset(msg, "") // process push message pushMsg := new(pb.PushMsg) if err := proto.Unmarshal(msg.Value, pushMsg); err != nil { log.Errorf("proto.Unmarshal(%v) error(%v)", msg, err) continue } if err := j.push(context.Background(), pushMsg); err != nil { log.Errorf("j.push(%v) error(%v)", pushMsg, err) } log.Infof("consume: %s/%d/%d\t%s\t%+v", msg.Topic, msg.Partition, msg.Offset, msg.Key, pushMsg) } } } func (j *Job) watchComet(c *naming.Config) { dis := naming.New(c) resolver := dis.Build("goim.comet") event := resolver.Watch() select { case _, ok := <-event: if !ok { panic("watchComet init failed") } if ins, ok := resolver.Fetch(); ok { if err := j.newAddress(ins.Instances); err != nil { panic(err) } log.Infof("watchComet init newAddress:%+v", ins) } case <-time.After(10 * time.Second): log.Error("watchComet init instances timeout") } go func() { for { if _, ok := <-event; !ok { log.Info("watchComet exit") return } ins, ok := resolver.Fetch() if ok { if err := j.newAddress(ins.Instances); err != nil { log.Errorf("watchComet newAddress(%+v) error(%+v)", ins, err) continue } log.Infof("watchComet change newAddress:%+v", ins) } } }() } func (j *Job) newAddress(insMap map[string][]*naming.Instance) error { ins := insMap[j.c.Env.Zone] if len(ins) == 0 { return fmt.Errorf("watchComet instance is empty") } comets := map[string]*Comet{} for _, in := range ins { if old, ok := j.cometServers[in.Hostname]; ok { comets[in.Hostname] = old continue } c, err := NewComet(in, j.c.Comet) if err != nil { log.Errorf("watchComet NewComet(%+v) error(%v)", in, err) return err } comets[in.Hostname] = c log.Infof("watchComet AddComet grpc:%+v", in) } for key, old := range j.cometServers { if _, ok := comets[key]; !ok { old.cancel() log.Infof("watchComet DelComet:%s", key) } } j.cometServers = comets return nil } ================================================ FILE: internal/job/push.go ================================================ package job import ( "context" "fmt" "github.com/Terry-Mao/goim/api/comet" pb "github.com/Terry-Mao/goim/api/logic" "github.com/Terry-Mao/goim/api/protocol" "github.com/Terry-Mao/goim/pkg/bytes" log "github.com/golang/glog" ) func (j *Job) push(ctx context.Context, pushMsg *pb.PushMsg) (err error) { switch pushMsg.Type { case pb.PushMsg_PUSH: err = j.pushKeys(pushMsg.Operation, pushMsg.Server, pushMsg.Keys, pushMsg.Msg) case pb.PushMsg_ROOM: err = j.getRoom(pushMsg.Room).Push(pushMsg.Operation, pushMsg.Msg) case pb.PushMsg_BROADCAST: err = j.broadcast(pushMsg.Operation, pushMsg.Msg, pushMsg.Speed) default: err = fmt.Errorf("no match push type: %s", pushMsg.Type) } return } // pushKeys push a message to a batch of subkeys. func (j *Job) pushKeys(operation int32, serverID string, subKeys []string, body []byte) (err error) { buf := bytes.NewWriterSize(len(body) + 64) p := &protocol.Proto{ Ver: 1, Op: operation, Body: body, } p.WriteTo(buf) p.Body = buf.Buffer() p.Op = protocol.OpRaw var args = comet.PushMsgReq{ Keys: subKeys, ProtoOp: operation, Proto: p, } if c, ok := j.cometServers[serverID]; ok { if err = c.Push(&args); err != nil { log.Errorf("c.Push(%v) serverID:%s error(%v)", args, serverID, err) } log.Infof("pushKey:%s comets:%d", serverID, len(j.cometServers)) } return } // broadcast broadcast a message to all. func (j *Job) broadcast(operation int32, body []byte, speed int32) (err error) { buf := bytes.NewWriterSize(len(body) + 64) p := &protocol.Proto{ Ver: 1, Op: operation, Body: body, } p.WriteTo(buf) p.Body = buf.Buffer() p.Op = protocol.OpRaw comets := j.cometServers speed /= int32(len(comets)) var args = comet.BroadcastReq{ ProtoOp: operation, Proto: p, Speed: speed, } for serverID, c := range comets { if err = c.Broadcast(&args); err != nil { log.Errorf("c.Broadcast(%v) serverID:%s error(%v)", args, serverID, err) } } log.Infof("broadcast comets:%d", len(comets)) return } // broadcastRoomRawBytes broadcast aggregation messages to room. func (j *Job) broadcastRoomRawBytes(roomID string, body []byte) (err error) { args := comet.BroadcastRoomReq{ RoomID: roomID, Proto: &protocol.Proto{ Ver: 1, Op: protocol.OpRaw, Body: body, }, } comets := j.cometServers for serverID, c := range comets { if err = c.BroadcastRoom(&args); err != nil { log.Errorf("c.BroadcastRoom(%v) roomID:%s serverID:%s error(%v)", args, roomID, serverID, err) } } log.Infof("broadcastRoom comets:%d", len(comets)) return } ================================================ FILE: internal/job/room.go ================================================ package job import ( "errors" "time" "github.com/Terry-Mao/goim/api/protocol" "github.com/Terry-Mao/goim/internal/job/conf" "github.com/Terry-Mao/goim/pkg/bytes" log "github.com/golang/glog" ) var ( // ErrComet commet error. ErrComet = errors.New("comet rpc is not available") // ErrCometFull comet chan full. ErrCometFull = errors.New("comet proto chan full") // ErrRoomFull room chan full. ErrRoomFull = errors.New("room proto chan full") roomReadyProto = new(protocol.Proto) ) // Room room. type Room struct { c *conf.Room job *Job id string proto chan *protocol.Proto } // NewRoom new a room struct, store channel room info. func NewRoom(job *Job, id string, c *conf.Room) (r *Room) { r = &Room{ c: c, id: id, job: job, proto: make(chan *protocol.Proto, c.Batch*2), } go r.pushproc(c.Batch, time.Duration(c.Signal)) return } // Push push msg to the room, if chan full discard it. func (r *Room) Push(op int32, msg []byte) (err error) { var p = &protocol.Proto{ Ver: 1, Op: op, Body: msg, } select { case r.proto <- p: default: err = ErrRoomFull } return } // pushproc merge proto and push msgs in batch. func (r *Room) pushproc(batch int, sigTime time.Duration) { var ( n int last time.Time p *protocol.Proto buf = bytes.NewWriterSize(int(protocol.MaxBodySize)) ) log.Infof("start room:%s goroutine", r.id) td := time.AfterFunc(sigTime, func() { select { case r.proto <- roomReadyProto: default: } }) defer td.Stop() for { if p = <-r.proto; p == nil { break // exit } else if p != roomReadyProto { // merge buffer ignore error, always nil p.WriteTo(buf) if n++; n == 1 { last = time.Now() td.Reset(sigTime) continue } else if n < batch { if sigTime > time.Since(last) { continue } } } else { if n == 0 { break } } _ = r.job.broadcastRoomRawBytes(r.id, buf.Buffer()) // TODO use reset buffer // after push to room channel, renew a buffer, let old buffer gc buf = bytes.NewWriterSize(buf.Size()) n = 0 if r.c.Idle != 0 { td.Reset(time.Duration(r.c.Idle)) } else { td.Reset(time.Minute) } } r.job.delRoom(r.id) log.Infof("room:%s goroutine exit", r.id) } func (j *Job) delRoom(roomID string) { j.roomsMutex.Lock() delete(j.rooms, roomID) j.roomsMutex.Unlock() } func (j *Job) getRoom(roomID string) *Room { j.roomsMutex.RLock() room, ok := j.rooms[roomID] j.roomsMutex.RUnlock() if !ok { j.roomsMutex.Lock() if room, ok = j.rooms[roomID]; !ok { room = NewRoom(j, roomID, j.c.Room) j.rooms[roomID] = room } j.roomsMutex.Unlock() log.Infof("new a room:%s active:%d", roomID, len(j.rooms)) } return room } ================================================ FILE: internal/logic/balancer.go ================================================ package logic import ( "fmt" "math" "sort" "strconv" "strings" "sync" "github.com/bilibili/discovery/naming" "github.com/Terry-Mao/goim/internal/logic/model" log "github.com/golang/glog" ) const ( _minWeight = 1 _maxWeight = 1 << 20 _maxNodes = 5 ) type weightedNode struct { region string hostname string addrs []string fixedWeight int64 currentWeight int64 currentConns int64 updated int64 } func (w *weightedNode) String() string { return fmt.Sprintf("region:%s fixedWeight:%d, currentWeight:%d, currentConns:%d", w.region, w.fixedWeight, w.currentWeight, w.currentConns) } func (w *weightedNode) chosen() { w.currentConns++ } func (w *weightedNode) reset() { w.currentWeight = 0 } func (w *weightedNode) calculateWeight(totalWeight, totalConns int64, gainWeight float64) { fixedWeight := float64(w.fixedWeight) * gainWeight totalWeight += int64(fixedWeight) - w.fixedWeight if totalConns > 0 { weightRatio := fixedWeight / float64(totalWeight) var connRatio float64 if totalConns != 0 { connRatio = float64(w.currentConns) / float64(totalConns) * 0.5 } diff := weightRatio - connRatio multiple := diff * float64(totalConns) floor := math.Floor(multiple) if floor-multiple >= -0.5 { w.currentWeight = int64(fixedWeight + floor) } else { w.currentWeight = int64(fixedWeight + math.Ceil(multiple)) } if diff < 0 { // we always return the max from minWeight and calculated Current weight if _minWeight > w.currentWeight { w.currentWeight = _minWeight } } else { // we always return the min from maxWeight and calculated Current weight if _maxWeight < w.currentWeight { w.currentWeight = _maxWeight } } } else { w.reset() } } // LoadBalancer load balancer. type LoadBalancer struct { totalConns int64 totalWeight int64 nodes map[string]*weightedNode nodesMutex sync.Mutex } // NewLoadBalancer new a load balancer. func NewLoadBalancer() *LoadBalancer { lb := &LoadBalancer{ nodes: make(map[string]*weightedNode), } return lb } // Size return node size. func (lb *LoadBalancer) Size() int { return len(lb.nodes) } func (lb *LoadBalancer) weightedNodes(region string, regionWeight float64) (nodes []*weightedNode) { for _, n := range lb.nodes { var gainWeight = float64(1.0) if n.region == region { gainWeight *= regionWeight } n.calculateWeight(lb.totalWeight, lb.totalConns, gainWeight) nodes = append(nodes, n) } sort.Slice(nodes, func(i, j int) bool { return nodes[i].currentWeight > nodes[j].currentWeight }) if len(nodes) > 0 { nodes[0].chosen() lb.totalConns++ } return } // NodeAddrs return node addrs. func (lb *LoadBalancer) NodeAddrs(region, domain string, regionWeight float64) (domains, addrs []string) { lb.nodesMutex.Lock() nodes := lb.weightedNodes(region, regionWeight) lb.nodesMutex.Unlock() for i, n := range nodes { if i == _maxNodes { break } domains = append(domains, n.hostname+domain) addrs = append(addrs, n.addrs...) } return } // Update update server nodes. func (lb *LoadBalancer) Update(ins []*naming.Instance) { var ( totalConns int64 totalWeight int64 nodes = make(map[string]*weightedNode, len(ins)) ) if len(ins) == 0 || float32(len(ins))/float32(len(lb.nodes)) < 0.5 { log.Errorf("load balancer update src:%d target:%d less than half", len(lb.nodes), len(ins)) return } lb.nodesMutex.Lock() for _, in := range ins { if old, ok := lb.nodes[in.Hostname]; ok && old.updated == in.LastTs { nodes[in.Hostname] = old totalConns += old.currentConns totalWeight += old.fixedWeight } else { meta := in.Metadata weight, err := strconv.ParseInt(meta[model.MetaWeight], 10, 32) if err != nil { log.Errorf("instance(%+v) strconv.ParseInt(weight:%s) error(%v)", in, meta[model.MetaWeight], err) continue } conns, err := strconv.ParseInt(meta[model.MetaConnCount], 10, 32) if err != nil { log.Errorf("instance(%+v) strconv.ParseInt(conns:%s) error(%v)", in, meta[model.MetaConnCount], err) continue } nodes[in.Hostname] = &weightedNode{ region: in.Region, hostname: in.Hostname, fixedWeight: weight, currentConns: conns, addrs: strings.Split(meta[model.MetaAddrs], ","), updated: in.LastTs, } totalConns += conns totalWeight += weight } } lb.nodes = nodes lb.totalConns = totalConns lb.totalWeight = totalWeight lb.nodesMutex.Unlock() } ================================================ FILE: internal/logic/balancer_test.go ================================================ package logic import ( "sort" "testing" "github.com/bilibili/discovery/naming" "github.com/Terry-Mao/goim/internal/logic/model" ) func TestWeightedNode(t *testing.T) { nodes := []*weightedNode{ &weightedNode{fixedWeight: 1, currentWeight: 1, currentConns: 1000000}, &weightedNode{fixedWeight: 2, currentWeight: 1, currentConns: 1000000}, &weightedNode{fixedWeight: 3, currentWeight: 1, currentConns: 1000000}, } for i := 0; i < 100; i++ { for _, n := range nodes { n.calculateWeight(6, nodes[0].currentConns+nodes[1].currentConns+nodes[2].currentConns, 1.0) } sort.Slice(nodes, func(i, j int) bool { return nodes[i].currentWeight > nodes[j].currentWeight }) nodes[0].chosen() } ft := float64(nodes[0].fixedWeight + nodes[1].fixedWeight + nodes[2].fixedWeight) ct := float64(nodes[0].currentConns + nodes[1].currentConns + nodes[2].currentConns) for _, n := range nodes { t.Logf("match ratio %d:%d", int(float64(n.fixedWeight)/ft*100*0.6), int(float64(n.currentConns)/ct*100)) } } func TestLoadBalancer(t *testing.T) { ss := []*naming.Instance{ &naming.Instance{ Region: "bj", Hostname: "01", Metadata: map[string]string{ model.MetaWeight: "10", model.MetaConnCount: "240590", model.MetaIPCount: "10", model.MetaAddrs: "ip_bj", }, }, &naming.Instance{ Region: "sh", Hostname: "02", Metadata: map[string]string{ model.MetaWeight: "10", model.MetaConnCount: "375420", model.MetaIPCount: "10", model.MetaAddrs: "ip_sh", }, }, &naming.Instance{ Region: "gz", Hostname: "03", Metadata: map[string]string{ model.MetaWeight: "10", model.MetaConnCount: "293430", model.MetaIPCount: "10", model.MetaAddrs: "ip_gz", }, }, } lb := NewLoadBalancer() lb.Update(ss) for i := 0; i < 5; i++ { t.Log(lb.NodeAddrs("sh", ".test", 1.6)) } } ================================================ FILE: internal/logic/conf/conf.go ================================================ package conf import ( "flag" "os" "strconv" "time" "github.com/bilibili/discovery/naming" xtime "github.com/Terry-Mao/goim/pkg/time" "github.com/BurntSushi/toml" ) var ( confPath string region string zone string deployEnv string host string weight int64 // Conf config Conf *Config ) func init() { var ( defHost, _ = os.Hostname() defWeight, _ = strconv.ParseInt(os.Getenv("WEIGHT"), 10, 32) ) flag.StringVar(&confPath, "conf", "logic-example.toml", "default config path") flag.StringVar(®ion, "region", os.Getenv("REGION"), "avaliable region. or use REGION env variable, value: sh etc.") flag.StringVar(&zone, "zone", os.Getenv("ZONE"), "avaliable zone. or use ZONE env variable, value: sh001/sh002 etc.") flag.StringVar(&deployEnv, "deploy.env", os.Getenv("DEPLOY_ENV"), "deploy env. or use DEPLOY_ENV env variable, value: dev/fat1/uat/pre/prod etc.") flag.StringVar(&host, "host", defHost, "machine hostname. or use default machine hostname.") flag.Int64Var(&weight, "weight", defWeight, "load balancing weight, or use WEIGHT env variable, value: 10 etc.") } // Init init config. func Init() (err error) { Conf = Default() _, err = toml.DecodeFile(confPath, &Conf) return } // Default new a config with specified defualt value. func Default() *Config { return &Config{ Env: &Env{Region: region, Zone: zone, DeployEnv: deployEnv, Host: host, Weight: weight}, Discovery: &naming.Config{Region: region, Zone: zone, Env: deployEnv, Host: host}, HTTPServer: &HTTPServer{ Network: "tcp", Addr: "3111", ReadTimeout: xtime.Duration(time.Second), WriteTimeout: xtime.Duration(time.Second), }, RPCClient: &RPCClient{Dial: xtime.Duration(time.Second), Timeout: xtime.Duration(time.Second)}, RPCServer: &RPCServer{ Network: "tcp", Addr: "3119", Timeout: xtime.Duration(time.Second), IdleTimeout: xtime.Duration(time.Second * 60), MaxLifeTime: xtime.Duration(time.Hour * 2), ForceCloseWait: xtime.Duration(time.Second * 20), KeepAliveInterval: xtime.Duration(time.Second * 60), KeepAliveTimeout: xtime.Duration(time.Second * 20), }, Backoff: &Backoff{MaxDelay: 300, BaseDelay: 3, Factor: 1.8, Jitter: 1.3}, } } // Config config. type Config struct { Env *Env Discovery *naming.Config RPCClient *RPCClient RPCServer *RPCServer HTTPServer *HTTPServer Kafka *Kafka Redis *Redis Node *Node Backoff *Backoff Regions map[string][]string } // Env is env config. type Env struct { Region string Zone string DeployEnv string Host string Weight int64 } // Node node config. type Node struct { DefaultDomain string HostDomain string TCPPort int WSPort int WSSPort int HeartbeatMax int Heartbeat xtime.Duration RegionWeight float64 } // Backoff backoff. type Backoff struct { MaxDelay int32 BaseDelay int32 Factor float32 Jitter float32 } // Redis . type Redis struct { Network string Addr string Auth string Active int Idle int DialTimeout xtime.Duration ReadTimeout xtime.Duration WriteTimeout xtime.Duration IdleTimeout xtime.Duration Expire xtime.Duration } // Kafka . type Kafka struct { Topic string Brokers []string } // RPCClient is RPC client config. type RPCClient struct { Dial xtime.Duration Timeout xtime.Duration } // RPCServer is RPC server config. type RPCServer struct { Network string Addr string Timeout xtime.Duration IdleTimeout xtime.Duration MaxLifeTime xtime.Duration ForceCloseWait xtime.Duration KeepAliveInterval xtime.Duration KeepAliveTimeout xtime.Duration } // HTTPServer is http server config. type HTTPServer struct { Network string Addr string ReadTimeout xtime.Duration WriteTimeout xtime.Duration } ================================================ FILE: internal/logic/conn.go ================================================ package logic import ( "context" "encoding/json" "time" "github.com/Terry-Mao/goim/api/protocol" "github.com/Terry-Mao/goim/internal/logic/model" log "github.com/golang/glog" "github.com/google/uuid" ) // Connect connected a conn. func (l *Logic) Connect(c context.Context, server, cookie string, token []byte) (mid int64, key, roomID string, accepts []int32, hb int64, err error) { var params struct { Mid int64 `json:"mid"` Key string `json:"key"` RoomID string `json:"room_id"` Platform string `json:"platform"` Accepts []int32 `json:"accepts"` } if err = json.Unmarshal(token, ¶ms); err != nil { log.Errorf("json.Unmarshal(%s) error(%v)", token, err) return } mid = params.Mid roomID = params.RoomID accepts = params.Accepts hb = int64(l.c.Node.Heartbeat) * int64(l.c.Node.HeartbeatMax) if key = params.Key; key == "" { key = uuid.New().String() } if err = l.dao.AddMapping(c, mid, key, server); err != nil { log.Errorf("l.dao.AddMapping(%d,%s,%s) error(%v)", mid, key, server, err) } log.Infof("conn connected key:%s server:%s mid:%d token:%s", key, server, mid, token) return } // Disconnect disconnect a conn. func (l *Logic) Disconnect(c context.Context, mid int64, key, server string) (has bool, err error) { if has, err = l.dao.DelMapping(c, mid, key, server); err != nil { log.Errorf("l.dao.DelMapping(%d,%s) error(%v)", mid, key, server) return } log.Infof("conn disconnected key:%s server:%s mid:%d", key, server, mid) return } // Heartbeat heartbeat a conn. func (l *Logic) Heartbeat(c context.Context, mid int64, key, server string) (err error) { has, err := l.dao.ExpireMapping(c, mid, key) if err != nil { log.Errorf("l.dao.ExpireMapping(%d,%s,%s) error(%v)", mid, key, server, err) return } if !has { if err = l.dao.AddMapping(c, mid, key, server); err != nil { log.Errorf("l.dao.AddMapping(%d,%s,%s) error(%v)", mid, key, server, err) return } } log.Infof("conn heartbeat key:%s server:%s mid:%d", key, server, mid) return } // RenewOnline renew a server online. func (l *Logic) RenewOnline(c context.Context, server string, roomCount map[string]int32) (map[string]int32, error) { online := &model.Online{ Server: server, RoomCount: roomCount, Updated: time.Now().Unix(), } if err := l.dao.AddServerOnline(context.Background(), server, online); err != nil { return nil, err } return l.roomCount, nil } // Receive receive a message. func (l *Logic) Receive(c context.Context, mid int64, proto *protocol.Proto) (err error) { log.Infof("receive mid:%d message:%+v", mid, proto) return } ================================================ FILE: internal/logic/conn_test.go ================================================ package logic import ( "context" "testing" "github.com/Terry-Mao/goim/api/protocol" "github.com/stretchr/testify/assert" ) func TestConnect(t *testing.T) { var ( server = "test_server" serverKey = "test_server_key" cookie = "" token = []byte(`{"mid":1, "key":"test_server_key", "room_id":"test://test_room", "platform":"web", "accepts":[1000,1001,1002]}`) ol = map[string]int32{"test://test_room": 100} c = context.Background() ) // connect mid, key, roomID, accepts, hb, err := lg.Connect(c, server, cookie, token) assert.Nil(t, err) assert.Equal(t, serverKey, key) assert.Equal(t, roomID, "test://test_room") assert.Equal(t, len(accepts), 3) assert.NotZero(t, hb) t.Log(mid, key, roomID, accepts, err) // heartbeat err = lg.Heartbeat(c, mid, key, server) assert.Nil(t, err) // disconnect has, err := lg.Disconnect(c, mid, key, server) assert.Nil(t, err) assert.Equal(t, true, has) // renew online, err := lg.RenewOnline(c, server, ol) assert.Nil(t, err) assert.NotNil(t, online) // message err = lg.Receive(c, mid, &protocol.Proto{}) assert.Nil(t, err) } ================================================ FILE: internal/logic/dao/dao.go ================================================ package dao import ( "context" "time" "github.com/Terry-Mao/goim/internal/logic/conf" "github.com/gomodule/redigo/redis" kafka "gopkg.in/Shopify/sarama.v1" ) // Dao dao. type Dao struct { c *conf.Config kafkaPub kafka.SyncProducer redis *redis.Pool redisExpire int32 } // New new a dao and return. func New(c *conf.Config) *Dao { d := &Dao{ c: c, kafkaPub: newKafkaPub(c.Kafka), redis: newRedis(c.Redis), redisExpire: int32(time.Duration(c.Redis.Expire) / time.Second), } return d } func newKafkaPub(c *conf.Kafka) kafka.SyncProducer { kc := kafka.NewConfig() kc.Producer.RequiredAcks = kafka.WaitForAll // Wait for all in-sync replicas to ack the message kc.Producer.Retry.Max = 10 // Retry up to 10 times to produce the message kc.Producer.Return.Successes = true pub, err := kafka.NewSyncProducer(c.Brokers, kc) if err != nil { panic(err) } return pub } func newRedis(c *conf.Redis) *redis.Pool { return &redis.Pool{ MaxIdle: c.Idle, MaxActive: c.Active, IdleTimeout: time.Duration(c.IdleTimeout), Dial: func() (redis.Conn, error) { conn, err := redis.Dial(c.Network, c.Addr, redis.DialConnectTimeout(time.Duration(c.DialTimeout)), redis.DialReadTimeout(time.Duration(c.ReadTimeout)), redis.DialWriteTimeout(time.Duration(c.WriteTimeout)), redis.DialPassword(c.Auth), ) if err != nil { return nil, err } return conn, nil }, } } // Close close the resource. func (d *Dao) Close() error { return d.redis.Close() } // Ping dao ping. func (d *Dao) Ping(c context.Context) error { return d.pingRedis(c) } ================================================ FILE: internal/logic/dao/dao_test.go ================================================ package dao import ( "context" "flag" "os" "testing" "github.com/Terry-Mao/goim/internal/logic/conf" ) var ( d *Dao ) func TestMain(m *testing.M) { if err := flag.Set("conf", "../../../cmd/logic/logic-example.toml"); err != nil { panic(err) } flag.Parse() if err := conf.Init(); err != nil { panic(err) } d = New(conf.Conf) if err := d.Ping(context.TODO()); err != nil { os.Exit(-1) } if err := d.Close(); err != nil { os.Exit(-1) } if err := d.Ping(context.TODO()); err == nil { os.Exit(-1) } d = New(conf.Conf) os.Exit(m.Run()) } ================================================ FILE: internal/logic/dao/kafka.go ================================================ package dao import ( "context" "strconv" pb "github.com/Terry-Mao/goim/api/logic" log "github.com/golang/glog" "github.com/golang/protobuf/proto" sarama "gopkg.in/Shopify/sarama.v1" ) // PushMsg push a message to databus. func (d *Dao) PushMsg(c context.Context, op int32, server string, keys []string, msg []byte) (err error) { pushMsg := &pb.PushMsg{ Type: pb.PushMsg_PUSH, Operation: op, Server: server, Keys: keys, Msg: msg, } b, err := proto.Marshal(pushMsg) if err != nil { return } m := &sarama.ProducerMessage{ Key: sarama.StringEncoder(keys[0]), Topic: d.c.Kafka.Topic, Value: sarama.ByteEncoder(b), } if _, _, err = d.kafkaPub.SendMessage(m); err != nil { log.Errorf("PushMsg.send(push pushMsg:%v) error(%v)", pushMsg, err) } return } // BroadcastRoomMsg push a message to databus. func (d *Dao) BroadcastRoomMsg(c context.Context, op int32, room string, msg []byte) (err error) { pushMsg := &pb.PushMsg{ Type: pb.PushMsg_ROOM, Operation: op, Room: room, Msg: msg, } b, err := proto.Marshal(pushMsg) if err != nil { return } m := &sarama.ProducerMessage{ Key: sarama.StringEncoder(room), Topic: d.c.Kafka.Topic, Value: sarama.ByteEncoder(b), } if _, _, err = d.kafkaPub.SendMessage(m); err != nil { log.Errorf("PushMsg.send(broadcast_room pushMsg:%v) error(%v)", pushMsg, err) } return } // BroadcastMsg push a message to databus. func (d *Dao) BroadcastMsg(c context.Context, op, speed int32, msg []byte) (err error) { pushMsg := &pb.PushMsg{ Type: pb.PushMsg_BROADCAST, Operation: op, Speed: speed, Msg: msg, } b, err := proto.Marshal(pushMsg) if err != nil { return } m := &sarama.ProducerMessage{ Key: sarama.StringEncoder(strconv.FormatInt(int64(op), 10)), Topic: d.c.Kafka.Topic, Value: sarama.ByteEncoder(b), } if _, _, err = d.kafkaPub.SendMessage(m); err != nil { log.Errorf("PushMsg.send(broadcast pushMsg:%v) error(%v)", pushMsg, err) } return } ================================================ FILE: internal/logic/dao/kafka_test.go ================================================ package dao import ( "context" "testing" "github.com/stretchr/testify/assert" ) func TestDaoPushMsg(t *testing.T) { var ( c = context.Background() op = int32(100) server = "test" msg = []byte("msg") keys = []string{"key"} ) err := d.PushMsg(c, op, server, keys, msg) assert.Nil(t, err) } func TestDaoBroadcastRoomMsg(t *testing.T) { var ( c = context.Background() op = int32(100) room = "test://1" msg = []byte("msg") ) err := d.BroadcastRoomMsg(c, op, room, msg) assert.Nil(t, err) } func TestDaoBroadcastMsg(t *testing.T) { var ( c = context.Background() op = int32(100) speed = int32(0) msg = []byte("") ) err := d.BroadcastMsg(c, op, speed, msg) assert.Nil(t, err) } ================================================ FILE: internal/logic/dao/redis.go ================================================ package dao import ( "context" "encoding/json" "fmt" "strconv" "github.com/Terry-Mao/goim/internal/logic/model" log "github.com/golang/glog" "github.com/gomodule/redigo/redis" "github.com/zhenjl/cityhash" ) const ( _prefixMidServer = "mid_%d" // mid -> key:server _prefixKeyServer = "key_%s" // key -> server _prefixServerOnline = "ol_%s" // server -> online ) func keyMidServer(mid int64) string { return fmt.Sprintf(_prefixMidServer, mid) } func keyKeyServer(key string) string { return fmt.Sprintf(_prefixKeyServer, key) } func keyServerOnline(key string) string { return fmt.Sprintf(_prefixServerOnline, key) } // pingRedis check redis connection. func (d *Dao) pingRedis(c context.Context) (err error) { conn := d.redis.Get() _, err = conn.Do("SET", "PING", "PONG") conn.Close() return } // AddMapping add a mapping. // Mapping: // mid -> key_server // key -> server func (d *Dao) AddMapping(c context.Context, mid int64, key, server string) (err error) { conn := d.redis.Get() defer conn.Close() var n = 2 if mid > 0 { if err = conn.Send("HSET", keyMidServer(mid), key, server); err != nil { log.Errorf("conn.Send(HSET %d,%s,%s) error(%v)", mid, server, key, err) return } if err = conn.Send("EXPIRE", keyMidServer(mid), d.redisExpire); err != nil { log.Errorf("conn.Send(EXPIRE %d,%s,%s) error(%v)", mid, key, server, err) return } n += 2 } if err = conn.Send("SET", keyKeyServer(key), server); err != nil { log.Errorf("conn.Send(HSET %d,%s,%s) error(%v)", mid, server, key, err) return } if err = conn.Send("EXPIRE", keyKeyServer(key), d.redisExpire); err != nil { log.Errorf("conn.Send(EXPIRE %d,%s,%s) error(%v)", mid, key, server, err) return } if err = conn.Flush(); err != nil { log.Errorf("conn.Flush() error(%v)", err) return } for i := 0; i < n; i++ { if _, err = conn.Receive(); err != nil { log.Errorf("conn.Receive() error(%v)", err) return } } return } // ExpireMapping expire a mapping. func (d *Dao) ExpireMapping(c context.Context, mid int64, key string) (has bool, err error) { conn := d.redis.Get() defer conn.Close() var n = 1 if mid > 0 { if err = conn.Send("EXPIRE", keyMidServer(mid), d.redisExpire); err != nil { log.Errorf("conn.Send(EXPIRE %d,%s) error(%v)", mid, key, err) return } n++ } if err = conn.Send("EXPIRE", keyKeyServer(key), d.redisExpire); err != nil { log.Errorf("conn.Send(EXPIRE %d,%s) error(%v)", mid, key, err) return } if err = conn.Flush(); err != nil { log.Errorf("conn.Flush() error(%v)", err) return } for i := 0; i < n; i++ { if has, err = redis.Bool(conn.Receive()); err != nil { log.Errorf("conn.Receive() error(%v)", err) return } } return } // DelMapping del a mapping. func (d *Dao) DelMapping(c context.Context, mid int64, key, server string) (has bool, err error) { conn := d.redis.Get() defer conn.Close() n := 1 if mid > 0 { if err = conn.Send("HDEL", keyMidServer(mid), key); err != nil { log.Errorf("conn.Send(HDEL %d,%s,%s) error(%v)", mid, key, server, err) return } n++ } if err = conn.Send("DEL", keyKeyServer(key)); err != nil { log.Errorf("conn.Send(HDEL %d,%s,%s) error(%v)", mid, key, server, err) return } if err = conn.Flush(); err != nil { log.Errorf("conn.Flush() error(%v)", err) return } for i := 0; i < n; i++ { if has, err = redis.Bool(conn.Receive()); err != nil { log.Errorf("conn.Receive() error(%v)", err) return } } return } // ServersByKeys get a server by key. func (d *Dao) ServersByKeys(c context.Context, keys []string) (res []string, err error) { conn := d.redis.Get() defer conn.Close() var args []interface{} for _, key := range keys { args = append(args, keyKeyServer(key)) } if res, err = redis.Strings(conn.Do("MGET", args...)); err != nil { log.Errorf("conn.Do(MGET %v) error(%v)", args, err) } return } // KeysByMids get a key server by mid. func (d *Dao) KeysByMids(c context.Context, mids []int64) (ress map[string]string, olMids []int64, err error) { conn := d.redis.Get() defer conn.Close() ress = make(map[string]string) for _, mid := range mids { if err = conn.Send("HGETALL", keyMidServer(mid)); err != nil { log.Errorf("conn.Do(HGETALL %d) error(%v)", mid, err) return } } if err = conn.Flush(); err != nil { log.Errorf("conn.Flush() error(%v)", err) return } for idx := 0; idx < len(mids); idx++ { var ( res map[string]string ) if res, err = redis.StringMap(conn.Receive()); err != nil { log.Errorf("conn.Receive() error(%v)", err) return } if len(res) > 0 { olMids = append(olMids, mids[idx]) } for k, v := range res { ress[k] = v } } return } // AddServerOnline add a server online. func (d *Dao) AddServerOnline(c context.Context, server string, online *model.Online) (err error) { roomsMap := map[uint32]map[string]int32{} for room, count := range online.RoomCount { rMap := roomsMap[cityhash.CityHash32([]byte(room), uint32(len(room)))%64] if rMap == nil { rMap = make(map[string]int32) roomsMap[cityhash.CityHash32([]byte(room), uint32(len(room)))%64] = rMap } rMap[room] = count } key := keyServerOnline(server) for hashKey, value := range roomsMap { err = d.addServerOnline(c, key, strconv.FormatInt(int64(hashKey), 10), &model.Online{RoomCount: value, Server: online.Server, Updated: online.Updated}) if err != nil { return } } return } func (d *Dao) addServerOnline(c context.Context, key string, hashKey string, online *model.Online) (err error) { conn := d.redis.Get() defer conn.Close() b, _ := json.Marshal(online) if err = conn.Send("HSET", key, hashKey, b); err != nil { log.Errorf("conn.Send(SET %s,%s) error(%v)", key, hashKey, err) return } if err = conn.Send("EXPIRE", key, d.redisExpire); err != nil { log.Errorf("conn.Send(EXPIRE %s) error(%v)", key, err) return } if err = conn.Flush(); err != nil { log.Errorf("conn.Flush() error(%v)", err) return } for i := 0; i < 2; i++ { if _, err = conn.Receive(); err != nil { log.Errorf("conn.Receive() error(%v)", err) return } } return } // ServerOnline get a server online. func (d *Dao) ServerOnline(c context.Context, server string) (online *model.Online, err error) { online = &model.Online{RoomCount: map[string]int32{}} key := keyServerOnline(server) for i := 0; i < 64; i++ { ol, err := d.serverOnline(c, key, strconv.FormatInt(int64(i), 10)) if err == nil && ol != nil { online.Server = ol.Server if ol.Updated > online.Updated { online.Updated = ol.Updated } for room, count := range ol.RoomCount { online.RoomCount[room] = count } } } return } func (d *Dao) serverOnline(c context.Context, key string, hashKey string) (online *model.Online, err error) { conn := d.redis.Get() defer conn.Close() b, err := redis.Bytes(conn.Do("HGET", key, hashKey)) if err != nil { if err != redis.ErrNil { log.Errorf("conn.Do(HGET %s %s) error(%v)", key, hashKey, err) } return } online = new(model.Online) if err = json.Unmarshal(b, online); err != nil { log.Errorf("serverOnline json.Unmarshal(%s) error(%v)", b, err) return } return } // DelServerOnline del a server online. func (d *Dao) DelServerOnline(c context.Context, server string) (err error) { conn := d.redis.Get() defer conn.Close() key := keyServerOnline(server) if _, err = conn.Do("DEL", key); err != nil { log.Errorf("conn.Do(DEL %s) error(%v)", key, err) } return } ================================================ FILE: internal/logic/dao/redis_test.go ================================================ package dao import ( "context" "testing" "github.com/Terry-Mao/goim/internal/logic/model" "github.com/stretchr/testify/assert" ) func TestDaopingRedis(t *testing.T) { err := d.pingRedis(context.Background()) assert.Nil(t, err) } func TestDaoAddMapping(t *testing.T) { var ( c = context.Background() mid = int64(1) key = "test_key" server = "test_server" ) err := d.AddMapping(c, 0, "test", server) assert.Nil(t, err) err = d.AddMapping(c, mid, key, server) assert.Nil(t, err) has, err := d.ExpireMapping(c, 0, "test") assert.Nil(t, err) assert.NotEqual(t, false, has) has, err = d.ExpireMapping(c, mid, key) assert.Nil(t, err) assert.NotEqual(t, false, has) res, err := d.ServersByKeys(c, []string{key}) assert.Nil(t, err) assert.Equal(t, server, res[0]) ress, mids, err := d.KeysByMids(c, []int64{mid}) assert.Nil(t, err) assert.Equal(t, server, ress[key]) assert.Equal(t, mid, mids[0]) has, err = d.DelMapping(c, 0, "test", server) assert.Nil(t, err) assert.NotEqual(t, false, has) has, err = d.DelMapping(c, mid, key, server) assert.Nil(t, err) assert.NotEqual(t, false, has) } func TestDaoAddServerOnline(t *testing.T) { var ( c = context.Background() server = "test_server" online = &model.Online{ RoomCount: map[string]int32{"room": 10}, } ) err := d.AddServerOnline(c, server, online) assert.Nil(t, err) r, err := d.ServerOnline(c, server) assert.Nil(t, err) assert.Equal(t, online.RoomCount["room"], r.RoomCount["room"]) err = d.DelServerOnline(c, server) assert.Nil(t, err) } ================================================ FILE: internal/logic/dao/size_coverage.out ================================================ mode: set github.com/Terry-Mao/goim/internal/logic/dao/dao.go:21.31,29.2 2 1 github.com/Terry-Mao/goim/internal/logic/dao/dao.go:31.52,38.16 7 1 github.com/Terry-Mao/goim/internal/logic/dao/dao.go:41.2,41.12 1 1 github.com/Terry-Mao/goim/internal/logic/dao/dao.go:38.16,39.13 1 0 github.com/Terry-Mao/goim/internal/logic/dao/dao.go:44.42,49.36 1 1 github.com/Terry-Mao/goim/internal/logic/dao/dao.go:49.36,56.18 2 1 github.com/Terry-Mao/goim/internal/logic/dao/dao.go:59.4,59.20 1 1 github.com/Terry-Mao/goim/internal/logic/dao/dao.go:56.18,58.5 1 0 github.com/Terry-Mao/goim/internal/logic/dao/dao.go:65.29,67.2 1 1 github.com/Terry-Mao/goim/internal/logic/dao/dao.go:70.45,72.2 1 1 github.com/Terry-Mao/goim/internal/logic/dao/kafka.go:14.106,23.16 3 1 github.com/Terry-Mao/goim/internal/logic/dao/kafka.go:26.2,31.55 2 1 github.com/Terry-Mao/goim/internal/logic/dao/kafka.go:34.2,34.8 1 1 github.com/Terry-Mao/goim/internal/logic/dao/kafka.go:23.16,25.3 1 0 github.com/Terry-Mao/goim/internal/logic/dao/kafka.go:31.55,33.3 1 0 github.com/Terry-Mao/goim/internal/logic/dao/kafka.go:38.98,46.16 3 1 github.com/Terry-Mao/goim/internal/logic/dao/kafka.go:49.2,54.55 2 1 github.com/Terry-Mao/goim/internal/logic/dao/kafka.go:57.2,57.8 1 1 github.com/Terry-Mao/goim/internal/logic/dao/kafka.go:46.16,48.3 1 0 github.com/Terry-Mao/goim/internal/logic/dao/kafka.go:54.55,56.3 1 0 github.com/Terry-Mao/goim/internal/logic/dao/kafka.go:61.100,70.16 3 1 github.com/Terry-Mao/goim/internal/logic/dao/kafka.go:73.2,78.55 2 1 github.com/Terry-Mao/goim/internal/logic/dao/kafka.go:81.2,81.8 1 1 github.com/Terry-Mao/goim/internal/logic/dao/kafka.go:70.16,72.3 1 0 github.com/Terry-Mao/goim/internal/logic/dao/kafka.go:78.55,80.3 1 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:23.37,25.2 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:27.38,29.2 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:31.41,33.2 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:36.56,41.2 4 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:47.88,51.13 4 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:62.2,62.67 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:66.2,66.77 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:70.2,70.36 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:74.2,74.25 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:80.2,80.8 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:51.13,52.74 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:56.3,56.78 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:60.3,60.9 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:52.74,55.4 2 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:56.78,59.4 2 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:62.67,65.3 2 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:66.77,69.3 2 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:70.36,73.3 2 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:74.25,75.42 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:75.42,78.4 2 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:84.93,88.13 4 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:95.2,95.77 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:99.2,99.36 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:103.2,103.25 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:109.2,109.8 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:88.13,89.78 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:93.3,93.6 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:89.78,92.4 2 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:95.77,98.3 2 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:99.36,102.3 2 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:103.25,104.56 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:104.56,107.4 2 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:113.98,117.13 4 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:124.2,124.59 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:128.2,128.36 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:132.2,132.25 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:138.2,138.8 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:117.13,118.66 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:122.3,122.6 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:118.66,121.4 2 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:124.59,127.3 2 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:128.36,131.3 2 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:132.25,133.56 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:133.56,136.4 2 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:142.89,146.27 4 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:149.2,149.68 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:152.2,152.8 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:146.27,148.3 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:149.68,151.3 1 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:156.111,160.27 4 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:166.2,166.36 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:170.2,170.39 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:185.2,185.8 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:160.27,161.64 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:161.64,164.4 2 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:166.36,169.3 2 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:170.39,174.61 2 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:178.3,178.19 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:181.3,181.25 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:174.61,177.4 2 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:178.19,180.4 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:181.25,183.4 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:189.99,191.44 2 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:199.2,200.39 2 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:206.2,206.8 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:191.44,193.18 2 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:197.3,197.21 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:193.18,196.4 2 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:200.39,202.17 2 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:202.17,204.4 1 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:209.112,213.58 4 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:217.2,217.63 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:221.2,221.36 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:225.2,225.25 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:231.2,231.8 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:213.58,216.3 2 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:217.63,220.3 2 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:221.36,224.3 2 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:225.25,226.42 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:226.42,229.4 2 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:235.96,238.26 3 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:250.2,250.8 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:238.26,240.30 2 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:240.30,242.35 2 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:245.4,245.42 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:242.35,244.5 1 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:245.42,247.5 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:253.109,257.16 4 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:263.2,264.49 2 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:268.2,268.8 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:257.16,258.26 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:261.3,261.9 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:258.26,260.4 1 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:264.49,267.3 2 0 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:272.77,276.46 4 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:279.2,279.8 1 1 github.com/Terry-Mao/goim/internal/logic/dao/redis.go:276.46,278.3 1 0 ================================================ FILE: internal/logic/grpc/server.go ================================================ package grpc import ( "context" "net" "time" pb "github.com/Terry-Mao/goim/api/logic" "github.com/Terry-Mao/goim/internal/logic" "github.com/Terry-Mao/goim/internal/logic/conf" "google.golang.org/grpc" "google.golang.org/grpc/keepalive" // use gzip decoder _ "google.golang.org/grpc/encoding/gzip" ) // New logic grpc server func New(c *conf.RPCServer, l *logic.Logic) *grpc.Server { keepParams := grpc.KeepaliveParams(keepalive.ServerParameters{ MaxConnectionIdle: time.Duration(c.IdleTimeout), MaxConnectionAgeGrace: time.Duration(c.ForceCloseWait), Time: time.Duration(c.KeepAliveInterval), Timeout: time.Duration(c.KeepAliveTimeout), MaxConnectionAge: time.Duration(c.MaxLifeTime), }) srv := grpc.NewServer(keepParams) pb.RegisterLogicServer(srv, &server{l}) lis, err := net.Listen(c.Network, c.Addr) if err != nil { panic(err) } go func() { if err := srv.Serve(lis); err != nil { panic(err) } }() return srv } type server struct { srv *logic.Logic } var _ pb.LogicServer = &server{} // Connect connect a conn. func (s *server) Connect(ctx context.Context, req *pb.ConnectReq) (*pb.ConnectReply, error) { mid, key, room, accepts, hb, err := s.srv.Connect(ctx, req.Server, req.Cookie, req.Token) if err != nil { return &pb.ConnectReply{}, err } return &pb.ConnectReply{Mid: mid, Key: key, RoomID: room, Accepts: accepts, Heartbeat: hb}, nil } // Disconnect disconnect a conn. func (s *server) Disconnect(ctx context.Context, req *pb.DisconnectReq) (*pb.DisconnectReply, error) { has, err := s.srv.Disconnect(ctx, req.Mid, req.Key, req.Server) if err != nil { return &pb.DisconnectReply{}, err } return &pb.DisconnectReply{Has: has}, nil } // Heartbeat beartbeat a conn. func (s *server) Heartbeat(ctx context.Context, req *pb.HeartbeatReq) (*pb.HeartbeatReply, error) { if err := s.srv.Heartbeat(ctx, req.Mid, req.Key, req.Server); err != nil { return &pb.HeartbeatReply{}, err } return &pb.HeartbeatReply{}, nil } // RenewOnline renew server online. func (s *server) RenewOnline(ctx context.Context, req *pb.OnlineReq) (*pb.OnlineReply, error) { allRoomCount, err := s.srv.RenewOnline(ctx, req.Server, req.RoomCount) if err != nil { return &pb.OnlineReply{}, err } return &pb.OnlineReply{AllRoomCount: allRoomCount}, nil } // Receive receive a message. func (s *server) Receive(ctx context.Context, req *pb.ReceiveReq) (*pb.ReceiveReply, error) { if err := s.srv.Receive(ctx, req.Mid, req.Proto); err != nil { return &pb.ReceiveReply{}, err } return &pb.ReceiveReply{}, nil } // nodes return nodes. func (s *server) Nodes(ctx context.Context, req *pb.NodesReq) (*pb.NodesReply, error) { return s.srv.NodesWeighted(ctx, req.Platform, req.ClientIP), nil } ================================================ FILE: internal/logic/http/middleware.go ================================================ package http import ( "fmt" "net/http/httputil" "runtime" "time" "github.com/gin-gonic/gin" log "github.com/golang/glog" ) func loggerHandler(c *gin.Context) { // Start timer start := time.Now() path := c.Request.URL.Path raw := c.Request.URL.RawQuery method := c.Request.Method // Process request c.Next() // Stop timer end := time.Now() latency := end.Sub(start) statusCode := c.Writer.Status() ecode := c.GetInt(contextErrCode) clientIP := c.ClientIP() if raw != "" { path = path + "?" + raw } log.Infof("METHOD:%s | PATH:%s | CODE:%d | IP:%s | TIME:%d | ECODE:%d", method, path, statusCode, clientIP, latency/time.Millisecond, ecode) } func recoverHandler(c *gin.Context) { defer func() { if err := recover(); err != nil { const size = 64 << 10 buf := make([]byte, size) buf = buf[:runtime.Stack(buf, false)] httprequest, _ := httputil.DumpRequest(c.Request, false) pnc := fmt.Sprintf("[Recovery] %s panic recovered:\n%s\n%s\n%s", time.Now().Format("2006-01-02 15:04:05"), string(httprequest), err, buf) fmt.Print(pnc) log.Error(pnc) c.AbortWithStatus(500) } }() c.Next() } ================================================ FILE: internal/logic/http/nodes.go ================================================ package http import ( "context" "github.com/gin-gonic/gin" ) func (s *Server) nodesWeighted(c *gin.Context) { var arg struct { Platform string `form:"platform"` } if err := c.BindQuery(&arg); err != nil { errors(c, RequestErr, err.Error()) return } res := s.logic.NodesWeighted(c, arg.Platform, c.ClientIP()) result(c, res, OK) } func (s *Server) nodesInstances(c *gin.Context) { res := s.logic.NodesInstances(context.TODO()) result(c, res, OK) } ================================================ FILE: internal/logic/http/online.go ================================================ package http import ( "context" "github.com/gin-gonic/gin" ) func (s *Server) onlineTop(c *gin.Context) { var arg struct { Type string `form:"type" binding:"required"` Limit int `form:"limit" binding:"required"` } if err := c.BindQuery(&arg); err != nil { errors(c, RequestErr, err.Error()) return } res, err := s.logic.OnlineTop(c, arg.Type, arg.Limit) if err != nil { result(c, nil, RequestErr) return } result(c, res, OK) } func (s *Server) onlineRoom(c *gin.Context) { var arg struct { Type string `form:"type" binding:"required"` Rooms []string `form:"rooms" binding:"required"` } if err := c.BindQuery(&arg); err != nil { errors(c, RequestErr, err.Error()) return } res, err := s.logic.OnlineRoom(c, arg.Type, arg.Rooms) if err != nil { result(c, nil, RequestErr) return } result(c, res, OK) } func (s *Server) onlineTotal(c *gin.Context) { ipCount, connCount := s.logic.OnlineTotal(context.TODO()) res := map[string]interface{}{ "ip_count": ipCount, "conn_count": connCount, } result(c, res, OK) } ================================================ FILE: internal/logic/http/push.go ================================================ package http import ( "context" "io/ioutil" "github.com/gin-gonic/gin" ) func (s *Server) pushKeys(c *gin.Context) { var arg struct { Op int32 `form:"operation"` Keys []string `form:"keys"` } if err := c.BindQuery(&arg); err != nil { errors(c, RequestErr, err.Error()) return } // read message msg, err := ioutil.ReadAll(c.Request.Body) if err != nil { errors(c, RequestErr, err.Error()) return } if err = s.logic.PushKeys(context.TODO(), arg.Op, arg.Keys, msg); err != nil { result(c, nil, RequestErr) return } result(c, nil, OK) } func (s *Server) pushMids(c *gin.Context) { var arg struct { Op int32 `form:"operation"` Mids []int64 `form:"mids"` } if err := c.BindQuery(&arg); err != nil { errors(c, RequestErr, err.Error()) return } // read message msg, err := ioutil.ReadAll(c.Request.Body) if err != nil { errors(c, RequestErr, err.Error()) return } if err = s.logic.PushMids(context.TODO(), arg.Op, arg.Mids, msg); err != nil { errors(c, ServerErr, err.Error()) return } result(c, nil, OK) } func (s *Server) pushRoom(c *gin.Context) { var arg struct { Op int32 `form:"operation" binding:"required"` Type string `form:"type" binding:"required"` Room string `form:"room" binding:"required"` } if err := c.BindQuery(&arg); err != nil { errors(c, RequestErr, err.Error()) return } // read message msg, err := ioutil.ReadAll(c.Request.Body) if err != nil { errors(c, RequestErr, err.Error()) return } if err = s.logic.PushRoom(c, arg.Op, arg.Type, arg.Room, msg); err != nil { errors(c, ServerErr, err.Error()) return } result(c, nil, OK) } func (s *Server) pushAll(c *gin.Context) { var arg struct { Op int32 `form:"operation" binding:"required"` Speed int32 `form:"speed"` } if err := c.BindQuery(&arg); err != nil { errors(c, RequestErr, err.Error()) return } msg, err := ioutil.ReadAll(c.Request.Body) if err != nil { errors(c, RequestErr, err.Error()) return } if err = s.logic.PushAll(c, arg.Op, arg.Speed, msg); err != nil { errors(c, ServerErr, err.Error()) return } result(c, nil, OK) } ================================================ FILE: internal/logic/http/result.go ================================================ package http import ( "github.com/gin-gonic/gin" ) const ( // OK ok OK = 0 // RequestErr request error RequestErr = -400 // ServerErr server error ServerErr = -500 contextErrCode = "context/err/code" ) type resp struct { Code int `json:"code"` Message string `json:"message"` Data interface{} `json:"data,omitempty"` } func errors(c *gin.Context, code int, msg string) { c.Set(contextErrCode, code) c.JSON(200, resp{ Code: code, Message: msg, }) } func result(c *gin.Context, data interface{}, code int) { c.Set(contextErrCode, code) c.JSON(200, resp{ Code: code, Data: data, }) } ================================================ FILE: internal/logic/http/server.go ================================================ package http import ( "github.com/Terry-Mao/goim/internal/logic" "github.com/Terry-Mao/goim/internal/logic/conf" "github.com/gin-gonic/gin" ) // Server is http server. type Server struct { engine *gin.Engine logic *logic.Logic } // New new a http server. func New(c *conf.HTTPServer, l *logic.Logic) *Server { engine := gin.New() engine.Use(loggerHandler, recoverHandler) go func() { if err := engine.Run(c.Addr); err != nil { panic(err) } }() s := &Server{ engine: engine, logic: l, } s.initRouter() return s } func (s *Server) initRouter() { group := s.engine.Group("/goim") group.POST("/push/keys", s.pushKeys) group.POST("/push/mids", s.pushMids) group.POST("/push/room", s.pushRoom) group.POST("/push/all", s.pushAll) group.GET("/online/top", s.onlineTop) group.GET("/online/room", s.onlineRoom) group.GET("/online/total", s.onlineTotal) group.GET("/nodes/weighted", s.nodesWeighted) group.GET("/nodes/instances", s.nodesInstances) } // Close close the server. func (s *Server) Close() { } ================================================ FILE: internal/logic/logic.go ================================================ package logic import ( "context" "strconv" "time" "github.com/Terry-Mao/goim/internal/logic/conf" "github.com/Terry-Mao/goim/internal/logic/dao" "github.com/Terry-Mao/goim/internal/logic/model" "github.com/bilibili/discovery/naming" log "github.com/golang/glog" ) const ( _onlineTick = time.Second * 10 _onlineDeadline = time.Minute * 5 ) // Logic struct type Logic struct { c *conf.Config dis *naming.Discovery dao *dao.Dao // online totalIPs int64 totalConns int64 roomCount map[string]int32 // load balancer nodes []*naming.Instance loadBalancer *LoadBalancer regions map[string]string // province -> region } // New init func New(c *conf.Config) (l *Logic) { l = &Logic{ c: c, dao: dao.New(c), dis: naming.New(c.Discovery), loadBalancer: NewLoadBalancer(), regions: make(map[string]string), } l.initRegions() l.initNodes() _ = l.loadOnline() go l.onlineproc() return l } // Ping ping resources is ok. func (l *Logic) Ping(c context.Context) (err error) { return l.dao.Ping(c) } // Close close resources. func (l *Logic) Close() { l.dao.Close() } func (l *Logic) initRegions() { for region, ps := range l.c.Regions { for _, province := range ps { l.regions[province] = region } } } func (l *Logic) initNodes() { res := l.dis.Build("goim.comet") event := res.Watch() select { case _, ok := <-event: if ok { l.newNodes(res) } else { panic("discovery watch failed") } case <-time.After(10 * time.Second): log.Error("discovery start timeout") } go func() { for { if _, ok := <-event; !ok { return } l.newNodes(res) } }() } func (l *Logic) newNodes(res naming.Resolver) { if zoneIns, ok := res.Fetch(); ok { var ( totalConns int64 totalIPs int64 allIns []*naming.Instance ) for _, zins := range zoneIns.Instances { for _, ins := range zins { if ins.Metadata == nil { log.Errorf("node instance metadata is empty(%+v)", ins) continue } offline, err := strconv.ParseBool(ins.Metadata[model.MetaOffline]) if err != nil || offline { log.Warningf("strconv.ParseBool(offline:%t) error(%v)", offline, err) continue } conns, err := strconv.ParseInt(ins.Metadata[model.MetaConnCount], 10, 32) if err != nil { log.Errorf("strconv.ParseInt(conns:%d) error(%v)", conns, err) continue } ips, err := strconv.ParseInt(ins.Metadata[model.MetaIPCount], 10, 32) if err != nil { log.Errorf("strconv.ParseInt(ips:%d) error(%v)", ips, err) continue } totalConns += conns totalIPs += ips allIns = append(allIns, ins) } } l.totalConns = totalConns l.totalIPs = totalIPs l.nodes = allIns l.loadBalancer.Update(allIns) } } func (l *Logic) onlineproc() { for { time.Sleep(_onlineTick) if err := l.loadOnline(); err != nil { log.Errorf("onlineproc error(%v)", err) } } } func (l *Logic) loadOnline() (err error) { var ( roomCount = make(map[string]int32) ) for _, server := range l.nodes { var online *model.Online online, err = l.dao.ServerOnline(context.Background(), server.Hostname) if err != nil { return } if time.Since(time.Unix(online.Updated, 0)) > _onlineDeadline { _ = l.dao.DelServerOnline(context.Background(), server.Hostname) continue } for roomID, count := range online.RoomCount { roomCount[roomID] += count } } l.roomCount = roomCount return } ================================================ FILE: internal/logic/logic_test.go ================================================ package logic import ( "context" "flag" "os" "testing" "github.com/Terry-Mao/goim/internal/logic/conf" ) var ( lg *Logic ) func TestMain(m *testing.M) { if err := flag.Set("conf", "../../cmd/logic/logic-example.toml"); err != nil { panic(err) } flag.Parse() if err := conf.Init(); err != nil { panic(err) } lg = New(conf.Conf) if err := lg.Ping(context.TODO()); err != nil { panic(err) } os.Exit(m.Run()) } ================================================ FILE: internal/logic/model/metadata.go ================================================ package model const ( // MetaWeight meta weight MetaWeight = "weight" // MetaOffline meta offline MetaOffline = "offline" // MetaAddrs meta public ip addrs MetaAddrs = "addrs" // MetaIPCount meta ip count MetaIPCount = "ip_count" // MetaConnCount meta conn count MetaConnCount = "conn_count" // PlatformWeb platform web PlatformWeb = "web" ) ================================================ FILE: internal/logic/model/online.go ================================================ package model // Online ip and room online. type Online struct { Server string `json:"server"` RoomCount map[string]int32 `json:"room_count"` Updated int64 `json:"updated"` } // Top top sorted. type Top struct { RoomID string `json:"room_id"` Count int32 `json:"count"` } ================================================ FILE: internal/logic/model/room.go ================================================ package model import ( "fmt" "net/url" ) // EncodeRoomKey encode a room key. func EncodeRoomKey(typ string, room string) string { return fmt.Sprintf("%s://%s", typ, room) } // DecodeRoomKey decode room key. func DecodeRoomKey(key string) (string, string, error) { u, err := url.Parse(key) if err != nil { return "", "", err } return u.Scheme, u.Host, nil } ================================================ FILE: internal/logic/nodes.go ================================================ package logic import ( "context" "time" pb "github.com/Terry-Mao/goim/api/logic" "github.com/Terry-Mao/goim/internal/logic/model" "github.com/bilibili/discovery/naming" log "github.com/golang/glog" ) // NodesInstances get servers info. func (l *Logic) NodesInstances(c context.Context) (res []*naming.Instance) { return l.nodes } // NodesWeighted get node list. func (l *Logic) NodesWeighted(c context.Context, platform, clientIP string) *pb.NodesReply { reply := &pb.NodesReply{ Domain: l.c.Node.DefaultDomain, TcpPort: int32(l.c.Node.TCPPort), WsPort: int32(l.c.Node.WSPort), WssPort: int32(l.c.Node.WSSPort), Heartbeat: int32(time.Duration(l.c.Node.Heartbeat) / time.Second), HeartbeatMax: int32(l.c.Node.HeartbeatMax), Backoff: &pb.Backoff{ MaxDelay: l.c.Backoff.MaxDelay, BaseDelay: l.c.Backoff.BaseDelay, Factor: l.c.Backoff.Factor, Jitter: l.c.Backoff.Jitter, }, } domains, addrs := l.nodeAddrs(c, clientIP) if platform == model.PlatformWeb { reply.Nodes = domains } else { reply.Nodes = addrs } if len(reply.Nodes) == 0 { reply.Nodes = []string{l.c.Node.DefaultDomain} } return reply } func (l *Logic) nodeAddrs(c context.Context, clientIP string) (domains, addrs []string) { var ( region string ) province, err := l.location(c, clientIP) if err == nil { region = l.regions[province] } log.Infof("nodeAddrs clientIP:%s region:%s province:%s domains:%v addrs:%v", clientIP, region, province, domains, addrs) return l.loadBalancer.NodeAddrs(region, l.c.Node.HostDomain, l.c.Node.RegionWeight) } // location find a geolocation of an IP address including province, region and country. func (l *Logic) location(c context.Context, clientIP string) (province string, err error) { // province: config mapping return } ================================================ FILE: internal/logic/nodes_test.go ================================================ package logic import ( "context" "testing" "github.com/bilibili/discovery/naming" "github.com/Terry-Mao/goim/internal/logic/model" "github.com/stretchr/testify/assert" ) func TestNodes(t *testing.T) { var ( c = context.TODO() clientIP = "127.0.0.1" ) lg.nodes = make([]*naming.Instance, 0) ins := lg.NodesInstances(c) assert.NotNil(t, ins) nodes := lg.NodesWeighted(c, model.PlatformWeb, clientIP) assert.NotNil(t, nodes) nodes = lg.NodesWeighted(c, "android", clientIP) assert.NotNil(t, nodes) } ================================================ FILE: internal/logic/online.go ================================================ package logic import ( "context" "sort" "strings" "github.com/Terry-Mao/goim/internal/logic/model" ) var ( _emptyTops = make([]*model.Top, 0) ) // OnlineTop get the top online. func (l *Logic) OnlineTop(c context.Context, typ string, n int) (tops []*model.Top, err error) { for key, cnt := range l.roomCount { if strings.HasPrefix(key, typ) { _, roomID, err := model.DecodeRoomKey(key) if err != nil { continue } top := &model.Top{ RoomID: roomID, Count: cnt, } tops = append(tops, top) } } sort.Slice(tops, func(i, j int) bool { return tops[i].Count > tops[j].Count }) if len(tops) > n { tops = tops[:n] } if len(tops) == 0 { tops = _emptyTops } return } // OnlineRoom get rooms online. func (l *Logic) OnlineRoom(c context.Context, typ string, rooms []string) (res map[string]int32, err error) { res = make(map[string]int32, len(rooms)) for _, room := range rooms { res[room] = l.roomCount[model.EncodeRoomKey(typ, room)] } return } // OnlineTotal get all online. func (l *Logic) OnlineTotal(c context.Context) (int64, int64) { return l.totalIPs, l.totalConns } ================================================ FILE: internal/logic/online_test.go ================================================ package logic import ( "context" "testing" "github.com/stretchr/testify/assert" ) func TestOnline(t *testing.T) { var ( c = context.TODO() typ = "test" n = 2 rooms = []string{"room_01", "room_02", "room_03"} ) lg.totalIPs = 100 lg.totalConns = 200 lg.roomCount = map[string]int32{ "test://room_01": 100, "test://room_02": 200, "test://room_03": 300, } tops, err := lg.OnlineTop(c, typ, n) assert.Nil(t, err) assert.Equal(t, len(tops), 2) onlines, err := lg.OnlineRoom(c, typ, rooms) assert.Nil(t, err) assert.Equal(t, onlines["room_01"], int32(100)) assert.Equal(t, onlines["room_02"], int32(200)) assert.Equal(t, onlines["room_03"], int32(300)) ips, conns := lg.OnlineTotal(c) assert.Equal(t, ips, int64(100)) assert.Equal(t, conns, int64(200)) } ================================================ FILE: internal/logic/push.go ================================================ package logic import ( "context" "github.com/Terry-Mao/goim/internal/logic/model" log "github.com/golang/glog" ) // PushKeys push a message by keys. func (l *Logic) PushKeys(c context.Context, op int32, keys []string, msg []byte) (err error) { servers, err := l.dao.ServersByKeys(c, keys) if err != nil { return } pushKeys := make(map[string][]string) for i, key := range keys { server := servers[i] if server != "" && key != "" { pushKeys[server] = append(pushKeys[server], key) } } for server := range pushKeys { if err = l.dao.PushMsg(c, op, server, pushKeys[server], msg); err != nil { return } } return } // PushMids push a message by mid. func (l *Logic) PushMids(c context.Context, op int32, mids []int64, msg []byte) (err error) { keyServers, _, err := l.dao.KeysByMids(c, mids) if err != nil { return } keys := make(map[string][]string) for key, server := range keyServers { if key == "" || server == "" { log.Warningf("push key:%s server:%s is empty", key, server) continue } keys[server] = append(keys[server], key) } for server, keys := range keys { if err = l.dao.PushMsg(c, op, server, keys, msg); err != nil { return } } return } // PushRoom push a message by room. func (l *Logic) PushRoom(c context.Context, op int32, typ, room string, msg []byte) (err error) { return l.dao.BroadcastRoomMsg(c, op, model.EncodeRoomKey(typ, room), msg) } // PushAll push a message to all. func (l *Logic) PushAll(c context.Context, op, speed int32, msg []byte) (err error) { return l.dao.BroadcastMsg(c, op, speed, msg) } ================================================ FILE: internal/logic/push_test.go ================================================ package logic import ( "context" "testing" "github.com/stretchr/testify/assert" ) func TestPushKeys(t *testing.T) { var ( c = context.TODO() op = int32(100) keys = []string{"test_key"} msg = []byte("hello") ) err := lg.PushKeys(c, op, keys, msg) assert.Nil(t, err) } func TestPushMids(t *testing.T) { var ( c = context.TODO() op = int32(100) mids = []int64{1, 2, 3} msg = []byte("hello") ) err := lg.PushMids(c, op, mids, msg) assert.Nil(t, err) } func TestPushRoom(t *testing.T) { var ( c = context.TODO() op = int32(100) typ = "test" room = "test_room" msg = []byte("hello") ) err := lg.PushRoom(c, op, typ, room, msg) assert.Nil(t, err) } func TestPushAll(t *testing.T) { var ( c = context.TODO() op = int32(100) speed = int32(100) msg = []byte("hello") ) err := lg.PushAll(c, op, speed, msg) assert.Nil(t, err) } ================================================ FILE: pkg/bufio/bufio.go ================================================ // Copyright 2009 The Go Authors. All rights reserved. // Use of this source code is governed by a BSD-style // license that can be found in the LICENSE file. // Package bufio implements buffered I/O. It wraps an io.Reader or io.Writer // object, creating another object (Reader or Writer) that also implements // the interface but provides buffering and some help for textual I/O. package bufio import ( "bytes" "errors" "io" ) const ( defaultBufSize = 4096 ) var ( // ErrInvalidUnreadByte invalid use of UnreadByete ErrInvalidUnreadByte = errors.New("bufio: invalid use of UnreadByte") // ErrInvalidUnreadRune invalid use of UnreadRune ErrInvalidUnreadRune = errors.New("bufio: invalid use of UnreadRune") // ErrBufferFull buffer full ErrBufferFull = errors.New("bufio: buffer full") // ErrNegativeCount negative count ErrNegativeCount = errors.New("bufio: negative count") ) // Buffered input. // Reader implements buffering for an io.Reader object. type Reader struct { buf []byte rd io.Reader // reader provided by the client r, w int // buf read and write positions err error } const minReadBufferSize = 16 const maxConsecutiveEmptyReads = 100 // NewReaderSize returns a new Reader whose buffer has at least the specified // size. If the argument io.Reader is already a Reader with large enough // size, it returns the underlying Reader. func NewReaderSize(rd io.Reader, size int) *Reader { // Is it already a Reader? b, ok := rd.(*Reader) if ok && len(b.buf) >= size { return b } if size < minReadBufferSize { size = minReadBufferSize } r := new(Reader) r.reset(make([]byte, size), rd) return r } // NewReader returns a new Reader whose buffer has the default size. func NewReader(rd io.Reader) *Reader { return NewReaderSize(rd, defaultBufSize) } // Reset discards any buffered data, resets all state, and switches // the buffered reader to read from r. func (b *Reader) Reset(r io.Reader) { b.reset(b.buf, r) } // ResetBuffer discards any buffered data, resets all state, and switches // the buffered reader to read from r. func (b *Reader) ResetBuffer(r io.Reader, buf []byte) { b.reset(buf, r) } func (b *Reader) reset(buf []byte, r io.Reader) { *b = Reader{ buf: buf, rd: r, } } var errNegativeRead = errors.New("bufio: reader returned negative count from Read") // fill reads a new chunk into the buffer. func (b *Reader) fill() { // Slide existing data to beginning. if b.r > 0 { copy(b.buf, b.buf[b.r:b.w]) b.w -= b.r b.r = 0 } if b.w >= len(b.buf) { panic("bufio: tried to fill full buffer") } // Read new data: try a limited number of times. for i := maxConsecutiveEmptyReads; i > 0; i-- { n, err := b.rd.Read(b.buf[b.w:]) if n < 0 { panic(errNegativeRead) } b.w += n if err != nil { b.err = err return } if n > 0 { return } } b.err = io.ErrNoProgress } func (b *Reader) readErr() error { err := b.err b.err = nil return err } // Peek returns the next n bytes without advancing the reader. The bytes stop // being valid at the next read call. If Peek returns fewer than n bytes, it // also returns an error explaining why the read is short. The error is // ErrBufferFull if n is larger than b's buffer size. func (b *Reader) Peek(n int) ([]byte, error) { if n < 0 { return nil, ErrNegativeCount } if n > len(b.buf) { return nil, ErrBufferFull } // 0 <= n <= len(b.buf) for b.w-b.r < n && b.err == nil { b.fill() // b.w-b.r < len(b.buf) => buffer is not full } var err error if avail := b.w - b.r; avail < n { // not enough data in buffer n = avail err = b.readErr() if err == nil { err = ErrBufferFull } } return b.buf[b.r : b.r+n], err } // Pop returns the next n bytes with advancing the reader. The bytes stop // being valid at the next read call. If Pop returns fewer than n bytes, it // also returns an error explaining why the read is short. The error is // ErrBufferFull if n is larger than b's buffer size. func (b *Reader) Pop(n int) ([]byte, error) { d, err := b.Peek(n) if err == nil { b.r += n return d, err } return nil, err } // Discard skips the next n bytes, returning the number of bytes discarded. // // If Discard skips fewer than n bytes, it also returns an error. // If 0 <= n <= b.Buffered(), Discard is guaranteed to succeed without // reading from the underlying io.Reader. func (b *Reader) Discard(n int) (discarded int, err error) { if n < 0 { return 0, ErrNegativeCount } if n == 0 { return } remain := n for { skip := b.Buffered() if skip == 0 { b.fill() skip = b.Buffered() } if skip > remain { skip = remain } b.r += skip remain -= skip if remain == 0 { return n, nil } if b.err != nil { return n - remain, b.readErr() } } } // Read reads data into p. // It returns the number of bytes read into p. // It calls Read at most once on the underlying Reader, // hence n may be less than len(p). // At EOF, the count will be zero and err will be io.EOF. func (b *Reader) Read(p []byte) (n int, err error) { n = len(p) if n == 0 { return 0, b.readErr() } if b.r == b.w { if b.err != nil { return 0, b.readErr() } if len(p) >= len(b.buf) { // Large read, empty buffer. // Read directly into p to avoid copy. n, b.err = b.rd.Read(p) if n < 0 { panic(errNegativeRead) } return n, b.readErr() } b.fill() // buffer is empty if b.r == b.w { return 0, b.readErr() } } // copy as much as we can n = copy(p, b.buf[b.r:b.w]) b.r += n return n, nil } // ReadByte reads and returns a single byte. // If no byte is available, returns an error. func (b *Reader) ReadByte() (c byte, err error) { //b.lastRuneSize = -1 for b.r == b.w { if b.err != nil { return 0, b.readErr() } b.fill() // buffer is empty } c = b.buf[b.r] b.r++ //b.lastByte = int(c) return c, nil } // ReadSlice reads until the first occurrence of delim in the input, // returning a slice pointing at the bytes in the buffer. // The bytes stop being valid at the next read. // If ReadSlice encounters an error before finding a delimiter, // it returns all the data in the buffer and the error itself (often io.EOF). // ReadSlice fails with error ErrBufferFull if the buffer fills without a delim. // Because the data returned from ReadSlice will be overwritten // by the next I/O operation, most clients should use // ReadBytes or ReadString instead. // ReadSlice returns err != nil if and only if line does not end in delim. func (b *Reader) ReadSlice(delim byte) (line []byte, err error) { for { // Search buffer. if i := bytes.IndexByte(b.buf[b.r:b.w], delim); i >= 0 { line = b.buf[b.r : b.r+i+1] b.r += i + 1 break } // Pending error? if b.err != nil { line = b.buf[b.r:b.w] b.r = b.w err = b.readErr() break } // Buffer full? if b.Buffered() >= len(b.buf) { b.r = b.w line = b.buf err = ErrBufferFull break } b.fill() // buffer is not full } return } // ReadLine is a low-level line-reading primitive. Most callers should use // ReadBytes('\n') or ReadString('\n') instead or use a Scanner. // // ReadLine tries to return a single line, not including the end-of-line bytes. // If the line was too long for the buffer then isPrefix is set and the // beginning of the line is returned. The rest of the line will be returned // from future calls. isPrefix will be false when returning the last fragment // of the line. The returned buffer is only valid until the next call to // ReadLine. ReadLine either returns a non-nil line or it returns an error, // never both. // // The text returned from ReadLine does not include the line end ("\r\n" or "\n"). // No indication or error is given if the input ends without a final line end. // Calling UnreadByte after ReadLine will always unread the last byte read // (possibly a character belonging to the line end) even if that byte is not // part of the line returned by ReadLine. func (b *Reader) ReadLine() (line []byte, isPrefix bool, err error) { line, err = b.ReadSlice('\n') if err == ErrBufferFull { // Handle the case where "\r\n" straddles the buffer. if len(line) > 0 && line[len(line)-1] == '\r' { // Put the '\r' back on buf and drop it from line. // Let the next call to ReadLine check for "\r\n". if b.r == 0 { // should be unreachable panic("bufio: tried to rewind past start of buffer") } b.r-- line = line[:len(line)-1] } return line, true, nil } if len(line) == 0 { if err != nil { line = nil } return } err = nil if line[len(line)-1] == '\n' { drop := 1 if len(line) > 1 && line[len(line)-2] == '\r' { drop = 2 } line = line[:len(line)-drop] } return } // Buffered returns the number of bytes that can be read from the current buffer. func (b *Reader) Buffered() int { return b.w - b.r } // buffered output // Writer implements buffering for an io.Writer object. // If an error occurs writing to a Writer, no more data will be // accepted and all subsequent writes will return the error. // After all data has been written, the client should call the // Flush method to guarantee all data has been forwarded to // the underlying io.Writer. type Writer struct { err error buf []byte n int wr io.Writer } // NewWriterSize returns a new Writer whose buffer has at least the specified // size. If the argument io.Writer is already a Writer with large enough // size, it returns the underlying Writer. func NewWriterSize(w io.Writer, size int) *Writer { // Is it already a Writer? b, ok := w.(*Writer) if ok && len(b.buf) >= size { return b } if size <= 0 { size = defaultBufSize } return &Writer{ buf: make([]byte, size), wr: w, } } // NewWriter returns a new Writer whose buffer has the default size. func NewWriter(w io.Writer) *Writer { return NewWriterSize(w, defaultBufSize) } // Reset discards any unflushed buffered data, clears any error, and // resets b to write its output to w. func (b *Writer) Reset(w io.Writer) { b.err = nil b.n = 0 b.wr = w } // ResetBuffer discards any unflushed buffered data, clears any error, and // resets b to write its output to w. func (b *Writer) ResetBuffer(w io.Writer, buf []byte) { b.buf = buf b.err = nil b.n = 0 b.wr = w } // Flush writes any buffered data to the underlying io.Writer. func (b *Writer) Flush() error { err := b.flush() return err } func (b *Writer) flush() error { if b.err != nil { return b.err } if b.n == 0 { return nil } n, err := b.wr.Write(b.buf[0:b.n]) if n < b.n && err == nil { err = io.ErrShortWrite } if err != nil { if n > 0 && n < b.n { copy(b.buf[0:b.n-n], b.buf[n:b.n]) } b.n -= n b.err = err return err } b.n = 0 return nil } // Available returns how many bytes are unused in the buffer. func (b *Writer) Available() int { return len(b.buf) - b.n } // Buffered returns the number of bytes that have been written into the current buffer. func (b *Writer) Buffered() int { return b.n } // Write writes the contents of p into the buffer. // It returns the number of bytes written. // If nn < len(p), it also returns an error explaining // why the write is short. func (b *Writer) Write(p []byte) (nn int, err error) { for len(p) > b.Available() && b.err == nil { var n int if b.Buffered() == 0 { // Large write, empty buffer. // Write directly from p to avoid copy. n, b.err = b.wr.Write(p) } else { n = copy(b.buf[b.n:], p) b.n += n b.flush() } nn += n p = p[n:] } if b.err != nil { return nn, b.err } n := copy(b.buf[b.n:], p) b.n += n nn += n return nn, nil } // WriteRaw writes the contents of p into the raw io.Writer without buffer. // It returns the number of bytes written. // If nn < len(p), it also returns an error explaining // why the write is short. func (b *Writer) WriteRaw(p []byte) (nn int, err error) { if b.err != nil { return 0, b.err } if b.Buffered() == 0 { // if no buffer data, write raw writer nn, err = b.wr.Write(p) b.err = err } else { nn, err = b.Write(p) } return } // Peek returns the next n bytes with advancing the writer. The bytes stop // being used at the next write call. If Peek returns fewer than n bytes, it // also returns an error explaining why the read is short. The error is // ErrBufferFull if n is larger than b's buffer size. func (b *Writer) Peek(n int) ([]byte, error) { if n < 0 { return nil, ErrNegativeCount } if n > len(b.buf) { return nil, ErrBufferFull } for b.Available() < n && b.err == nil { b.flush() } if b.err != nil { return nil, b.err } d := b.buf[b.n : b.n+n] b.n += n return d, nil } // WriteString writes a string. // It returns the number of bytes written. // If the count is less than len(s), it also returns an error explaining // why the write is short. func (b *Writer) WriteString(s string) (int, error) { nn := 0 for len(s) > b.Available() && b.err == nil { n := copy(b.buf[b.n:], s) b.n += n nn += n s = s[n:] b.flush() } if b.err != nil { return nn, b.err } n := copy(b.buf[b.n:], s) b.n += n nn += n return nn, nil } ================================================ FILE: pkg/bufio/bufio_test.go ================================================ // Copyright 2009 The Go Authors. All rights reserved. // Use of this source code is governed by a BSD-style // license that can be found in the LICENSE file. package bufio_test import ( "bytes" "errors" "fmt" "io" "io/ioutil" "strings" "testing" "testing/iotest" "time" . "github.com/Terry-Mao/goim/pkg/bufio" ) // Reads from a reader and rot13s the result. type rot13Reader struct { r io.Reader } func newRot13Reader(r io.Reader) *rot13Reader { r13 := new(rot13Reader) r13.r = r return r13 } func (r13 *rot13Reader) Read(p []byte) (int, error) { n, err := r13.r.Read(p) for i := 0; i < n; i++ { c := p[i] | 0x20 // lowercase byte if 'a' <= c && c <= 'm' { p[i] += 13 } else if 'n' <= c && c <= 'z' { p[i] -= 13 } } return n, err } // Call ReadByte to accumulate the text of a file func readBytes(buf *Reader) string { var b [1000]byte nb := 0 for { c, err := buf.ReadByte() if err == io.EOF { break } if err == nil { b[nb] = c nb++ } else if err != iotest.ErrTimeout { panic("Data: " + err.Error()) } } return string(b[0:nb]) } func TestReaderSimple(t *testing.T) { data := "hello world" b := NewReader(strings.NewReader(data)) if s := readBytes(b); s != "hello world" { t.Errorf("simple hello world test failed: got %q", s) } b = NewReader(newRot13Reader(strings.NewReader(data))) if s := readBytes(b); s != "uryyb jbeyq" { t.Errorf("rot13 hello world test failed: got %q", s) } } type readMaker struct { name string fn func(io.Reader) io.Reader } var readMakers = []readMaker{ {"full", func(r io.Reader) io.Reader { return r }}, {"byte", iotest.OneByteReader}, {"half", iotest.HalfReader}, {"data+err", iotest.DataErrReader}, {"timeout", iotest.TimeoutReader}, } // Call Read to accumulate the text of a file func reads(buf *Reader, m int) string { var b [1000]byte nb := 0 for { n, err := buf.Read(b[nb : nb+m]) nb += n if err == io.EOF { break } } return string(b[0:nb]) } type bufReader struct { name string fn func(*Reader) string } var bufreaders = []bufReader{ {"1", func(b *Reader) string { return reads(b, 1) }}, {"2", func(b *Reader) string { return reads(b, 2) }}, {"3", func(b *Reader) string { return reads(b, 3) }}, {"4", func(b *Reader) string { return reads(b, 4) }}, {"5", func(b *Reader) string { return reads(b, 5) }}, {"7", func(b *Reader) string { return reads(b, 7) }}, {"bytes", readBytes}, } const minReadBufferSize = 16 var bufsizes = []int{ 0, minReadBufferSize, 23, 32, 46, 64, 93, 128, 1024, 4096, } func TestReader(t *testing.T) { var texts [31]string str := "" all := "" for i := 0; i < len(texts)-1; i++ { texts[i] = str + "\n" all += texts[i] str += fmt.Sprintf("%x", i%26+'a') } texts[len(texts)-1] = all for h := 0; h < len(texts); h++ { text := texts[h] for i := 0; i < len(readMakers); i++ { for j := 0; j < len(bufreaders); j++ { for k := 0; k < len(bufsizes); k++ { readmaker := readMakers[i] bufreader := bufreaders[j] bufsize := bufsizes[k] read := readmaker.fn(strings.NewReader(text)) buf := NewReaderSize(read, bufsize) s := bufreader.fn(buf) if s != text { t.Errorf("reader=%s fn=%s bufsize=%d want=%q got=%q", readmaker.name, bufreader.name, bufsize, text, s) } } } } } } type zeroReader struct{} func (zeroReader) Read(p []byte) (int, error) { return 0, nil } func TestZeroReader(t *testing.T) { var z zeroReader r := NewReader(z) c := make(chan error) go func() { _, err := r.ReadByte() c <- err }() select { case err := <-c: if err == nil { t.Error("error expected") } else if err != io.ErrNoProgress { t.Error("unexpected error:", err) } case <-time.After(time.Second): t.Error("test timed out (endless loop in ReadByte?)") } } func TestWriter(t *testing.T) { var data [8192]byte for i := 0; i < len(data); i++ { data[i] = byte(' ' + i%('~'-' ')) } w := new(bytes.Buffer) for i := 0; i < len(bufsizes); i++ { for j := 0; j < len(bufsizes); j++ { nwrite := bufsizes[i] bs := bufsizes[j] // Write nwrite bytes using buffer size bs. // Check that the right amount makes it out // and that the data is correct. w.Reset() buf := NewWriterSize(w, bs) context := fmt.Sprintf("nwrite=%d bufsize=%d", nwrite, bs) n, e1 := buf.Write(data[0:nwrite]) if e1 != nil || n != nwrite { t.Errorf("%s: buf.Write %d = %d, %v", context, nwrite, n, e1) continue } if e := buf.Flush(); e != nil { t.Errorf("%s: buf.Flush = %v", context, e) } written := w.Bytes() if len(written) != nwrite { t.Errorf("%s: %d bytes written", context, len(written)) } for l := 0; l < len(written); l++ { if written[i] != data[i] { t.Errorf("wrong bytes written") t.Errorf("want=%q", data[0:len(written)]) t.Errorf("have=%q", written) } } } } } // Check that write errors are returned properly. type errorWriterTest struct { n, m int err error expect error } func (w errorWriterTest) Write(p []byte) (int, error) { return len(p) * w.n / w.m, w.err } var errorWriterTests = []errorWriterTest{ {0, 1, nil, io.ErrShortWrite}, {1, 2, nil, io.ErrShortWrite}, {1, 1, nil, nil}, {0, 1, io.ErrClosedPipe, io.ErrClosedPipe}, {1, 2, io.ErrClosedPipe, io.ErrClosedPipe}, {1, 1, io.ErrClosedPipe, io.ErrClosedPipe}, } func TestWriteErrors(t *testing.T) { for _, w := range errorWriterTests { buf := NewWriter(w) _, e := buf.Write([]byte("hello world")) if e != nil { t.Errorf("Write hello to %v: %v", w, e) continue } // Two flushes, to verify the error is sticky. for i := 0; i < 2; i++ { e = buf.Flush() if e != w.expect { t.Errorf("Flush %d/2 %v: got %v, wanted %v", i+1, w, e, w.expect) } } } } func TestNewReaderSizeIdempotent(t *testing.T) { const BufSize = 1000 b := NewReaderSize(strings.NewReader("hello world"), BufSize) // Does it recognize itself? b1 := NewReaderSize(b, BufSize) if b1 != b { t.Error("NewReaderSize did not detect underlying Reader") } // Does it wrap if existing buffer is too small? b2 := NewReaderSize(b, 2*BufSize) if b2 == b { t.Error("NewReaderSize did not enlarge buffer") } } func TestNewWriterSizeIdempotent(t *testing.T) { const BufSize = 1000 b := NewWriterSize(new(bytes.Buffer), BufSize) // Does it recognize itself? b1 := NewWriterSize(b, BufSize) if b1 != b { t.Error("NewWriterSize did not detect underlying Writer") } // Does it wrap if existing buffer is too small? b2 := NewWriterSize(b, 2*BufSize) if b2 == b { t.Error("NewWriterSize did not enlarge buffer") } } func TestWriteString(t *testing.T) { const BufSize = 8 buf := new(bytes.Buffer) b := NewWriterSize(buf, BufSize) _, _ = b.WriteString("0") // easy _, _ = b.WriteString("123456") // still easy _, _ = b.WriteString("7890") // easy after flush _, _ = b.WriteString("abcdefghijklmnopqrstuvwxy") // hard _, _ = b.WriteString("z") if err := b.Flush(); err != nil { t.Error("WriteString", err) } s := "01234567890abcdefghijklmnopqrstuvwxyz" if buf.String() != s { t.Errorf("WriteString wants %q gets %q", s, buf.String()) } } func TestBufferFull(t *testing.T) { const longString = "And now, hello, world! It is the time for all good men to come to the aid of their party" buf := NewReaderSize(strings.NewReader(longString), minReadBufferSize) line, err := buf.ReadSlice('!') if string(line) != "And now, hello, " || err != ErrBufferFull { t.Errorf("first ReadSlice(,) = %q, %v", line, err) } line, err = buf.ReadSlice('!') if string(line) != "world!" || err != nil { t.Errorf("second ReadSlice(,) = %q, %v", line, err) } } func TestPeek(t *testing.T) { p := make([]byte, 10) // string is 16 (minReadBufferSize) long. buf := NewReaderSize(strings.NewReader("abcdefghijklmnop"), minReadBufferSize) if s, err := buf.Peek(1); string(s) != "a" || err != nil { t.Fatalf("want %q got %q, err=%v", "a", string(s), err) } if s, err := buf.Peek(4); string(s) != "abcd" || err != nil { t.Fatalf("want %q got %q, err=%v", "abcd", string(s), err) } if _, err := buf.Peek(-1); err != ErrNegativeCount { t.Fatalf("want ErrNegativeCount got %v", err) } if _, err := buf.Peek(32); err != ErrBufferFull { t.Fatalf("want ErrBufFull got %v", err) } if _, err := buf.Read(p[0:3]); string(p[0:3]) != "abc" || err != nil { t.Fatalf("want %q got %q, err=%v", "abc", string(p[0:3]), err) } if s, err := buf.Peek(1); string(s) != "d" || err != nil { t.Fatalf("want %q got %q, err=%v", "d", string(s), err) } if s, err := buf.Peek(2); string(s) != "de" || err != nil { t.Fatalf("want %q got %q, err=%v", "de", string(s), err) } if _, err := buf.Read(p[0:3]); string(p[0:3]) != "def" || err != nil { t.Fatalf("want %q got %q, err=%v", "def", string(p[0:3]), err) } if s, err := buf.Peek(4); string(s) != "ghij" || err != nil { t.Fatalf("want %q got %q, err=%v", "ghij", string(s), err) } if _, err := buf.Read(p[0:]); string(p[0:]) != "ghijklmnop" || err != nil { t.Fatalf("want %q got %q, err=%v", "ghijklmnop", string(p[0:minReadBufferSize]), err) } if s, err := buf.Peek(0); string(s) != "" || err != nil { t.Fatalf("want %q got %q, err=%v", "", string(s), err) } if _, err := buf.Peek(1); err != io.EOF { t.Fatalf("want EOF got %v", err) } // Test for issue 3022, not exposing a reader's error on a successful Peek. buf = NewReaderSize(dataAndEOFReader("abcd"), 32) if s, err := buf.Peek(2); string(s) != "ab" || err != nil { t.Errorf(`Peek(2) on "abcd", EOF = %q, %v; want "ab", nil`, string(s), err) } if s, err := buf.Peek(4); string(s) != "abcd" || err != nil { t.Errorf(`Peek(4) on "abcd", EOF = %q, %v; want "abcd", nil`, string(s), err) } if n, err := buf.Read(p[0:5]); string(p[0:n]) != "abcd" || err != nil { t.Fatalf("Read after peek = %q, %v; want abcd, EOF", p[0:n], err) } if n, err := buf.Read(p[0:1]); string(p[0:n]) != "" || err != io.EOF { t.Fatalf(`second Read after peek = %q, %v; want "", EOF`, p[0:n], err) } } type dataAndEOFReader string func (r dataAndEOFReader) Read(p []byte) (int, error) { return copy(p, r), io.EOF } var testOutput = []byte("0123456789abcdefghijklmnopqrstuvwxy") var testInput = []byte("012\n345\n678\n9ab\ncde\nfgh\nijk\nlmn\nopq\nrst\nuvw\nxy") var testInputrn = []byte("012\r\n345\r\n678\r\n9ab\r\ncde\r\nfgh\r\nijk\r\nlmn\r\nopq\r\nrst\r\nuvw\r\nxy\r\n\n\r\n") // TestReader wraps a []byte and returns reads of a specific length. type testReader struct { data []byte stride int } func (t *testReader) Read(buf []byte) (n int, err error) { n = t.stride if n > len(t.data) { n = len(t.data) } if n > len(buf) { n = len(buf) } copy(buf, t.data) t.data = t.data[n:] if len(t.data) == 0 { err = io.EOF } return } func testReadLine(t *testing.T, input []byte) { //for stride := 1; stride < len(input); stride++ { for stride := 1; stride < 2; stride++ { done := 0 reader := testReader{input, stride} l := NewReaderSize(&reader, len(input)+1) for { line, isPrefix, err := l.ReadLine() if len(line) > 0 && err != nil { t.Errorf("ReadLine returned both data and error: %s", err) } if isPrefix { t.Errorf("ReadLine returned prefix") } if err != nil { if err != io.EOF { t.Fatalf("Got unknown error: %s", err) } break } if want := testOutput[done : done+len(line)]; !bytes.Equal(want, line) { t.Errorf("Bad line at stride %d: want: %x got: %x", stride, want, line) } done += len(line) } if done != len(testOutput) { t.Errorf("ReadLine didn't return everything: got: %d, want: %d (stride: %d)", done, len(testOutput), stride) } } } func TestReadLine(t *testing.T) { testReadLine(t, testInput) testReadLine(t, testInputrn) } func TestLineTooLong(t *testing.T) { data := make([]byte, 0) for i := 0; i < minReadBufferSize*5/2; i++ { data = append(data, '0'+byte(i%10)) } buf := bytes.NewReader(data) l := NewReaderSize(buf, minReadBufferSize) line, isPrefix, err := l.ReadLine() if !isPrefix || !bytes.Equal(line, data[:minReadBufferSize]) || err != nil { t.Errorf("bad result for first line: got %q want %q %v", line, data[:minReadBufferSize], err) } data = data[len(line):] line, isPrefix, err = l.ReadLine() if !isPrefix || !bytes.Equal(line, data[:minReadBufferSize]) || err != nil { t.Errorf("bad result for second line: got %q want %q %v", line, data[:minReadBufferSize], err) } data = data[len(line):] line, isPrefix, err = l.ReadLine() if isPrefix || !bytes.Equal(line, data[:minReadBufferSize/2]) || err != nil { t.Errorf("bad result for third line: got %q want %q %v", line, data[:minReadBufferSize/2], err) } line, isPrefix, err = l.ReadLine() if isPrefix || err == nil { t.Errorf("expected no more lines: %x %s", line, err) } } func TestReadAfterLines(t *testing.T) { line1 := "this is line1" restData := "this is line2\nthis is line 3\n" inbuf := bytes.NewReader([]byte(line1 + "\n" + restData)) outbuf := new(bytes.Buffer) maxLineLength := len(line1) + len(restData)/2 l := NewReaderSize(inbuf, maxLineLength) line, isPrefix, err := l.ReadLine() if isPrefix || err != nil || string(line) != line1 { t.Errorf("bad result for first line: isPrefix=%v err=%v line=%q", isPrefix, err, string(line)) } n, err := io.Copy(outbuf, l) if int(n) != len(restData) || err != nil { t.Errorf("bad result for Read: n=%d err=%v", n, err) } if outbuf.String() != restData { t.Errorf("bad result for Read: got %q; expected %q", outbuf.String(), restData) } } func TestReadEmptyBuffer(t *testing.T) { l := NewReaderSize(new(bytes.Buffer), minReadBufferSize) line, isPrefix, err := l.ReadLine() if err != io.EOF { t.Errorf("expected EOF from ReadLine, got '%s' %t %s", line, isPrefix, err) } } func TestLinesAfterRead(t *testing.T) { l := NewReaderSize(bytes.NewReader([]byte("foo")), minReadBufferSize) _, err := ioutil.ReadAll(l) if err != nil { t.Error(err) return } line, isPrefix, err := l.ReadLine() if err != io.EOF { t.Errorf("expected EOF from ReadLine, got '%s' %t %s", line, isPrefix, err) } } func TestReadLineNonNilLineOrError(t *testing.T) { r := NewReader(strings.NewReader("line 1\n")) for i := 0; i < 2; i++ { l, _, err := r.ReadLine() if l != nil && err != nil { t.Fatalf("on line %d/2; ReadLine=%#v, %v; want non-nil line or Error, but not both", i+1, l, err) } } } type readLineResult struct { line []byte isPrefix bool err error } var readLineNewlinesTests = []struct { input string expect []readLineResult }{ {"012345678901234\r\n012345678901234\r\n", []readLineResult{ {[]byte("012345678901234"), true, nil}, {nil, false, nil}, {[]byte("012345678901234"), true, nil}, {nil, false, nil}, {nil, false, io.EOF}, }}, {"0123456789012345\r012345678901234\r", []readLineResult{ {[]byte("0123456789012345"), true, nil}, {[]byte("\r012345678901234"), true, nil}, {[]byte("\r"), false, nil}, {nil, false, io.EOF}, }}, } func TestReadLineNewlines(t *testing.T) { for _, e := range readLineNewlinesTests { testReadLineNewlines(t, e.input, e.expect) } } func testReadLineNewlines(t *testing.T, input string, expect []readLineResult) { b := NewReaderSize(strings.NewReader(input), minReadBufferSize) for i, e := range expect { line, isPrefix, err := b.ReadLine() if !bytes.Equal(line, e.line) { t.Errorf("%q call %d, line == %q, want %q", input, i, line, e.line) return } if isPrefix != e.isPrefix { t.Errorf("%q call %d, isPrefix == %v, want %v", input, i, isPrefix, e.isPrefix) return } if err != e.err { t.Errorf("%q call %d, err == %v, want %v", input, i, err, e.err) return } } } // TestWriterReadFromCounts tests that using io.Copy to copy into a // bufio.Writer does not prematurely flush the buffer. For example, when // buffering writes to a network socket, excessive network writes should be // avoided. func TestWriterReadFromCounts(t *testing.T) { var w0 writeCountingDiscard b0 := NewWriterSize(&w0, 1234) _, _ = b0.WriteString(strings.Repeat("x", 1000)) if w0 != 0 { t.Fatalf("write 1000 'x's: got %d writes, want 0", w0) } _, _ = b0.WriteString(strings.Repeat("x", 200)) if w0 != 0 { t.Fatalf("write 1200 'x's: got %d writes, want 0", w0) } _, _ = io.Copy(b0, onlyReader{strings.NewReader(strings.Repeat("x", 30))}) if w0 != 0 { t.Fatalf("write 1230 'x's: got %d writes, want 0", w0) } _, _ = io.Copy(b0, onlyReader{strings.NewReader(strings.Repeat("x", 9))}) if w0 != 1 { t.Fatalf("write 1239 'x's: got %d writes, want 1", w0) } var w1 writeCountingDiscard b1 := NewWriterSize(&w1, 1234) _, _ = b1.WriteString(strings.Repeat("x", 1200)) _ = b1.Flush() if w1 != 1 { t.Fatalf("flush 1200 'x's: got %d writes, want 1", w1) } _, _ = b1.WriteString(strings.Repeat("x", 89)) if w1 != 1 { t.Fatalf("write 1200 + 89 'x's: got %d writes, want 1", w1) } _, _ = io.Copy(b1, onlyReader{strings.NewReader(strings.Repeat("x", 700))}) if w1 != 1 { t.Fatalf("write 1200 + 789 'x's: got %d writes, want 1", w1) } _, _ = io.Copy(b1, onlyReader{strings.NewReader(strings.Repeat("x", 600))}) if w1 != 2 { t.Fatalf("write 1200 + 1389 'x's: got %d writes, want 2", w1) } _ = b1.Flush() if w1 != 3 { t.Fatalf("flush 1200 + 1389 'x's: got %d writes, want 3", w1) } } // A writeCountingDiscard is like ioutil.Discard and counts the number of times // Write is called on it. type writeCountingDiscard int func (w *writeCountingDiscard) Write(p []byte) (int, error) { *w++ return len(p), nil } type negativeReader int func (r *negativeReader) Read([]byte) (int, error) { return -1, nil } func TestNegativeRead(t *testing.T) { // should panic with a description pointing at the reader, not at itself. // (should NOT panic with slice index error, for example.) b := NewReader(new(negativeReader)) defer func() { switch err := recover().(type) { case nil: t.Fatal("read did not panic") case error: if !strings.Contains(err.Error(), "reader returned negative count from Read") { t.Fatalf("wrong panic: %v", err) } default: t.Fatalf("unexpected panic value: %T(%v)", err, err) } }() _, _ = b.Read(make([]byte, 100)) } var errFake = errors.New("fake error") type errorThenGoodReader struct { didErr bool nread int } func (r *errorThenGoodReader) Read(p []byte) (int, error) { r.nread++ if !r.didErr { r.didErr = true return 0, errFake } return len(p), nil } func TestReaderClearError(t *testing.T) { r := &errorThenGoodReader{} b := NewReader(r) buf := make([]byte, 1) if _, err := b.Read(nil); err != nil { t.Fatalf("1st nil Read = %v; want nil", err) } if _, err := b.Read(buf); err != errFake { t.Fatalf("1st Read = %v; want errFake", err) } if _, err := b.Read(nil); err != nil { t.Fatalf("2nd nil Read = %v; want nil", err) } if _, err := b.Read(buf); err != nil { t.Fatalf("3rd Read with buffer = %v; want nil", err) } if r.nread != 2 { t.Errorf("num reads = %d; want 2", r.nread) } } func TestReaderReset(t *testing.T) { r := NewReader(strings.NewReader("foo foo")) buf := make([]byte, 3) _, _ = r.Read(buf) if string(buf) != "foo" { t.Errorf("buf = %q; want foo", buf) } r.Reset(strings.NewReader("bar bar")) all, err := ioutil.ReadAll(r) if err != nil { t.Fatal(err) } if string(all) != "bar bar" { t.Errorf("ReadAll = %q; want bar bar", all) } } func TestWriterReset(t *testing.T) { var buf1, buf2 bytes.Buffer w := NewWriter(&buf1) _, _ = w.WriteString("foo") w.Reset(&buf2) // and not flushed _, _ = w.WriteString("bar") _ = w.Flush() if buf1.String() != "" { t.Errorf("buf1 = %q; want empty", buf1.String()) } if buf2.String() != "bar" { t.Errorf("buf2 = %q; want bar", buf2.String()) } } func TestReaderDiscard(t *testing.T) { tests := []struct { name string r io.Reader bufSize int // 0 means 16 peekSize int n int // input to Discard want int // from Discard wantErr error // from Discard wantBuffered int }{ { name: "normal case", r: strings.NewReader("abcdefghijklmnopqrstuvwxyz"), peekSize: 16, n: 6, want: 6, wantBuffered: 10, }, { name: "discard causing read", r: strings.NewReader("abcdefghijklmnopqrstuvwxyz"), n: 6, want: 6, wantBuffered: 10, }, { name: "discard all without peek", r: strings.NewReader("abcdefghijklmnopqrstuvwxyz"), n: 26, want: 26, wantBuffered: 0, }, { name: "discard more than end", r: strings.NewReader("abcdefghijklmnopqrstuvwxyz"), n: 27, want: 26, wantErr: io.EOF, wantBuffered: 0, }, // Any error from filling shouldn't show up until we // get past the valid bytes. Here we return we return 5 valid bytes at the same time // as an error, but test that we don't see the error from Discard. { name: "fill error, discard less", r: newScriptedReader(func(p []byte) (n int, err error) { if len(p) < 5 { panic("unexpected small read") } return 5, errors.New("5-then-error") }), n: 4, want: 4, wantErr: nil, wantBuffered: 1, }, { name: "fill error, discard equal", r: newScriptedReader(func(p []byte) (n int, err error) { if len(p) < 5 { panic("unexpected small read") } return 5, errors.New("5-then-error") }), n: 5, want: 5, wantErr: nil, wantBuffered: 0, }, { name: "fill error, discard more", r: newScriptedReader(func(p []byte) (n int, err error) { if len(p) < 5 { panic("unexpected small read") } return 5, errors.New("5-then-error") }), n: 6, want: 5, wantErr: errors.New("5-then-error"), wantBuffered: 0, }, // Discard of 0 shouldn't cause a read: { name: "discard zero", r: newScriptedReader(), // will panic on Read n: 0, want: 0, wantErr: nil, wantBuffered: 0, }, { name: "discard negative", r: newScriptedReader(), // will panic on Read n: -1, want: 0, wantErr: ErrNegativeCount, wantBuffered: 0, }, } for _, tt := range tests { br := NewReaderSize(tt.r, tt.bufSize) if tt.peekSize > 0 { peekBuf, err := br.Peek(tt.peekSize) if err != nil { t.Errorf("%s: Peek(%d): %v", tt.name, tt.peekSize, err) continue } if len(peekBuf) != tt.peekSize { t.Errorf("%s: len(Peek(%d)) = %v; want %v", tt.name, tt.peekSize, len(peekBuf), tt.peekSize) continue } } discarded, err := br.Discard(tt.n) if ge, we := fmt.Sprint(err), fmt.Sprint(tt.wantErr); discarded != tt.want || ge != we { t.Errorf("%s: Discard(%d) = (%v, %v); want (%v, %v)", tt.name, tt.n, discarded, ge, tt.want, we) continue } if bn := br.Buffered(); bn != tt.wantBuffered { t.Errorf("%s: after Discard, Buffered = %d; want %d", tt.name, bn, tt.wantBuffered) } } } // An onlyReader only implements io.Reader, no matter what other methods the underlying implementation may have. type onlyReader struct { io.Reader } // An onlyWriter only implements io.Writer, no matter what other methods the underlying implementation may have. type onlyWriter struct { io.Writer } // A scriptedReader is an io.Reader that executes its steps sequentially. type scriptedReader []func(p []byte) (n int, err error) func (sr *scriptedReader) Read(p []byte) (n int, err error) { if len(*sr) == 0 { panic("too many Read calls on scripted Reader. No steps remain.") } step := (*sr)[0] *sr = (*sr)[1:] return step(p) } func newScriptedReader(steps ...func(p []byte) (n int, err error)) io.Reader { sr := scriptedReader(steps) return &sr } func BenchmarkReaderCopyOptimal(b *testing.B) { // Optimal case is where the underlying reader implements io.WriterTo srcBuf := bytes.NewBuffer(make([]byte, 8192)) src := NewReader(srcBuf) dstBuf := new(bytes.Buffer) dst := onlyWriter{dstBuf} for i := 0; i < b.N; i++ { srcBuf.Reset() src.Reset(srcBuf) dstBuf.Reset() _, _ = io.Copy(dst, src) } } func BenchmarkReaderCopyUnoptimal(b *testing.B) { // Unoptimal case is where the underlying reader doesn't implement io.WriterTo srcBuf := bytes.NewBuffer(make([]byte, 8192)) src := NewReader(onlyReader{srcBuf}) dstBuf := new(bytes.Buffer) dst := onlyWriter{dstBuf} for i := 0; i < b.N; i++ { srcBuf.Reset() src.Reset(onlyReader{srcBuf}) dstBuf.Reset() _, _ = io.Copy(dst, src) } } func BenchmarkReaderCopyNoWriteTo(b *testing.B) { srcBuf := bytes.NewBuffer(make([]byte, 8192)) srcReader := NewReader(srcBuf) src := onlyReader{srcReader} dstBuf := new(bytes.Buffer) dst := onlyWriter{dstBuf} for i := 0; i < b.N; i++ { srcBuf.Reset() srcReader.Reset(srcBuf) dstBuf.Reset() _, _ = io.Copy(dst, src) } } func BenchmarkWriterCopyOptimal(b *testing.B) { // Optimal case is where the underlying writer implements io.ReaderFrom srcBuf := bytes.NewBuffer(make([]byte, 8192)) src := onlyReader{srcBuf} dstBuf := new(bytes.Buffer) dst := NewWriter(dstBuf) for i := 0; i < b.N; i++ { srcBuf.Reset() dstBuf.Reset() dst.Reset(dstBuf) _, _ = io.Copy(dst, src) } } func BenchmarkWriterCopyUnoptimal(b *testing.B) { srcBuf := bytes.NewBuffer(make([]byte, 8192)) src := onlyReader{srcBuf} dstBuf := new(bytes.Buffer) dst := NewWriter(onlyWriter{dstBuf}) for i := 0; i < b.N; i++ { srcBuf.Reset() dstBuf.Reset() dst.Reset(onlyWriter{dstBuf}) _, _ = io.Copy(dst, src) } } func BenchmarkWriterCopyNoReadFrom(b *testing.B) { srcBuf := bytes.NewBuffer(make([]byte, 8192)) src := onlyReader{srcBuf} dstBuf := new(bytes.Buffer) dstWriter := NewWriter(dstBuf) dst := onlyWriter{dstWriter} for i := 0; i < b.N; i++ { srcBuf.Reset() dstBuf.Reset() dstWriter.Reset(dstBuf) _, _ = io.Copy(dst, src) } } func BenchmarkReaderEmpty(b *testing.B) { b.ReportAllocs() str := strings.Repeat("x", 16<<10) for i := 0; i < b.N; i++ { br := NewReader(strings.NewReader(str)) n, err := io.Copy(ioutil.Discard, br) if err != nil { b.Fatal(err) } if n != int64(len(str)) { b.Fatal("wrong length") } } } func BenchmarkWriterEmpty(b *testing.B) { b.ReportAllocs() str := strings.Repeat("x", 1<<10) bs := []byte(str) for i := 0; i < b.N; i++ { bw := NewWriter(ioutil.Discard) _ = bw.Flush() _ = bw.Flush() _, _ = bw.Write(bs) _ = bw.Flush() _, _ = bw.WriteString(str) _ = bw.Flush() } } func BenchmarkWriterFlush(b *testing.B) { b.ReportAllocs() bw := NewWriter(ioutil.Discard) str := strings.Repeat("x", 50) for i := 0; i < b.N; i++ { _, _ = bw.WriteString(str) _ = bw.Flush() } } ================================================ FILE: pkg/bytes/buffer.go ================================================ package bytes import ( "sync" ) // Buffer buffer. type Buffer struct { buf []byte next *Buffer // next free buffer } // Bytes bytes. func (b *Buffer) Bytes() []byte { return b.buf } // Pool is a buffer pool. type Pool struct { lock sync.Mutex free *Buffer max int num int size int } // NewPool new a memory buffer pool struct. func NewPool(num, size int) (p *Pool) { p = new(Pool) p.init(num, size) return } // Init init the memory buffer. func (p *Pool) Init(num, size int) { p.init(num, size) } // init init the memory buffer. func (p *Pool) init(num, size int) { p.num = num p.size = size p.max = num * size p.grow() } // grow grow the memory buffer size, and update free pointer. func (p *Pool) grow() { var ( i int b *Buffer bs []Buffer buf []byte ) buf = make([]byte, p.max) bs = make([]Buffer, p.num) p.free = &bs[0] b = p.free for i = 1; i < p.num; i++ { b.buf = buf[(i-1)*p.size : i*p.size] b.next = &bs[i] b = b.next } b.buf = buf[(i-1)*p.size : i*p.size] b.next = nil } // Get get a free memory buffer. func (p *Pool) Get() (b *Buffer) { p.lock.Lock() if b = p.free; b == nil { p.grow() b = p.free } p.free = b.next p.lock.Unlock() return } // Put put back a memory buffer to free. func (p *Pool) Put(b *Buffer) { p.lock.Lock() b.next = p.free p.free = b p.lock.Unlock() } ================================================ FILE: pkg/bytes/buffer_test.go ================================================ package bytes import ( "testing" ) func TestBuffer(t *testing.T) { p := NewPool(2, 10) b := p.Get() if b.Bytes() == nil || len(b.Bytes()) == 0 { t.FailNow() } b = p.Get() if b.Bytes() == nil || len(b.Bytes()) == 0 { t.FailNow() } b = p.Get() if b.Bytes() == nil || len(b.Bytes()) == 0 { t.FailNow() } } ================================================ FILE: pkg/bytes/writer.go ================================================ package bytes // Writer writer. type Writer struct { n int buf []byte } // NewWriterSize new a writer with size. func NewWriterSize(n int) *Writer { return &Writer{buf: make([]byte, n)} } // Len buff len. func (w *Writer) Len() int { return w.n } // Size buff cap. func (w *Writer) Size() int { return len(w.buf) } // Reset reset the buff. func (w *Writer) Reset() { w.n = 0 } // Buffer return buff. func (w *Writer) Buffer() []byte { return w.buf[:w.n] } // Peek peek a buf. func (w *Writer) Peek(n int) []byte { var buf []byte w.grow(n) buf = w.buf[w.n : w.n+n] w.n += n return buf } // Write write a buff. func (w *Writer) Write(p []byte) { w.grow(len(p)) w.n += copy(w.buf[w.n:], p) } func (w *Writer) grow(n int) { var buf []byte if w.n+n < len(w.buf) { return } buf = make([]byte, 2*len(w.buf)+n) copy(buf, w.buf[:w.n]) w.buf = buf } ================================================ FILE: pkg/bytes/writer_test.go ================================================ package bytes import ( "reflect" "testing" ) func TestWriter(t *testing.T) { w := NewWriterSize(64) if w.Len() != 0 && w.Size() != 64 { t.FailNow() } b := []byte("hello") w.Write(b) if !reflect.DeepEqual(b, w.Buffer()) { t.FailNow() } w.Peek(len(b)) w.Reset() for i := 0; i < 1024; i++ { w.Write(b) } w.Reset() if w.Len() != 0 { t.FailNow() } } ================================================ FILE: pkg/encoding/binary/endian.go ================================================ package binary // BigEndian big endian. var BigEndian bigEndian type bigEndian struct{} func (bigEndian) Int8(b []byte) int8 { return int8(b[0]) } func (bigEndian) PutInt8(b []byte, v int8) { b[0] = byte(v) } func (bigEndian) Int16(b []byte) int16 { return int16(b[1]) | int16(b[0])<<8 } func (bigEndian) PutInt16(b []byte, v int16) { _ = b[1] b[0] = byte(v >> 8) b[1] = byte(v) } func (bigEndian) Int32(b []byte) int32 { return int32(b[3]) | int32(b[2])<<8 | int32(b[1])<<16 | int32(b[0])<<24 } func (bigEndian) PutInt32(b []byte, v int32) { _ = b[3] b[0] = byte(v >> 24) b[1] = byte(v >> 16) b[2] = byte(v >> 8) b[3] = byte(v) } ================================================ FILE: pkg/encoding/binary/endian_test.go ================================================ package binary import "testing" func TestInt8(t *testing.T) { b := make([]byte, 1) BigEndian.PutInt8(b, 100) i := BigEndian.Int8(b) if i != 100 { t.FailNow() } } func TestInt16(t *testing.T) { b := make([]byte, 2) BigEndian.PutInt16(b, 100) i := BigEndian.Int16(b) if i != 100 { t.FailNow() } } func TestInt32(t *testing.T) { b := make([]byte, 4) BigEndian.PutInt32(b, 100) i := BigEndian.Int32(b) if i != 100 { t.FailNow() } } ================================================ FILE: pkg/ip/ip.go ================================================ package ip import ( "net" "strings" ) // InternalIP return internal ip. func InternalIP() string { inters, err := net.Interfaces() if err != nil { return "" } for _, inter := range inters { if inter.Flags&net.FlagUp != 0 && !strings.HasPrefix(inter.Name, "lo") { addrs, err := inter.Addrs() if err != nil { continue } for _, addr := range addrs { if ipnet, ok := addr.(*net.IPNet); ok && !ipnet.IP.IsLoopback() { if ipnet.IP.To4() != nil { return ipnet.IP.String() } } } } } return "" } ================================================ FILE: pkg/ip/ip_test.go ================================================ package ip import "testing" func TestIP(t *testing.T) { ip := InternalIP() if ip == "" { t.FailNow() } } ================================================ FILE: pkg/strings/ints.go ================================================ package strings import ( "bytes" "strconv" "strings" "sync" ) var ( bfPool = sync.Pool{ New: func() interface{} { return bytes.NewBuffer([]byte{}) }, } ) // JoinInt32s format int32 slice like:n1,n2,n3. func JoinInt32s(is []int32, p string) string { if len(is) == 0 { return "" } if len(is) == 1 { return strconv.FormatInt(int64(is[0]), 10) } buf := bfPool.Get().(*bytes.Buffer) for _, i := range is { buf.WriteString(strconv.FormatInt(int64(i), 10)) buf.WriteString(p) } if buf.Len() > 0 { buf.Truncate(buf.Len() - 1) } s := buf.String() buf.Reset() bfPool.Put(buf) return s } // SplitInt32s split string into int32 slice. func SplitInt32s(s, p string) ([]int32, error) { if s == "" { return nil, nil } sArr := strings.Split(s, p) res := make([]int32, 0, len(sArr)) for _, sc := range sArr { i, err := strconv.ParseInt(sc, 10, 32) if err != nil { return nil, err } res = append(res, int32(i)) } return res, nil } // JoinInt64s format int64 slice like:n1,n2,n3. func JoinInt64s(is []int64, p string) string { if len(is) == 0 { return "" } if len(is) == 1 { return strconv.FormatInt(is[0], 10) } buf := bfPool.Get().(*bytes.Buffer) for _, i := range is { buf.WriteString(strconv.FormatInt(i, 10)) buf.WriteString(p) } if buf.Len() > 0 { buf.Truncate(buf.Len() - 1) } s := buf.String() buf.Reset() bfPool.Put(buf) return s } // SplitInt64s split string into int64 slice. func SplitInt64s(s, p string) ([]int64, error) { if s == "" { return nil, nil } sArr := strings.Split(s, p) res := make([]int64, 0, len(sArr)) for _, sc := range sArr { i, err := strconv.ParseInt(sc, 10, 64) if err != nil { return nil, err } res = append(res, i) } return res, nil } ================================================ FILE: pkg/strings/ints_test.go ================================================ package strings import ( "reflect" "testing" ) func TestInt32(t *testing.T) { i := []int32{1, 2, 3} s := JoinInt32s(i, ",") ii, _ := SplitInt32s(s, ",") if !reflect.DeepEqual(i, ii) { t.FailNow() } } func TestInt64(t *testing.T) { i := []int64{1, 2, 3} s := JoinInt64s(i, ",") ii, _ := SplitInt64s(s, ",") if !reflect.DeepEqual(i, ii) { t.FailNow() } } ================================================ FILE: pkg/time/debug.go ================================================ package time const ( // Debug debug switch Debug = false ) ================================================ FILE: pkg/time/duration.go ================================================ package time import ( xtime "time" ) // Duration be used toml unmarshal string time, like 1s, 500ms. type Duration xtime.Duration // UnmarshalText unmarshal text to duration. func (d *Duration) UnmarshalText(text []byte) error { tmp, err := xtime.ParseDuration(string(text)) if err == nil { *d = Duration(tmp) } return err } ================================================ FILE: pkg/time/duration_test.go ================================================ package time import ( "testing" "time" ) func TestDurationText(t *testing.T) { var ( input = []byte("10s") output = time.Second * 10 d Duration ) if err := d.UnmarshalText(input); err != nil { t.FailNow() } if int64(output) != int64(d) { t.FailNow() } } ================================================ FILE: pkg/time/timer.go ================================================ package time import ( "sync" itime "time" log "github.com/golang/glog" ) const ( timerFormat = "2006-01-02 15:04:05" infiniteDuration = itime.Duration(1<<63 - 1) ) // TimerData timer data. type TimerData struct { Key string expire itime.Time fn func() index int next *TimerData } // Delay delay duration. func (td *TimerData) Delay() itime.Duration { return itime.Until(td.expire) } // ExpireString expire string. func (td *TimerData) ExpireString() string { return td.expire.Format(timerFormat) } // Timer timer. type Timer struct { lock sync.Mutex free *TimerData timers []*TimerData signal *itime.Timer num int } // NewTimer new a timer. // A heap must be initialized before any of the heap operations // can be used. Init is idempotent with respect to the heap invariants // and may be called whenever the heap invariants may have been invalidated. // Its complexity is O(n) where n = h.Len(). // func NewTimer(num int) (t *Timer) { t = new(Timer) t.init(num) return t } // Init init the timer. func (t *Timer) Init(num int) { t.init(num) } func (t *Timer) init(num int) { t.signal = itime.NewTimer(infiniteDuration) t.timers = make([]*TimerData, 0, num) t.num = num t.grow() go t.start() } func (t *Timer) grow() { var ( i int td *TimerData tds = make([]TimerData, t.num) ) t.free = &(tds[0]) td = t.free for i = 1; i < t.num; i++ { td.next = &(tds[i]) td = td.next } td.next = nil } // get get a free timer data. func (t *Timer) get() (td *TimerData) { if td = t.free; td == nil { t.grow() td = t.free } t.free = td.next return } // put put back a timer data. func (t *Timer) put(td *TimerData) { td.fn = nil td.next = t.free t.free = td } // Add add the element x onto the heap. The complexity is // O(log(n)) where n = h.Len(). func (t *Timer) Add(expire itime.Duration, fn func()) (td *TimerData) { t.lock.Lock() td = t.get() td.expire = itime.Now().Add(expire) td.fn = fn t.add(td) t.lock.Unlock() return } // Del removes the element at index i from the heap. // The complexity is O(log(n)) where n = h.Len(). func (t *Timer) Del(td *TimerData) { t.lock.Lock() t.del(td) t.put(td) t.lock.Unlock() } // Push pushes the element x onto the heap. The complexity is // O(log(n)) where n = h.Len(). func (t *Timer) add(td *TimerData) { var d itime.Duration td.index = len(t.timers) // add to the minheap last node t.timers = append(t.timers, td) t.up(td.index) if td.index == 0 { // if first node, signal start goroutine d = td.Delay() t.signal.Reset(d) if Debug { log.Infof("timer: add reset delay %d ms", int64(d)/int64(itime.Millisecond)) } } if Debug { log.Infof("timer: push item key: %s, expire: %s, index: %d", td.Key, td.ExpireString(), td.index) } } func (t *Timer) del(td *TimerData) { var ( i = td.index last = len(t.timers) - 1 ) if i < 0 || i > last || t.timers[i] != td { // already remove, usually by expire if Debug { log.Infof("timer del i: %d, last: %d, %p", i, last, td) } return } if i != last { t.swap(i, last) t.down(i, last) t.up(i) } // remove item is the last node t.timers[last].index = -1 // for safety t.timers = t.timers[:last] if Debug { log.Infof("timer: remove item key: %s, expire: %s, index: %d", td.Key, td.ExpireString(), td.index) } } // Set update timer data. func (t *Timer) Set(td *TimerData, expire itime.Duration) { t.lock.Lock() t.del(td) td.expire = itime.Now().Add(expire) t.add(td) t.lock.Unlock() } // start start the timer. func (t *Timer) start() { for { t.expire() <-t.signal.C } } // expire removes the minimum element (according to Less) from the heap. // The complexity is O(log(n)) where n = max. // It is equivalent to Del(0). func (t *Timer) expire() { var ( fn func() td *TimerData d itime.Duration ) t.lock.Lock() for { if len(t.timers) == 0 { d = infiniteDuration if Debug { log.Info("timer: no other instance") } break } td = t.timers[0] if d = td.Delay(); d > 0 { break } fn = td.fn // let caller put back t.del(td) t.lock.Unlock() if fn == nil { log.Warning("expire timer no fn") } else { if Debug { log.Infof("timer key: %s, expire: %s, index: %d expired, call fn", td.Key, td.ExpireString(), td.index) } fn() } t.lock.Lock() } t.signal.Reset(d) if Debug { log.Infof("timer: expier reset delay %d ms", int64(d)/int64(itime.Millisecond)) } t.lock.Unlock() } func (t *Timer) up(j int) { for { i := (j - 1) / 2 // parent if i >= j || !t.less(j, i) { break } t.swap(i, j) j = i } } func (t *Timer) down(i, n int) { for { j1 := 2*i + 1 if j1 >= n || j1 < 0 { // j1 < 0 after int overflow break } j := j1 // left child if j2 := j1 + 1; j2 < n && !t.less(j1, j2) { j = j2 // = 2*i + 2 // right child } if !t.less(j, i) { break } t.swap(i, j) i = j } } func (t *Timer) less(i, j int) bool { return t.timers[i].expire.Before(t.timers[j].expire) } func (t *Timer) swap(i, j int) { t.timers[i], t.timers[j] = t.timers[j], t.timers[i] t.timers[i].index = i t.timers[j].index = j } ================================================ FILE: pkg/time/timer_test.go ================================================ package time import ( "testing" "time" log "github.com/golang/glog" ) func TestTimer(t *testing.T) { timer := NewTimer(100) tds := make([]*TimerData, 100) for i := 0; i < 100; i++ { tds[i] = timer.Add(time.Duration(i)*time.Second+5*time.Minute, nil) } printTimer(timer) for i := 0; i < 100; i++ { log.Infof("td: %s, %s, %d", tds[i].Key, tds[i].ExpireString(), tds[i].index) timer.Del(tds[i]) } printTimer(timer) for i := 0; i < 100; i++ { tds[i] = timer.Add(time.Duration(i)*time.Second+5*time.Minute, nil) } printTimer(timer) for i := 0; i < 100; i++ { timer.Del(tds[i]) } printTimer(timer) timer.Add(time.Second, nil) time.Sleep(time.Second * 2) if len(timer.timers) != 0 { t.FailNow() } } func printTimer(timer *Timer) { log.Infof("----------timers: %d ----------", len(timer.timers)) for i := 0; i < len(timer.timers); i++ { log.Infof("timer: %s, %s, index: %d", timer.timers[i].Key, timer.timers[i].ExpireString(), timer.timers[i].index) } log.Infof("--------------------") } ================================================ FILE: pkg/websocket/conn.go ================================================ package websocket import ( "encoding/binary" "errors" "fmt" "io" "github.com/Terry-Mao/goim/pkg/bufio" ) const ( // Frame header byte 0 bits from Section 5.2 of RFC 6455 finBit = 1 << 7 rsv1Bit = 1 << 6 rsv2Bit = 1 << 5 rsv3Bit = 1 << 4 opBit = 0x0f // Frame header byte 1 bits from Section 5.2 of RFC 6455 maskBit = 1 << 7 lenBit = 0x7f continuationFrame = 0 continuationFrameMaxRead = 100 ) // The message types are defined in RFC 6455, section 11.8. const ( // TextMessage denotes a text data message. The text message payload is // interpreted as UTF-8 encoded text data. TextMessage = 1 // BinaryMessage denotes a binary data message. BinaryMessage = 2 // CloseMessage denotes a close control message. The optional message // payload contains a numeric code and text. Use the FormatCloseMessage // function to format a close message payload. CloseMessage = 8 // PingMessage denotes a ping control message. The optional message payload // is UTF-8 encoded text. PingMessage = 9 // PongMessage denotes a ping control message. The optional message payload // is UTF-8 encoded text. PongMessage = 10 ) var ( // ErrMessageClose close control message ErrMessageClose = errors.New("close control message") // ErrMessageMaxRead continuation frame max read ErrMessageMaxRead = errors.New("continuation frame max read") ) // Conn represents a WebSocket connection. type Conn struct { rwc io.ReadWriteCloser r *bufio.Reader w *bufio.Writer maskKey []byte } // new connection func newConn(rwc io.ReadWriteCloser, r *bufio.Reader, w *bufio.Writer) *Conn { return &Conn{rwc: rwc, r: r, w: w, maskKey: make([]byte, 4)} } // WriteMessage write a message by type. func (c *Conn) WriteMessage(msgType int, msg []byte) (err error) { if err = c.WriteHeader(msgType, len(msg)); err != nil { return } err = c.WriteBody(msg) return } // WriteHeader write header frame. func (c *Conn) WriteHeader(msgType int, length int) (err error) { var h []byte if h, err = c.w.Peek(2); err != nil { return } // 1.First byte. FIN/RSV1/RSV2/RSV3/OpCode(4bits) h[0] = 0 h[0] |= finBit | byte(msgType) // 2.Second byte. Mask/Payload len(7bits) h[1] = 0 switch { case length <= 125: // 7 bits h[1] |= byte(length) case length < 65536: // 16 bits h[1] |= 126 if h, err = c.w.Peek(2); err != nil { return } binary.BigEndian.PutUint16(h, uint16(length)) default: // 64 bits h[1] |= 127 if h, err = c.w.Peek(8); err != nil { return } binary.BigEndian.PutUint64(h, uint64(length)) } return } // WriteBody write a message body. func (c *Conn) WriteBody(b []byte) (err error) { if len(b) > 0 { _, err = c.w.Write(b) } return } // Peek write peek. func (c *Conn) Peek(n int) ([]byte, error) { return c.w.Peek(n) } // Flush flush writer buffer func (c *Conn) Flush() error { return c.w.Flush() } // ReadMessage read a message. func (c *Conn) ReadMessage() (op int, payload []byte, err error) { var ( fin bool finOp, n int partPayload []byte ) for { // read frame if fin, op, partPayload, err = c.readFrame(); err != nil { return } switch op { case BinaryMessage, TextMessage, continuationFrame: if fin && len(payload) == 0 { return op, partPayload, nil } // continuation frame payload = append(payload, partPayload...) if op != continuationFrame { finOp = op } // final frame if fin { op = finOp return } case PingMessage: // handler ping if err = c.WriteMessage(PongMessage, partPayload); err != nil { return } case PongMessage: // handler pong case CloseMessage: // handler close err = ErrMessageClose return default: err = fmt.Errorf("unknown control message, fin=%t, op=%d", fin, op) return } if n > continuationFrameMaxRead { err = ErrMessageMaxRead return } n++ } } func (c *Conn) readFrame() (fin bool, op int, payload []byte, err error) { var ( b byte p []byte mask bool maskKey []byte payloadLen int64 ) // 1.First byte. FIN/RSV1/RSV2/RSV3/OpCode(4bits) b, err = c.r.ReadByte() if err != nil { return } // final frame fin = (b & finBit) != 0 // rsv MUST be 0 if rsv := b & (rsv1Bit | rsv2Bit | rsv3Bit); rsv != 0 { return false, 0, nil, fmt.Errorf("unexpected reserved bits rsv1=%d, rsv2=%d, rsv3=%d", b&rsv1Bit, b&rsv2Bit, b&rsv3Bit) } // op code op = int(b & opBit) // 2.Second byte. Mask/Payload len(7bits) b, err = c.r.ReadByte() if err != nil { return } // is mask payload mask = (b & maskBit) != 0 // payload length switch b & lenBit { case 126: // 16 bits if p, err = c.r.Pop(2); err != nil { return } payloadLen = int64(binary.BigEndian.Uint16(p)) case 127: // 64 bits if p, err = c.r.Pop(8); err != nil { return } payloadLen = int64(binary.BigEndian.Uint64(p)) default: // 7 bits payloadLen = int64(b & lenBit) } // read mask key if mask { maskKey, err = c.r.Pop(4) if err != nil { return } if c.maskKey == nil { c.maskKey = make([]byte, 4) } copy(c.maskKey, maskKey) } // read payload if payloadLen > 0 { if payload, err = c.r.Pop(int(payloadLen)); err != nil { return } if mask { maskBytes(c.maskKey, 0, payload) } } return } // Close close the connection. func (c *Conn) Close() error { return c.rwc.Close() } func maskBytes(key []byte, pos int, b []byte) int { for i := range b { b[i] ^= key[pos&3] pos++ } return pos & 3 } ================================================ FILE: pkg/websocket/request.go ================================================ package websocket import ( "bytes" "fmt" "net/http" "strings" "github.com/Terry-Mao/goim/pkg/bufio" ) // Request request. type Request struct { Method string RequestURI string Proto string Host string Header http.Header reader *bufio.Reader } // ReadRequest reads and parses an incoming request from b. func ReadRequest(r *bufio.Reader) (req *Request, err error) { var ( b []byte ok bool ) req = &Request{reader: r} if b, err = req.readLine(); err != nil { return } if req.Method, req.RequestURI, req.Proto, ok = parseRequestLine(string(b)); !ok { return nil, fmt.Errorf("malformed HTTP request %s", b) } if req.Header, err = req.readMIMEHeader(); err != nil { return } req.Host = req.Header.Get("Host") return req, nil } func (r *Request) readLine() ([]byte, error) { var line []byte for { l, more, err := r.reader.ReadLine() if err != nil { return nil, err } // Avoid the copy if the first call produced a full line. if line == nil && !more { return l, nil } line = append(line, l...) if !more { break } } return line, nil } func (r *Request) readMIMEHeader() (header http.Header, err error) { var ( line []byte i int k, v string ) header = make(http.Header, 16) for { if line, err = r.readLine(); err != nil { return } line = trim(line) if len(line) == 0 { return } if i = bytes.IndexByte(line, ':'); i <= 0 { err = fmt.Errorf("malformed MIME header line: " + string(line)) return } k = string(line[:i]) // Skip initial spaces in value. i++ // skip colon for i < len(line) && (line[i] == ' ' || line[i] == '\t') { i++ } v = string(line[i:]) header.Add(k, v) } } // parseRequestLine parses "GET /foo HTTP/1.1" into its three parts. func parseRequestLine(line string) (method, requestURI, proto string, ok bool) { s1 := strings.Index(line, " ") s2 := strings.Index(line[s1+1:], " ") if s1 < 0 || s2 < 0 { return } s2 += s1 + 1 return line[:s1], line[s1+1 : s2], line[s2+1:], true } // trim returns s with leading and trailing spaces and tabs removed. // It does not assume Unicode or UTF-8. func trim(s []byte) []byte { i := 0 for i < len(s) && (s[i] == ' ' || s[i] == '\t') { i++ } n := len(s) for n > i && (s[n-1] == ' ' || s[n-1] == '\t') { n-- } return s[i:n] } ================================================ FILE: pkg/websocket/server.go ================================================ package websocket import ( "crypto/sha1" "encoding/base64" "errors" "io" "strings" "github.com/Terry-Mao/goim/pkg/bufio" ) var ( keyGUID = []byte("258EAFA5-E914-47DA-95CA-C5AB0DC85B11") // ErrBadRequestMethod bad request method ErrBadRequestMethod = errors.New("bad method") // ErrNotWebSocket not websocket protocal ErrNotWebSocket = errors.New("not websocket protocol") // ErrBadWebSocketVersion bad websocket version ErrBadWebSocketVersion = errors.New("missing or bad WebSocket Version") // ErrChallengeResponse mismatch challenge response ErrChallengeResponse = errors.New("mismatch challenge/response") ) // Upgrade Switching Protocols func Upgrade(rwc io.ReadWriteCloser, rr *bufio.Reader, wr *bufio.Writer, req *Request) (conn *Conn, err error) { if req.Method != "GET" { return nil, ErrBadRequestMethod } if req.Header.Get("Sec-Websocket-Version") != "13" { return nil, ErrBadWebSocketVersion } if strings.ToLower(req.Header.Get("Upgrade")) != "websocket" { return nil, ErrNotWebSocket } if !strings.Contains(strings.ToLower(req.Header.Get("Connection")), "upgrade") { return nil, ErrNotWebSocket } challengeKey := req.Header.Get("Sec-Websocket-Key") if challengeKey == "" { return nil, ErrChallengeResponse } _, _ = wr.WriteString("HTTP/1.1 101 Switching Protocols\r\nUpgrade: websocket\r\nConnection: Upgrade\r\n") _, _ = wr.WriteString("Sec-WebSocket-Accept: " + computeAcceptKey(challengeKey) + "\r\n\r\n") if err = wr.Flush(); err != nil { return } return newConn(rwc, rr, wr), nil } func computeAcceptKey(challengeKey string) string { h := sha1.New() _, _ = h.Write([]byte(challengeKey)) _, _ = h.Write(keyGUID) return base64.StdEncoding.EncodeToString(h.Sum(nil)) } ================================================ FILE: pkg/websocket/server_test.go ================================================ package websocket import ( "net" "reflect" "testing" "time" "golang.org/x/net/websocket" "github.com/Terry-Mao/goim/pkg/bufio" ) func TestServer(t *testing.T) { var ( data = []byte{0, 1, 2} ) ln, err := net.Listen("tcp", ":8080") if err != nil { t.FailNow() } go func() { conn, err := ln.Accept() if err != nil { t.Error(err) } rd := bufio.NewReader(conn) wr := bufio.NewWriter(conn) req, err := ReadRequest(rd) if err != nil { t.Error(err) } if req.RequestURI != "/sub" { t.Error(err) } ws, err := Upgrade(conn, rd, wr, req) if err != nil { t.Error(err) } if err = ws.WriteMessage(BinaryMessage, data); err != nil { t.Error(err) } if err = ws.Flush(); err != nil { t.Error(err) } op, b, err := ws.ReadMessage() if err != nil || op != BinaryMessage || !reflect.DeepEqual(b, data) { t.Error(err) } }() time.Sleep(time.Millisecond * 100) // ws client ws, err := websocket.Dial("ws://127.0.0.1:8080/sub", "", "*") if err != nil { t.FailNow() } // receive binary frame var b []byte if err = websocket.Message.Receive(ws, &b); err != nil { t.FailNow() } if !reflect.DeepEqual(b, data) { t.FailNow() } // send binary frame if err = websocket.Message.Send(ws, data); err != nil { t.FailNow() } } ================================================ FILE: scripts/README.md ================================================ # Installing ### Install JDK ``` ./jdk.sh ``` ### Install Zookeeper ``` ./zk.sh ``` ### Install Kafka ``` ./kafka.sh ``` ================================================ FILE: scripts/jdk8.sh ================================================ #!/bin/bash # config parameters JDK_HOME=/usr/local/jdk8/ # download jdk curl -L https://download.oracle.com/otn-pub/java/jdk/8u191-b12/2787e4a523244c269598db4e85c51e0c/jdk-8u191-linux-x64.tar.gz -o jdk-8u191-linux-x64.tar.gz tar zxf jdk-8u191-linux-x64.tar.gz # install jdk mkdir -p $JDK_HOME mv jdk1.8.0/* $JDK_HOME update-alternatives --install "/usr/bin/java" "java" "$JDK_HOME/bin/java" 1500 update-alternatives --install "/usr/bin/javac" "javac" "$JDK_HOME/bin/javac" 1500 update-alternatives --install "/usr/bin/javaws" "javaws" "$JDK_HOME/bin/javaws" 1500 ================================================ FILE: scripts/kafka.sh ================================================ #!/bin/bash # config parameters JDK_HOME=/usr/local/jdk8/ KAFKA_HOME=/data/app/kafka/ KAFKA_CONF=/data/app/kafka/config/server.properties KAFKA_DATA=/data/kafka_data/ KAFKA_SUPERVISOR=/etc/supervisor/conf.d/kafka.conf USAGE="./kafka.sh {broker.id} {current.addr} {zk.addr1,zk.addr2,zk.addr3}\r\neg: ./kafka.sh 1 127.0.0.1 127.0.0.1:2181,127.0.0.2:2181,127.0.0.3:2181" # check parammeters BROKER_ID=$1 CURRENT_ADDR=$2 ZK_ADDRS=$3 if [ ! "$BROKER_ID" ]; then echo "kafka {broker.id} can not be null" echo -e $USAGE exit 1 fi if [ ! "$CURRENT_ADDR" ]; then echo "current addr can not be null" exit 1 fi if [ ! "$ZK_ADDRS" ]; then echo "zookeeper addrs can not be null" exit 1 fi if [ ! -d "$JDK_HOME" ]; then echo "jdk not exist down jdk8" ./jdk8.sh fi # download kafka curl "http://apache.stu.edu.tw/kafka/2.1.0/kafka_2.11-2.1.0.tgz" -o kafka_2.11-2.1.0.tgz tar zxf kafka_2.11-2.1.0.tgz # install kafka mkdir -p $KAFKA_HOME mkdir -p $KAFKA_DATA mv kafka_2.11-2.1.0/* $KAFKA_HOME # kafka config echo "broker.id=$BROKER_ID">$KAFKA_CONF echo "host.name=$CURRENT_ADDR">>$KAFKA_CONF echo "zookeeper.connect=$ZK_ADDRS/kafka">>$KAFKA_CONF echo "num.network.threads=4 num.io.threads=4 socket.send.buffer.bytes=1024000 socket.receive.buffer.bytes=1024000 socket.request.max.bytes=52428800 log.dirs=$KAFKA_DATA num.partitions=9 num.recovery.threads.per.data.dir=1 log.cleanup.policy=delete log.retention.hours=24 log.segment.bytes=536870912 log.retention.check.interval.ms=300000 log.cleaner.enable=false zookeeper.connection.timeout.ms=6000 default.replication.factor=2 delete.topic.enable=false auto.create.topics.enable=false">>$KAFKA_CONF # install supervisor if ! type "supervisorctl" > /dev/null; then apt-get install -y supervisor fi # kafka supervisor config mkdir -p /etc/supervisor/conf.d/ mkdir -p /data/log/kafka/ echo "[program:kafka] command=$KAFKA_HOME/bin/kafka-server-start.sh $KAFKA_CONF user=root autostart=true autorestart=true exitcodes=0 startsecs=10 startretries=10 stopwaitsecs=10 stopsignal=KILL stdout_logfile=/data/log/kafka/stdout.log stderr_logfile=/data/log/kafka/stderr.log stdout_logfile_maxbytes=100MB stdout_logfile_backups=5 stderr_logfile_maxbytes=100MB stderr_logfile_backups=5 environment=JAVA_HOME=$JDK_HOME,JRE_HOME='$JDK_HOME/jre',KAFKA_HEAP_OPTS='-Xmx6g -Xms6g -XX:MetaspaceSize=96m -XX:G1HeapRegionSize=16M -XX:MinMetaspaceFreeRatio=50 -XX:MaxMetaspaceFreeRatio=80'">$_KAFKA_SUPERVISOR supervisorctl update ================================================ FILE: scripts/zk.sh ================================================ #!/bin/bash # config parameters JDK_HOME=/usr/local/jdk8 ZK_HOME=/data/server/zookeeper ZK_SUPERVISOR=/etc/supervisor/conf.d/zookeeper.conf USAGE="./zk.sh {zk.id} {zk.addr1,zk.addr2,zk.addr3}\r\neg:./zk.sh 1 127.0.0.1:2181,127.0.0.2:2181,127.0.0.3:2181" # check parammeters ZK_ID=$1 str=$2 ZK_ADDRS=(${str//,/ }) ZK_ADDRS_SIZE=${#ZK_ADDRS[@]} if [ ! -n "$1" ];then echo -e $USAGE exit 1 fi if [ ! -n "$2" ];then echo -e $USAGE exit 1 fi echo $ZK_ID echo ${ZK_ADDRS[@]} set -e curl -L http://apache.website-solution.net/zookeeper/zookeeper-3.4.12/zookeeper-3.4.12.tar.gz -o zookeeper-3.4.12.tar.gz tar zxf zookeeper-3.4.12.tar.gz mkdir -p $ZK_HOME echo $ZK_ID>$ZK_HOME/myid mv zookeeper-3.4.12/* $ZK_HOME echo "tickTime=2000" > $ZK_HOME/conf/zoo.cfg echo "initLimit=10" >> $ZK_HOME/conf/zoo.cfg echo "syncLimit=5" >> $ZK_HOME/conf/zoo.cfg echo "dataDir=/data/server/zookeeper" >> $ZK_HOME/conf/zoo.cfg echo "clientPort=2181" >> $ZK_HOME/conf/zoo.cfg echo "autopurge.snapRetainCount=5" >> $ZK_HOME/conf/zoo.cfg echo "autopurge.purgeInterval=24" >> $ZK_HOME/conf/zoo.cfg for ((index=0;index<$ZK_ADDRS_SIZE;index++)) do id=$[$index+1] echo "server.$id=${ZK_ADDRS[index]}:2888:3888" >> $ZK_HOME/conf/zoo.cfg done # install supervisor if ! type "supervisorctl" > /dev/null; then apt-get install -y supervisor fi # zookeeper supervisor config mkdir -p /etc/supervisor/conf.d/ mkdir -p /data/log/zookeeper/ echo "[program:zookeeper] command=${ZK_HOME}/bin/zkServer.sh start-foreground directory=${ZK_HOME} user=root autostart=true autorestart=true exitcodes=0 startsecs=10 startretries=10 stopwaitsecs=10 stopsignal=KILL stdout_logfile=/data/log/zookeeper/stdout.log stderr_logfile=/data/log/zookeeper/stderr.log stdout_logfile_maxbytes=100MB stdout_logfile_backups=5 stderr_logfile_maxbytes=100MB stderr_logfile_backups=5 environment=JAVA_HOME=$JDK_HOME,JRE_HOME='$JDK_HOME/jre' ">$ZK_SUPERVISOR supervisorctl update