[ { "path": ".vscode/settings.json", "content": "{\n \"go.inferGopath\": false\n}" }, { "path": "1-etcd环境安装与使用/README.md", "content": "### etcd简介\n[etcd](https://github.com/etcd-io/etcd)是开源的、高可用的分布式key-value存储系统，可用于配置共享和服务的注册和发现，它专注于：\n\n* 简单：定义清晰、面向用户的API（gRPC）\n\n* 安全：可选的客户端TLS证书自动认证\n\n* 快速：支持每秒10,000次写入\n\n* 可靠：基于Raft算法确保强一致性\n\n##### etcd与redis差异\netcd和redis都支持键值存储，也支持分布式特性，redis支持的数据格式更加丰富，但是他们两个定位和应用场景不一样，关键差异如下：\n\n* redis在分布式环境下不是强一致性的，可能会丢失数据，或者读取不到最新数据\n\n* redis的数据变化监听机制没有etcd完善\n\n* etcd强一致性保证数据可靠性，导致性能上要低于redis\n\n* etcd和ZooKeeper是定位类似的项目，跟redis定位不一样\n\n##### 为什么用 etcd 而不用ZooKeeper？\n相较之下，ZooKeeper有如下缺点：\n\n* `复杂`：ZooKeeper的部署维护复杂，管理员需要掌握一系列的知识和技能；而 Paxos 强一致性算法也是素来以复杂难懂而闻名于世；另外，ZooKeeper的使用也比较复杂，需要安装客户端，官方只提供了 Java 和 C 两种语言的接口。\n\n* `难以维护`：Java 编写。这里不是对 Java 有偏见，而是 Java 本身就偏向于重型应用，它会引入大量的依赖。而运维人员则普遍希望保持强一致、高可用的机器集群尽可能简单，维护起来也不易出错。\n\n* `发展缓慢`：Apache 基金会项目特有的“Apache Way”在开源界饱受争议，其中一大原因就是由于基金会庞大的结构以及松散的管理导致项目发展缓慢。\n\n而 etcd 作为一个后起之秀，其优点也很明显。\n\n* `简单`：使用 Go 语言编写部署简单；使用 HTTP 作为接口使用简单；使用 Raft 算法保证强一致性让用户易于理解。\n\n* `数据持久化`：tcd 默认数据一更新就进行持久化。\n\n* `安全`：etcd 支持 SSL 客户端安全认证。\n\n### 单机部署\n\n（1）到etcd的github地址，下载最新的安装包（目前最新版本：v3.4.7）\n\n下载地址：https://github.com/etcd-io/etcd/releases/\n\n（2）解压，把`etcd`和`etcdctl`文件复制到已经配置了环境变量的目录中\n\n* 方法一：把`etcd`和`etcdctl`文件复制到`GOBIN`目录下。\n\n* 方法二：在环境变量里添加`etcd`和`etcdctl`文件所在的目录。\n\n（3）验证是否安装成功\n```\n$ etcd --version\netcd Version: 3.4.7\nGit SHA: e694b7bb0\nGo Version: go1.12.17\nGo OS/Arch: linux/amd64\n```\n\n正常显示etcd版本信息，则证明安装成功。\n\n### API学习\n`etcdctl`用于与`etcd`交互的控制台程序。`API`版本可以通过`ETCDCTL_API`环境变量设置为2或3版本。默认情况下，`v3.4`以上的`etcdctl`使用`v3 API`，`v3.3`及更早的版本默认使用`v2 API`。\n\n> 注意：用`v2 API`创建的任何key将不能通过`v3 API`查询。同样，用`v3 API`创建的任何key将不能通过`v2 API`查询。\n\n运行etcd，在终端输入：`etcd`\n\n在另一个终端运行ctcdctl测试。\n```\n#查看默认API版本\n$ etcdctl version \netcdctl version: 3.4.7\nAPI version: 3.4 #v3 API\n\n#写入key:/test/foo value:hello etcd (双引号可去掉)\n$ etcdctl put /test/foo \"hello etcd\" \nOK\n$ etcdctl get /test/foo\n/test/foo\nhello etcd\n\n#手动切换到v2 API\n$ export ETCDCTL_API=2 \n$ etcdctl --version\netcdctl version: 3.4.7\nAPI version: 2\n\n$ etcdctl get /test/foo\nError: client: response is invalid json. The endpoint is probably not valid etcd cluster endpoint #查询不到/test/foo的值\n```\n\n#### 写入key\n```\n$ etcdctl put foo bar\nOK\n```\n#### 读取key值\n```\n$ etcdctl get foo\nfoo\nbar\n\n#只是获取值\n$ etcdctl get foo --print-value-only\nbar\n```\n\n```\n$ etcdctl put foo1 bar1\n$ etcdctl put foo2 bar2\n$ etcdctl put foo3 bar3\n\n#获取从foo到foo3的值，不包括foo3\n$ etcdctl get foo foo3 --print-value-only \nbar\nbar1\nbar2\n\n# 获取前缀为foo的值\n$ etcdctl get --prefix foo --print-value-only\nbar\nbar1\nbar2\nbar3\n\n#获取符合前缀的前两个值\n$ etcdctl get --prefix --limit=2 foo --print-value-only\nbar\nbar1\n```\n\n#### 删除key\n```\n#删除foo\n$ etcdctl del foo\n1\n\n#删除foo到foo2，不包括foo2\n$ etcdctl del foo foo2\n1\n#删除key前缀为foo的\n$ etcdctl del --prefix foo\n2\n```\n\n#### 监视值变化\n```\n#监视foo单个key\n$ etcdctl watch foo\n#另一个控制台执行： etcdctl put foo bar\nPUT\nfoo\nbar\n\n#同时监视多个值\n$ etcdctl watch -i\n$ watch foo\n$ watch zoo\n# 另一个控制台执行: etcdctl put foo bar\nPUT\nfoo\nbar\n# 另一个控制台执行: etcdctl put zoo val\nPUT\nzoo\nval\n\n#监视foo前缀的key\n$ etcdctl watch --prefix foo\n#另一个控制台执行： etcdctl put foo1 bar1\nPUT\nfoo1\nbar1\n#另一个控制台执行： etcdctl put fooz1 barz1\nPUT\nfooz1\nbarz1\n```\n\n#### 设置租约（Grant leases）\n\n当一个key被绑定到一个租约上时，它的生命周期与租约的生命周期绑定。\n\n```\n#设置60秒后过期时间\n$ etcdctl lease grant 60\nlease 32695410dcc0ca06 granted with TTL(60s)\n\n#把foo和租约绑定，设置成60秒后过期\n$ etcdctl put --lease=32695410dcc0ca06 foo bar\nOK\n$ etcdctl get foo\nfoo\nbar\n\n#60秒后，获取不到foo\n$ etcdctl get foo\n#返回空\n```\n\n#### 主动撤销租约（Revoke leases）\n\n通过租赁ID（此处指：`32695410dcc0ca06`）撤销租约。`撤销租约将删除其所有绑定的key`。\n\n```\n$ etcdctl lease grant 60\nlease 32695410dcc0ca06 granted with TTL(60s)\n$ etcdctl put foo bar --lease=32695410dcc0ca06\nOK\n\n#主动撤销租约\n$ etcdctl lease revoke 32695410dcc0ca06\nlease 32695410dcc0ca06 revoked\n\n$ etcdctl get foo\n#返回空\n```\n\n#### 续租约（Keep leases alive）\n\n通过刷新其TTL来保持租约的有效，使其不会过期。\n\n```\n#设置60秒后过期租约\n$ etcdctl lease grant 60\nlease 32695410dcc0ca06 granted with TTL(60s)\n\n#把foo和租约绑定，设置成60秒后过期\n$ etcdctl put foo bar --lease=32695410dcc0ca06\n\n#续租约，自动定时执行续租约，续约成功后每次租约为60秒\n$ etcdctl lease keep-alive 32695410dcc0ca06\nlease 32695410dcc0ca06 keepalived with TTL(60)\nlease 32695410dcc0ca06 keepalived with TTL(60)\nlease 32695410dcc0ca06 keepalived with TTL(60)\n...\n```\n\n#### 获取租约信息（Get lease information）\n\n 获取租约信息，以便续租或查看租约是否仍然存在或已过期\n \n```\n#设置500秒TTL\n$ etcdctl lease grant 500\nlease 694d5765fc71500b granted with TTL(500s)\n\n#keyzoo1绑定694d5765fc71500b租约\n$ etcdctl put zoo1 val1 --lease=694d5765fc71500b\nOK\n\n#查看租约信息，remaining(132s)剩余有效时间132秒；--keys获取租约绑定的key\n$ etcdctl lease timetolive --keys 694d5765fc71500b\nlease 694d5765fc71500b granted with TTL(500s), remaining(132s), attached keys([zoo1])\n```\n\n值得注意的地方，一个租约可以绑定多个`key`\n```\n$ etcdctl lease grant 500\nlease 694d5765fc71500b granted with TTL(500s)\n\n$ etcdctl put zoo1 val1 --lease=694d5765fc71500b\nOK\n\n$ etcdctl put zoo2 val2 --lease=694d5765fc71500b\nOK\n```\n当租约过期后，所有key值会被删除。\n\n当一个租约只绑定了一个`key`时，想删除这个`key`，最好的办法是撤销它的租约，而不是直接删除这个`key`。\n\n看下面这个例子：\n```\n#方法一：直接删除`key`\n#设置租约并绑定zoo1\n$ etcdctl lease grant 60\nlease 694d71f80ed8bf1e granted with TTL(60s)\n$ etcdctl put zoo1 val1 --lease=694d71f80ed8bf1e\nOK\n\n#续租约\n$ etcdctl lease keep-alive 694d71f80ed8bf1e\nlease 694d71f80ed8bf1e keepalived with TTL(60)\n\n#另一个控制台执行：etcdctl del zoo1\n\n#单纯删除key后，续约操作还会一直进行，造成内存泄露\nlease 694d71f80ed8bf1e keepalived with TTL(60)\nlease 694d71f80ed8bf1e keepalived with TTL(60)\nlease 694d71f80ed8bf1e keepalived with TTL(60)\n...\n```\n\n```\n方法二：撤销`key`的租约\n#设置租约并绑定zoo1\n$ etcdctl lease grant 60\nlease 694d71f80ed8bf1e granted with TTL(60s)\n$ etcdctl put zoo1 val1 --lease=694d71f80ed8bf1e\nOK\n\n#续租约\n$ etcdctl lease keep-alive 694d71f80ed8bf1e\nlease 694d71f80ed8bf1e keepalived with TTL(60)\nlease 694d71f80ed8bf1e keepalived with TTL(60)\n\n#另一个控制台执行：etcdctl lease revoke 694d71f80ed8bf1e\n\n#续约操作并退出\nlease 694d71f80ed8bf1e expired or revoked.\n```\n\n当租约没有绑定`key`时，应主动把它撤销掉。\n\n### 应用场景\n\n根据以上特性和API，etcd有应用场景以下应用场景：\n\n#### 场景一：服务发现\n服务发现要解决的也是分布式系统中最常见的问题之一，即在同一个分布式集群中的进程或服务，要如何才能找到对方并建立连接。本质上来说，服务发现就是想要了解集群中是否有进程在监听 udp 或 tcp 端口，并且通过名字就可以查找和连接。\n\n#### 场景二：配置中心\netcd的应用场景优化都是围绕存储的东西是“配置” 来设定的。\n* 配置的数据量通常都不大，所以默认etcd的存储上限是1GB\n* 配置通常对历史版本信息是比较关心的，所以etcd会保存 版本（revision） 信息\n* 配置变更是比较常见的，并且业务程序会需要实时知道，所以etcd提供了watch机制，基本就是实时通知配置变化\n* 配置的准确性一致性极其重要，所以etcd采用raft算法，保证系统的CP\n* 同一份配置通常会被大量客户端同时访问，针对这个做了grpc proxy对同一个key的watcher做了优化\n* 配置会被不同的业务部门使用，提供了权限控制和namespace机制\n\n#### 场景三：负载均衡\n此处指的负载均衡均为软负载均衡，分布式系统中，为了保证服务的高可用以及数据的一致性，通常都会把数据和服务部署多份，以此达到对等服务，即使其中的某一个服务失效了，也不影响使用。由此带来的坏处是数据写入性能下降，而好处则是数据访问时的负载均衡。因为每个对等服务节点上都存有完整的数据，所以用户的访问流量就可以分流到不同的机器上。\n\n#### 场景四：分布式锁\n因为 etcd 使用 Raft 算法保持了数据的强一致性，某次操作存储到集群中的值必然是全局一致的，所以很容易实现分布式锁。\n\n#### 场景五：集群监控与 Leader 竞选\n通过 etcd 来进行监控实现起来非常简单并且实时性强。\n\n* 前面几个场景已经提到 Watcher 机制，当某个节点消失或有变动时，Watcher 会第一时间发现并告知用户。\n* 节点可以设置TTL key，比如每隔 30s 发送一次心跳使代表该机器存活的节点继续存在，否则节点消失。\n\n这样就可以第一时间检测到各节点的健康状态，以完成集群的监控要求。\n\n另外，使用分布式锁，可以完成 Leader 竞选。这种场景通常是一些长时间 CPU 计算或者使用 IO 操作的机器，只需要竞选出的 Leader 计算或处理一次，就可以把结果复制给其他的 Follower。从而避免重复劳动，节省计算资源。\n\n参考：\n* https://github.com/etcd-io/etcd\n* https://etcd.io/docs/v3.4.0/dev-guide/interacting_v3/\n* https://www.infoq.cn/article/etcd-interpretation-application-scenario-implement-principle/" }, { "path": "2-etcd-go-client/README.md", "content": "代码来源：https://github.com/etcd-io/etcd/tree/master/clientv3" }, { "path": "2-etcd-go-client/example_auth_test.go", "content": "// Copyright 2016 The etcd Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage clientv3_test\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\n\t\"github.com/coreos/etcd/clientv3\"\n)\n\nfunc ExampleAuth() {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: dialTimeout,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close()\n\n\tif _, err = cli.RoleAdd(context.TODO(), \"root\"); err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tif _, err = cli.UserAdd(context.TODO(), \"root\", \"123\"); err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tif _, err = cli.UserGrantRole(context.TODO(), \"root\", \"root\"); err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tif _, err = cli.RoleAdd(context.TODO(), \"r\"); err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tif _, err = cli.RoleGrantPermission(\n\t\tcontext.TODO(),\n\t\t\"r\", // role name\n\t\t\"foo\", // key\n\t\t\"zoo\", // range end\n\t\tclientv3.PermissionType(clientv3.PermReadWrite),\n\t); err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tif _, err = cli.UserAdd(context.TODO(), \"u\", \"123\"); err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tif _, err = cli.UserGrantRole(context.TODO(), \"u\", \"r\"); err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tif _, err = cli.AuthEnable(context.TODO()); err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tcliAuth, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: dialTimeout,\n\t\tUsername: \"u\",\n\t\tPassword: \"123\",\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cliAuth.Close()\n\n\tif _, err = cliAuth.Put(context.TODO(), \"foo1\", \"bar\"); err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\t_, err = cliAuth.Txn(context.TODO()).\n\t\tIf(clientv3.Compare(clientv3.Value(\"zoo1\"), \">\", \"abc\")).\n\t\tThen(clientv3.OpPut(\"zoo1\", \"XYZ\")).\n\t\tElse(clientv3.OpPut(\"zoo1\", \"ABC\")).\n\t\tCommit()\n\tfmt.Println(err)\n\n\t// now check the permission with the root account\n\trootCli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: dialTimeout,\n\t\tUsername: \"root\",\n\t\tPassword: \"123\",\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer rootCli.Close()\n\n\tresp, err := rootCli.RoleGet(context.TODO(), \"r\")\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tfmt.Printf(\"user u permission: key %q, range end %q\\n\", resp.Perm[0].Key, resp.Perm[0].RangeEnd)\n\n\tif _, err = rootCli.AuthDisable(context.TODO()); err != nil {\n\t\tlog.Fatal(err)\n\t}\n\t// Output: etcdserver: permission denied\n\t// user u permission: key \"foo\", range end \"zoo\"\n}\n" }, { "path": "2-etcd-go-client/example_cluster_test.go", "content": "// Copyright 2016 The etcd Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage clientv3_test\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\n\t\"github.com/coreos/etcd/clientv3\"\n)\n\nfunc ExampleCluster_memberList() {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: dialTimeout,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close()\n\n\tresp, err := cli.MemberList(context.Background())\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tfmt.Println(\"members:\", len(resp.Members))\n\t// Output: members: 3\n}\n\nfunc ExampleCluster_memberAdd() {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints[:2],\n\t\tDialTimeout: dialTimeout,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close()\n\n\tpeerURLs := endpoints[2:]\n\tmresp, err := cli.MemberAdd(context.Background(), peerURLs)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tfmt.Println(\"added member.PeerURLs:\", mresp.Member.PeerURLs)\n\t// added member.PeerURLs: [http://localhost:32380]\n}\n\nfunc ExampleCluster_memberRemove() {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints[1:],\n\t\tDialTimeout: dialTimeout,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close()\n\n\tresp, err := cli.MemberList(context.Background())\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\t_, err = cli.MemberRemove(context.Background(), resp.Members[0].ID)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n}\n\nfunc ExampleCluster_memberUpdate() {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: dialTimeout,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close()\n\n\tresp, err := cli.MemberList(context.Background())\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tpeerURLs := []string{\"http://localhost:12380\"}\n\t_, err = cli.MemberUpdate(context.Background(), resp.Members[0].ID, peerURLs)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n}\n" }, { "path": "2-etcd-go-client/example_kv_test.go", "content": "// Copyright 2016 The etcd Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage clientv3_test\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\n\t\"github.com/coreos/etcd/clientv3\"\n\t\"github.com/coreos/etcd/etcdserver/api/v3rpc/rpctypes\"\n)\n\nfunc ExampleKV_put() {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: dialTimeout,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close()\n\n\tctx, cancel := context.WithTimeout(context.Background(), requestTimeout)\n\t_, err = cli.Put(ctx, \"sample_key\", \"sample_value\")\n\tcancel()\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n}\n\nfunc ExampleKV_putErrorHandling() {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: dialTimeout,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close()\n\n\tctx, cancel := context.WithTimeout(context.Background(), requestTimeout)\n\t_, err = cli.Put(ctx, \"\", \"sample_value\")\n\tcancel()\n\tif err != nil {\n\t\tswitch err {\n\t\tcase context.Canceled:\n\t\t\tfmt.Printf(\"ctx is canceled by another routine: %v\\n\", err)\n\t\tcase context.DeadlineExceeded:\n\t\t\tfmt.Printf(\"ctx is attached with a deadline is exceeded: %v\\n\", err)\n\t\tcase rpctypes.ErrEmptyKey:\n\t\t\tfmt.Printf(\"client-side error: %v\\n\", err)\n\t\tdefault:\n\t\t\tfmt.Printf(\"bad cluster endpoints, which are not etcd servers: %v\\n\", err)\n\t\t}\n\t}\n\t// Output: client-side error: etcdserver: key is not provided\n}\n\nfunc ExampleKV_get() {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: dialTimeout,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close()\n\n\t_, err = cli.Put(context.TODO(), \"foo\", \"bar\")\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tctx, cancel := context.WithTimeout(context.Background(), requestTimeout)\n\tresp, err := cli.Get(ctx, \"foo\")\n\tcancel()\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tfor _, ev := range resp.Kvs {\n\t\tfmt.Printf(\"%s : %s\\n\", ev.Key, ev.Value)\n\t}\n\t// Output: foo : bar\n}\n\nfunc ExampleKV_getWithRev() {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: dialTimeout,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close()\n\n\tpresp, err := cli.Put(context.TODO(), \"foo\", \"bar1\")\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\t_, err = cli.Put(context.TODO(), \"foo\", \"bar2\")\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tctx, cancel := context.WithTimeout(context.Background(), requestTimeout)\n\tresp, err := cli.Get(ctx, \"foo\", clientv3.WithRev(presp.Header.Revision))\n\tcancel()\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tfor _, ev := range resp.Kvs {\n\t\tfmt.Printf(\"%s : %s\\n\", ev.Key, ev.Value)\n\t}\n\t// Output: foo : bar1\n}\n\nfunc ExampleKV_getSortedPrefix() {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: dialTimeout,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close()\n\n\tfor i := range make([]int, 3) {\n\t\tctx, cancel := context.WithTimeout(context.Background(), requestTimeout)\n\t\t_, err = cli.Put(ctx, fmt.Sprintf(\"key_%d\", i), \"value\")\n\t\tcancel()\n\t\tif err != nil {\n\t\t\tlog.Fatal(err)\n\t\t}\n\t}\n\n\tctx, cancel := context.WithTimeout(context.Background(), requestTimeout)\n\tresp, err := cli.Get(ctx, \"key\", clientv3.WithPrefix(), clientv3.WithSort(clientv3.SortByKey, clientv3.SortDescend))\n\tcancel()\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tfor _, ev := range resp.Kvs {\n\t\tfmt.Printf(\"%s : %s\\n\", ev.Key, ev.Value)\n\t}\n\t// Output:\n\t// key_2 : value\n\t// key_1 : value\n\t// key_0 : value\n}\n\nfunc ExampleKV_delete() {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: dialTimeout,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close()\n\n\tctx, cancel := context.WithTimeout(context.Background(), requestTimeout)\n\tdefer cancel()\n\n\t// count keys about to be deleted\n\tgresp, err := cli.Get(ctx, \"key\", clientv3.WithPrefix())\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\t// delete the keys\n\tdresp, err := cli.Delete(ctx, \"key\", clientv3.WithPrefix())\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tfmt.Println(\"Deleted all keys:\", int64(len(gresp.Kvs)) == dresp.Deleted)\n\t// Output:\n\t// Deleted all keys: true\n}\n\nfunc ExampleKV_compact() {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: dialTimeout,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close()\n\n\tctx, cancel := context.WithTimeout(context.Background(), requestTimeout)\n\tresp, err := cli.Get(ctx, \"foo\")\n\tcancel()\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tcompRev := resp.Header.Revision // specify compact revision of your choice\n\n\tctx, cancel = context.WithTimeout(context.Background(), requestTimeout)\n\t_, err = cli.Compact(ctx, compRev)\n\tcancel()\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n}\n\nfunc ExampleKV_txn() {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: dialTimeout,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close()\n\n\tkvc := clientv3.NewKV(cli)\n\n\t_, err = kvc.Put(context.TODO(), \"key\", \"xyz\")\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tctx, cancel := context.WithTimeout(context.Background(), requestTimeout)\n\t_, err = kvc.Txn(ctx).\n\t\t// txn value comparisons are lexical\n\t\tIf(clientv3.Compare(clientv3.Value(\"key\"), \">\", \"abc\")).\n\t\t// the \"Then\" runs, since \"xyz\" > \"abc\"\n\t\tThen(clientv3.OpPut(\"key\", \"XYZ\")).\n\t\t// the \"Else\" does not run\n\t\tElse(clientv3.OpPut(\"key\", \"ABC\")).\n\t\tCommit()\n\tcancel()\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tgresp, err := kvc.Get(context.TODO(), \"key\")\n\tcancel()\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tfor _, ev := range gresp.Kvs {\n\t\tfmt.Printf(\"%s : %s\\n\", ev.Key, ev.Value)\n\t}\n\t// Output: key : XYZ\n}\n\nfunc ExampleKV_do() {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: dialTimeout,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close()\n\n\tops := []clientv3.Op{\n\t\tclientv3.OpPut(\"put-key\", \"123\"),\n\t\tclientv3.OpGet(\"put-key\"),\n\t\tclientv3.OpPut(\"put-key\", \"456\")}\n\n\tfor _, op := range ops {\n\t\tif _, err := cli.Do(context.TODO(), op); err != nil {\n\t\t\tlog.Fatal(err)\n\t\t}\n\t}\n}\n" }, { "path": "2-etcd-go-client/example_lease_test.go", "content": "// Copyright 2016 The etcd Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage clientv3_test\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\n\t\"github.com/coreos/etcd/clientv3\"\n)\n\nfunc ExampleLease_grant() {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: dialTimeout,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close()\n\n\t// minimum lease TTL is 5-second\n\tresp, err := cli.Grant(context.TODO(), 5)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\t// after 5 seconds, the key 'foo' will be removed\n\t_, err = cli.Put(context.TODO(), \"foo\", \"bar\", clientv3.WithLease(resp.ID))\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n}\n\nfunc ExampleLease_revoke() {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: dialTimeout,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close()\n\n\tresp, err := cli.Grant(context.TODO(), 5)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\t_, err = cli.Put(context.TODO(), \"foo\", \"bar\", clientv3.WithLease(resp.ID))\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\t// revoking lease expires the key attached to its lease ID\n\t_, err = cli.Revoke(context.TODO(), resp.ID)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tgresp, err := cli.Get(context.TODO(), \"foo\")\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tfmt.Println(\"number of keys:\", len(gresp.Kvs))\n\t// Output: number of keys: 0\n}\n\nfunc ExampleLease_keepAlive() {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: dialTimeout,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close()\n\n\tresp, err := cli.Grant(context.TODO(), 5)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\t_, err = cli.Put(context.TODO(), \"foo\", \"bar\", clientv3.WithLease(resp.ID))\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\t// the key 'foo' will be kept forever\n\tch, kaerr := cli.KeepAlive(context.TODO(), resp.ID)\n\tif kaerr != nil {\n\t\tlog.Fatal(kaerr)\n\t}\n\n\tka := <-ch\n\tfmt.Println(\"ttl:\", ka.TTL)\n\t// Output: ttl: 5\n}\n\nfunc ExampleLease_keepAliveOnce() {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: dialTimeout,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close()\n\n\tresp, err := cli.Grant(context.TODO(), 5)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\t_, err = cli.Put(context.TODO(), \"foo\", \"bar\", clientv3.WithLease(resp.ID))\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\t// to renew the lease only once\n\tka, kaerr := cli.KeepAliveOnce(context.TODO(), resp.ID)\n\tif kaerr != nil {\n\t\tlog.Fatal(kaerr)\n\t}\n\n\tfmt.Println(\"ttl:\", ka.TTL)\n\t// Output: ttl: 5\n}\n" }, { "path": "2-etcd-go-client/example_maintenence_test.go", "content": "// Copyright 2016 The etcd Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage clientv3_test\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\n\t\"github.com/coreos/etcd/clientv3\"\n)\n\nfunc ExampleMaintenance_status() {\n\tfor _, ep := range endpoints {\n\t\tcli, err := clientv3.New(clientv3.Config{\n\t\t\tEndpoints: []string{ep},\n\t\t\tDialTimeout: dialTimeout,\n\t\t})\n\t\tif err != nil {\n\t\t\tlog.Fatal(err)\n\t\t}\n\t\tdefer cli.Close()\n\n\t\tresp, err := cli.Status(context.Background(), ep)\n\t\tif err != nil {\n\t\t\tlog.Fatal(err)\n\t\t}\n\t\tfmt.Printf(\"endpoint: %s / Leader: %v\\n\", ep, resp.Header.MemberId == resp.Leader)\n\t}\n\t// endpoint: localhost:2379 / Leader: false\n\t// endpoint: localhost:22379 / Leader: false\n\t// endpoint: localhost:32379 / Leader: true\n}\n\nfunc ExampleMaintenance_defragment() {\n\tfor _, ep := range endpoints {\n\t\tcli, err := clientv3.New(clientv3.Config{\n\t\t\tEndpoints: []string{ep},\n\t\t\tDialTimeout: dialTimeout,\n\t\t})\n\t\tif err != nil {\n\t\t\tlog.Fatal(err)\n\t\t}\n\t\tdefer cli.Close()\n\n\t\tif _, err = cli.Defragment(context.TODO(), ep); err != nil {\n\t\t\tlog.Fatal(err)\n\t\t}\n\t}\n}\n" }, { "path": "2-etcd-go-client/example_metrics_test.go", "content": "// Copyright 2016 The etcd Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage clientv3_test\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"io/ioutil\"\n\t\"log\"\n\t\"net\"\n\t\"net/http\"\n\t\"strings\"\n\n\t\"github.com/coreos/etcd/clientv3\"\n\n\tgrpcprom \"github.com/grpc-ecosystem/go-grpc-prometheus\"\n\t\"github.com/prometheus/client_golang/prometheus/promhttp\"\n\t\"google.golang.org/grpc\"\n)\n\nfunc ExampleClient_metrics() {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialOptions: []grpc.DialOption{\n\t\t\tgrpc.WithUnaryInterceptor(grpcprom.UnaryClientInterceptor),\n\t\t\tgrpc.WithStreamInterceptor(grpcprom.StreamClientInterceptor),\n\t\t},\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close()\n\n\t// get a key so it shows up in the metrics as a range RPC\n\tcli.Get(context.TODO(), \"test_key\")\n\n\t// listen for all Prometheus metrics\n\tln, err := net.Listen(\"tcp\", \":0\")\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdonec := make(chan struct{})\n\tgo func() {\n\t\tdefer close(donec)\n\t\thttp.Serve(ln, promhttp.Handler())\n\t}()\n\tdefer func() {\n\t\tln.Close()\n\t\t<-donec\n\t}()\n\n\t// make an http request to fetch all Prometheus metrics\n\turl := \"http://\" + ln.Addr().String() + \"/metrics\"\n\tresp, err := http.Get(url)\n\tif err != nil {\n\t\tlog.Fatalf(\"fetch error: %v\", err)\n\t}\n\tb, err := ioutil.ReadAll(resp.Body)\n\tresp.Body.Close()\n\tif err != nil {\n\t\tlog.Fatalf(\"fetch error: reading %s: %v\", url, err)\n\t}\n\n\t// confirm range request in metrics\n\tfor _, l := range strings.Split(string(b), \"\\n\") {\n\t\tif strings.Contains(l, `grpc_client_started_total{grpc_method=\"Range\"`) {\n\t\t\tfmt.Println(l)\n\t\t\tbreak\n\t\t}\n\t}\n\t// Output:\n\t//\tgrpc_client_started_total{grpc_method=\"Range\",grpc_service=\"etcdserverpb.KV\",grpc_type=\"unary\"} 1\n}\n" }, { "path": "2-etcd-go-client/example_test.go", "content": "// Copyright 2016 The etcd Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage clientv3_test\n\nimport (\n\t\"context\"\n\t\"log\"\n\t\"os\"\n\t\"time\"\n\n\t\"github.com/coreos/etcd/clientv3\"\n\t\"github.com/coreos/etcd/pkg/transport\"\n\n\t\"google.golang.org/grpc/grpclog\"\n)\n\nvar (\n\tdialTimeout = 5 * time.Second\n\trequestTimeout = 10 * time.Second\n\tendpoints = []string{\"localhost:2379\"}\n\t//endpoints = []string{\"localhost:2379\", \"localhost:22379\", \"localhost:32379\"}\n)\n\nfunc Example() {\n\tclientv3.SetLogger(grpclog.NewLoggerV2(os.Stderr, os.Stderr, os.Stderr))\n\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: dialTimeout,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close() // make sure to close the client\n\n\t_, err = cli.Put(context.TODO(), \"foo\", \"bar\")\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n}\n\nfunc ExampleConfig_withTLS() {\n\ttlsInfo := transport.TLSInfo{\n\t\tCertFile: \"/tmp/test-certs/test-name-1.pem\",\n\t\tKeyFile: \"/tmp/test-certs/test-name-1-key.pem\",\n\t\tTrustedCAFile: \"/tmp/test-certs/trusted-ca.pem\",\n\t}\n\ttlsConfig, err := tlsInfo.ClientConfig()\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: dialTimeout,\n\t\tTLS: tlsConfig,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close() // make sure to close the client\n\n\t_, err = cli.Put(context.TODO(), \"foo\", \"bar\")\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n}\n" }, { "path": "2-etcd-go-client/example_watch_test.go", "content": "// Copyright 2016 The etcd Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage clientv3_test\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\n\t\"github.com/coreos/etcd/clientv3\"\n)\n\nfunc ExampleWatcher_watch() {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: dialTimeout,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close()\n\n\trch := cli.Watch(context.Background(), \"foo\")\n\tfor wresp := range rch {\n\t\tfor _, ev := range wresp.Events {\n\t\t\tfmt.Printf(\"%s %q : %q\\n\", ev.Type, ev.Kv.Key, ev.Kv.Value)\n\t\t}\n\t}\n\t// PUT \"foo\" : \"bar\"\n}\n\nfunc ExampleWatcher_watchWithPrefix() {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: dialTimeout,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close()\n\n\trch := cli.Watch(context.Background(), \"foo\", clientv3.WithPrefix())\n\tfor wresp := range rch {\n\t\tfor _, ev := range wresp.Events {\n\t\t\tfmt.Printf(\"%s %q : %q\\n\", ev.Type, ev.Kv.Key, ev.Kv.Value)\n\t\t}\n\t}\n\t// PUT \"foo1\" : \"bar\"\n}\n\nfunc ExampleWatcher_watchWithRange() {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: dialTimeout,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close()\n\n\t// watches within ['foo1', 'foo4'), in lexicographical order\n\trch := cli.Watch(context.Background(), \"foo1\", clientv3.WithRange(\"foo4\"))\n\tfor wresp := range rch {\n\t\tfor _, ev := range wresp.Events {\n\t\t\tfmt.Printf(\"%s %q : %q\\n\", ev.Type, ev.Kv.Key, ev.Kv.Value)\n\t\t}\n\t}\n\t// PUT \"foo1\" : \"bar\"\n\t// PUT \"foo2\" : \"bar\"\n\t// PUT \"foo3\" : \"bar\"\n}\n\nfunc ExampleWatcher_watchWithProgressNotify() {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: dialTimeout,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\trch := cli.Watch(context.Background(), \"foo\", clientv3.WithProgressNotify())\n\twresp := <-rch\n\tfmt.Printf(\"wresp.Header.Revision: %d\\n\", wresp.Header.Revision)\n\tfmt.Println(\"wresp.IsProgressNotify:\", wresp.IsProgressNotify())\n\t// wresp.Header.Revision: 0\n\t// wresp.IsProgressNotify: true\n}\n" }, { "path": "3-etcd-service-discovery/.vscode/launch.json", "content": "{\n // 使用 IntelliSense 了解相关属性。 \n // 悬停以查看现有属性的描述。\n // 欲了解更多信息，请访问: https://go.microsoft.com/fwlink/?linkid=830387\n \"version\": \"0.2.0\",\n \"configurations\": [\n {\n \"name\": \"Launch\",\n \"type\": \"go\",\n \"request\": \"launch\",\n \"mode\": \"auto\",\n \"program\": \"${file}\",\n \"env\": {},\n \"args\": []\n }\n ]\n}" }, { "path": "3-etcd-service-discovery/README.md", "content": "### etcd实现服务发现\n\n### 前言\n\n[etcd环境安装与使用](https://bingjian-zhu.github.io/2020/05/09/etcd%E7%8E%AF%E5%A2%83%E5%AE%89%E8%A3%85%E4%B8%8E%E4%BD%BF%E7%94%A8/)文章中介绍了etcd的安装及`v3 API`使用，本篇将介绍如何使用etcd实现服务发现功能。\n\n### 服务发现介绍\n服务发现要解决的也是分布式系统中最常见的问题之一，即在同一个分布式集群中的进程或服务，要如何才能找到对方并建立连接。本质上来说，服务发现就是想要了解集群中是否有进程在监听 udp 或 tcp 端口，并且通过名字就可以查找和连接。\n\n![](https://img2020.cnblogs.com/blog/1508611/202005/1508611-20200514171049345-955603950.png)\n\n服务发现需要实现一下基本功能：\n\n* `服务注册`：同一service的所有节点注册到相同目录下，节点启动后将自己的信息注册到所属服务的目录中。\n\n* `健康检查`：服务节点定时进行健康检查。注册到服务目录中的信息设置一个较短的TTL，运行正常的服务节点每隔一段时间会去更新信息的TTL ，从而达到健康检查效果。\n\n* `服务发现`：通过服务节点能查询到服务提供外部访问的 IP 和端口号。比如网关代理服务时能够及时的发现服务中新增节点、丢弃不可用的服务节点。\n\n接下来介绍如何使用etcd实现服务发现。\n\n### 服务注册及健康检查\n\n根据etcd的`v3 API`，当启动一个服务时候，我们把服务的地址写进etcd，注册服务。同时绑定租约（lease），并以续租约（keep leases alive）的方式检测服务是否正常运行，从而实现健康检查。\n\ngo代码实现：\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"log\"\n\t\"time\"\n\n\t\"go.etcd.io/etcd/clientv3\"\n)\n\n//ServiceRegister 创建租约注册服务\ntype ServiceRegister struct {\n\tcli *clientv3.Client //etcd client\n\tleaseID clientv3.LeaseID //租约ID\n\t//租约keepalieve相应chan\n\tkeepAliveChan <-chan *clientv3.LeaseKeepAliveResponse\n\tkey string //key\n\tval string //value\n}\n\n//NewServiceRegister 新建注册服务\nfunc NewServiceRegister(endpoints []string, key, val string, lease int64) (*ServiceRegister, error) {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: 5 * time.Second,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tser := &ServiceRegister{\n\t\tcli: cli,\n\t\tkey: key,\n\t\tval: val,\n\t}\n\n\t//申请租约设置时间keepalive\n\tif err := ser.putKeyWithLease(lease); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn ser, nil\n}\n\n//设置租约\nfunc (s *ServiceRegister) putKeyWithLease(lease int64) error {\n\t//设置租约时间\n\tresp, err := s.cli.Grant(context.Background(), lease)\n\tif err != nil {\n\t\treturn err\n\t}\n\t//注册服务并绑定租约\n\t_, err = s.cli.Put(context.Background(), s.key, s.val, clientv3.WithLease(resp.ID))\n\tif err != nil {\n\t\treturn err\n\t}\n\t//设置续租 定期发送需求请求\n\tleaseRespChan, err := s.cli.KeepAlive(context.Background(), resp.ID)\n\n\tif err != nil {\n\t\treturn err\n\t}\n\ts.leaseID = resp.ID\n\tlog.Println(s.leaseID)\n\ts.keepAliveChan = leaseRespChan\n\tlog.Printf(\"Put key:%s val:%s success!\", s.key, s.val)\n\treturn nil\n}\n\n//ListenLeaseRespChan 监听 续租情况\nfunc (s *ServiceRegister) ListenLeaseRespChan() {\n\tfor leaseKeepResp := range s.keepAliveChan {\n\t\tlog.Println(\"续约成功\", leaseKeepResp)\n\t}\n\tlog.Println(\"关闭续租\")\n}\n\n// Close 注销服务\nfunc (s *ServiceRegister) Close() error {\n\t//撤销租约\n\tif _, err := s.cli.Revoke(context.Background(), s.leaseID); err != nil {\n\t\treturn err\n\t}\n\tlog.Println(\"撤销租约\")\n\treturn s.cli.Close()\n}\n\nfunc main() {\n\tvar endpoints = []string{\"localhost:2379\"}\n\tser, err := NewServiceRegister(endpoints, \"/web/node1\", \"localhost:8000\", 5)\n\tif err != nil {\n\t\tlog.Fatalln(err)\n\t}\n\t//监听续租相应chan\n\tgo ser.ListenLeaseRespChan()\n\tselect {\n\t// case <-time.After(20 * time.Second):\n\t// \tser.Close()\n\t}\n}\n```\n\n主动退出服务时，可以调用Close()方法，撤销租约，从而注销服务。\n\n### 服务发现\n\n根据etcd的`v3 API`，很容易想到使用`Watch`监视某类服务，通过`Watch`感知服务的`添加`，`修改`或`删除`操作，修改服务列表。\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"log\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/coreos/etcd/mvcc/mvccpb\"\n\t\"go.etcd.io/etcd/clientv3\"\n)\n\n//ServiceDiscovery 服务发现\ntype ServiceDiscovery struct {\n\tcli *clientv3.Client //etcd client\n\tserverList map[string]string //服务列表\n\tlock sync.Mutex\n}\n\n//NewServiceDiscovery 新建发现服务\nfunc NewServiceDiscovery(endpoints []string) *ServiceDiscovery {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: 5 * time.Second,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\treturn &ServiceDiscovery{\n\t\tcli: cli,\n\t\tserverList: make(map[string]string),\n\t}\n}\n\n//WatchService 初始化服务列表和监视\nfunc (s *ServiceDiscovery) WatchService(prefix string) error {\n\t//根据前缀获取现有的key\n\tresp, err := s.cli.Get(context.Background(), prefix, clientv3.WithPrefix())\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfor _, ev := range resp.Kvs {\n\t\ts.SetServiceList(string(ev.Key), string(ev.Value))\n\t}\n\n\t//监视前缀，修改变更的server\n\tgo s.watcher(prefix)\n\treturn nil\n}\n\n//watcher 监听前缀\nfunc (s *ServiceDiscovery) watcher(prefix string) {\n\trch := s.cli.Watch(context.Background(), prefix, clientv3.WithPrefix())\n\tlog.Printf(\"watching prefix:%s now...\", prefix)\n\tfor wresp := range rch {\n\t\tfor _, ev := range wresp.Events {\n\t\t\tswitch ev.Type {\n\t\t\tcase mvccpb.PUT: //修改或者新增\n\t\t\t\ts.SetServiceList(string(ev.Kv.Key), string(ev.Kv.Value))\n\t\t\tcase mvccpb.DELETE: //删除\n\t\t\t\ts.DelServiceList(string(ev.Kv.Key))\n\t\t\t}\n\t\t}\n\t}\n}\n\n//SetServiceList 新增服务地址\nfunc (s *ServiceDiscovery) SetServiceList(key, val string) {\n\ts.lock.Lock()\n\tdefer s.lock.Unlock()\n\ts.serverList[key] = string(val)\n\tlog.Println(\"put key :\", key, \"val:\", val)\n}\n\n//DelServiceList 删除服务地址\nfunc (s *ServiceDiscovery) DelServiceList(key string) {\n\ts.lock.Lock()\n\tdefer s.lock.Unlock()\n\tdelete(s.serverList, key)\n\tlog.Println(\"del key:\", key)\n}\n\n//GetServices 获取服务地址\nfunc (s *ServiceDiscovery) GetServices() []string {\n\ts.lock.Lock()\n\tdefer s.lock.Unlock()\n\taddrs := make([]string, 0)\n\n\tfor _, v := range s.serverList {\n\t\taddrs = append(addrs, v)\n\t}\n\treturn addrs\n}\n\n//Close 关闭服务\nfunc (s *ServiceDiscovery) Close() error {\n\treturn s.cli.Close()\n}\n\nfunc main() {\n\tvar endpoints = []string{\"localhost:2379\"}\n\tser := NewServiceDiscovery(endpoints)\n\tdefer ser.Close()\n\tser.WatchService(\"/web/\")\n\tser.WatchService(\"/gRPC/\")\n\tfor {\n\t\tselect {\n\t\tcase <-time.Tick(10 * time.Second):\n\t\t\tlog.Println(ser.GetServices())\n\t\t}\n\t}\n}\n```\n运行：\n```\n#运行服务发现\n$go run discovery.go\nwatching prefix:/web/ now...\nput key : /web/node1 val:localhost:8000\n[localhost:8000]\n\n#另一个终端运行服务注册\n$go run register.go\nPut key:/web/node1 val:localhost:8000 success!\n续约成功 cluster_id:14841639068965178418 member_id:10276657743932975437 revision:29 raft_term:7 \n续约成功 cluster_id:14841639068965178418 member_id:10276657743932975437 revision:29 raft_term:7 \n...\n```\n\n### 总结\n基于 Raft 算法的 etcd 天生是一个强一致性高可用的服务存储目录，用户可以在 etcd 中注册服务，并且对注册的服务设置key TTL，定时保持服务的心跳以达到监控健康状态的效果。通过在 etcd 指定的主题下注册的服务也能在对应的主题下查找到。\n\n为了确保连接，我们可以在每个服务机器上都部署一个 Proxy 模式的 etcd，这样就可以确保能访问 etcd 集群的服务都能互相连接。\n\n参考：\n* https://segmentfault.com/a/1190000020944777\n* https://blog.csdn.net/blogsun/article/details/102861648\n* https://www.infoq.cn/article/etcd-interpretation-application-scenario-implement-principle/\n" }, { "path": "3-etcd-service-discovery/discovery/discovery.go", "content": "package main\n\nimport (\n\t\"context\"\n\t\"log\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/coreos/etcd/mvcc/mvccpb\"\n\t\"go.etcd.io/etcd/clientv3\"\n)\n\n//ServiceDiscovery 服务发现\ntype ServiceDiscovery struct {\n\tcli *clientv3.Client //etcd client\n\tserverList sync.Map\n}\n\n//NewServiceDiscovery 新建发现服务\nfunc NewServiceDiscovery(endpoints []string) *ServiceDiscovery {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: 5 * time.Second,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\treturn &ServiceDiscovery{\n\t\tcli: cli,\n\t}\n}\n\n//WatchService 初始化服务列表和监视\nfunc (s *ServiceDiscovery) WatchService(prefix string) error {\n\t//根据前缀获取现有的key\n\tresp, err := s.cli.Get(context.Background(), prefix, clientv3.WithPrefix())\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfor _, ev := range resp.Kvs {\n\t\ts.SetServiceList(string(ev.Key), string(ev.Value))\n\t}\n\n\t//监视前缀，修改变更的server\n\tgo s.watcher(prefix)\n\treturn nil\n}\n\n//watcher 监听前缀\nfunc (s *ServiceDiscovery) watcher(prefix string) {\n\trch := s.cli.Watch(context.Background(), prefix, clientv3.WithPrefix())\n\tlog.Printf(\"watching prefix:%s now...\", prefix)\n\tfor wresp := range rch {\n\t\tfor _, ev := range wresp.Events {\n\t\t\tswitch ev.Type {\n\t\t\tcase mvccpb.PUT: //修改或者新增\n\t\t\t\ts.SetServiceList(string(ev.Kv.Key), string(ev.Kv.Value))\n\t\t\tcase mvccpb.DELETE: //删除\n\t\t\t\ts.DelServiceList(string(ev.Kv.Key))\n\t\t\t}\n\t\t}\n\t}\n}\n\n//SetServiceList 新增服务地址\nfunc (s *ServiceDiscovery) SetServiceList(key, val string) {\n\ts.serverList.Store(key, val)\n\tlog.Println(\"put key :\", key, \"val:\", val)\n}\n\n//DelServiceList 删除服务地址\nfunc (s *ServiceDiscovery) DelServiceList(key string) {\n\ts.serverList.Delete(key)\n\tlog.Println(\"del key:\", key)\n}\n\n//GetServices 获取服务地址\nfunc (s *ServiceDiscovery) GetServices() []string {\n\taddrs := make([]string, 0, 10)\n\ts.serverList.Range(func(k, v interface{}) bool {\n\t\taddrs = append(addrs, v.(string))\n\t\treturn true\n\t})\n\treturn addrs\n}\n\n//Close 关闭服务\nfunc (s *ServiceDiscovery) Close() error {\n\treturn s.cli.Close()\n}\n\nfunc main() {\n\tvar endpoints = []string{\"localhost:2379\"}\n\tser := NewServiceDiscovery(endpoints)\n\tdefer ser.Close()\n\tser.WatchService(\"/web/\")\n\tser.WatchService(\"/gRPC/\")\n\tfor {\n\t\tselect {\n\t\tcase <-time.Tick(10 * time.Second):\n\t\t\tlog.Println(ser.GetServices())\n\t\t}\n\t}\n}\n" }, { "path": "3-etcd-service-discovery/register/register.go", "content": "package main\n\nimport (\n\t\"context\"\n\t\"log\"\n\t\"time\"\n\n\t\"go.etcd.io/etcd/clientv3\"\n)\n\n//ServiceRegister 创建租约注册服务\ntype ServiceRegister struct {\n\tcli *clientv3.Client //etcd client\n\tleaseID clientv3.LeaseID //租约ID\n\t//租约keepalieve相应chan\n\tkeepAliveChan <-chan *clientv3.LeaseKeepAliveResponse\n\tkey string //key\n\tval string //value\n}\n\n//NewServiceRegister 新建注册服务\nfunc NewServiceRegister(endpoints []string, key, val string, lease int64) (*ServiceRegister, error) {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: 5 * time.Second,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tser := &ServiceRegister{\n\t\tcli: cli,\n\t\tkey: key,\n\t\tval: val,\n\t}\n\n\t//申请租约设置时间keepalive\n\tif err := ser.putKeyWithLease(lease); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn ser, nil\n}\n\n//设置租约\nfunc (s *ServiceRegister) putKeyWithLease(lease int64) error {\n\t//设置租约时间\n\tresp, err := s.cli.Grant(context.Background(), lease)\n\tif err != nil {\n\t\treturn err\n\t}\n\t//注册服务并绑定租约\n\t_, err = s.cli.Put(context.Background(), s.key, s.val, clientv3.WithLease(resp.ID))\n\tif err != nil {\n\t\treturn err\n\t}\n\t//设置续租 定期发送需求请求\n\tleaseRespChan, err := s.cli.KeepAlive(context.Background(), resp.ID)\n\n\tif err != nil {\n\t\treturn err\n\t}\n\ts.leaseID = resp.ID\n\ts.keepAliveChan = leaseRespChan\n\tlog.Printf(\"Put key:%s val:%s success!\", s.key, s.val)\n\treturn nil\n}\n\n//ListenLeaseRespChan 监听 续租情况\nfunc (s *ServiceRegister) ListenLeaseRespChan() {\n\tfor leaseKeepResp := range s.keepAliveChan {\n\t\tlog.Println(\"续约成功\", leaseKeepResp)\n\t}\n\tlog.Println(\"关闭续租\")\n}\n\n// Close 注销服务\nfunc (s *ServiceRegister) Close() error {\n\t//撤销租约\n\tif _, err := s.cli.Revoke(context.Background(), s.leaseID); err != nil {\n\t\treturn err\n\t}\n\tlog.Println(\"撤销租约\")\n\treturn s.cli.Close()\n}\n\nfunc main() {\n\tvar endpoints = []string{\"localhost:2379\"}\n\tser, err := NewServiceRegister(endpoints, \"/web/node1\", \"localhost:8000\", 5)\n\tif err != nil {\n\t\tlog.Fatalln(err)\n\t}\n\t//监听续租相应chan\n\tgo ser.ListenLeaseRespChan()\n\tselect {\n\t// case <-time.After(20 * time.Second):\n\t// \tser.Close()\n\t}\n}\n" }, { "path": "4-etcd-grpclb/README.md", "content": "### gRPC负载均衡（客户端负载均衡）\n\n### 前言\n[上篇](https://bingjian-zhu.github.io/2020/05/14/etcd%E5%AE%9E%E7%8E%B0%E6%9C%8D%E5%8A%A1%E5%8F%91%E7%8E%B0/)介绍了如何使用`etcd`实现服务发现，本篇将基于etcd的服务发现前提下，介绍如何实现gRPC客户端负载均衡。\n\n### gRPC负载均衡\ngRPC官方文档提供了关于gRPC负载均衡方案[Load Balancing in gRPC](https://github.com/grpc/grpc/blob/master/doc/load-balancing.md)，此方案是为gRPC设计的，下面我们对此进行分析。\n\n#### 1、对每次调用进行负载均衡\ngRPC中的负载平衡是以每次调用为基础，而不是以每个连接为基础。换句话说，即使所有的请求都来自一个客户端，我们仍希望它们在所有的服务器上实现负载平衡。\n\n#### 2、负载均衡的方法\n\n* `集中式`（Proxy Model）\n\n![](https://img2020.cnblogs.com/blog/1508611/202005/1508611-20200518153536494-684598725.png)\n\n在服务消费者和服务提供者之间有一个独立的负载均衡（LB），通常是专门的硬件设备如 F5，或者基于软件如 LVS，HAproxy等实现。LB上有所有服务的地址映射表，通常由运维配置注册，当服务消费方调用某个目标服务时，它向LB发起请求，由LB以某种策略，比如轮询（Round-Robin）做负载均衡后将请求转发到目标服务。LB一般具备健康检查能力，能自动摘除不健康的服务实例。 \n\n该方案主要问题：服务消费方、提供方之间增加了一级，有一定性能开销，请求量大时，效率较低。\n\n> 可能有读者会认为集中式负载均衡存在这样的问题，一旦负载均衡服务挂掉，那整个系统将不能使用。\n> 解决方案：可以对负载均衡服务进行DNS负载均衡，通过对一个域名设置多个IP地址，每次DNS解析时轮询返回负载均衡服务地址，从而实现简单的DNS负载均衡。\n\n* `客户端负载`（Balancing-aware Client）\n\n![](https://img2020.cnblogs.com/blog/1508611/202005/1508611-20200518155900462-1370526164.png)\n\n针对第一个方案的不足，此方案将LB的功能集成到服务消费方进程里，也被称为软负载或者客户端负载方案。服务提供方启动时，首先将服务地址注册到服务注册表，同时定期报心跳到服务注册表以表明服务的存活状态，相当于健康检查，服务消费方要访问某个服务时，它通过内置的LB组件向服务注册表查询，同时缓存并定期刷新目标服务地址列表，然后以某种负载均衡策略选择一个目标服务地址，最后向目标服务发起请求。LB和服务发现能力被分散到每一个服务消费者的进程内部，同时服务消费方和服务提供方之间是直接调用，没有额外开销，性能比较好。\n\n该方案主要问题：要用多种语言、多个版本的客户端编写和维护负载均衡策略，使客户端的代码大大复杂化。\n\n* `独立LB服务`（External Load Balancing Service）\n\n![](https://img2020.cnblogs.com/blog/1508611/202005/1508611-20200518170636421-1833253282.png)\n\n该方案是针对第二种方案的不足而提出的一种折中方案，原理和第二种方案基本类似。\n\n不同之处是将LB和服务发现功能从进程内移出来，变成主机上的一个独立进程。主机上的一个或者多个服务要访问目标服务时，他们都通过同一主机上的独立LB进程做服务发现和负载均衡。该方案也是一种分布式方案没有单点问题，服务调用方和LB之间是进程内调用性能好，同时该方案还简化了服务调用方，不需要为不同语言开发客户库。 \n\n本篇将介绍第二种负载均衡方法，客户端负载均衡。\n\n### 实现gRPC客户端负载均衡\n\ngRPC已提供了简单的负载均衡策略（如：Round Robin），我们只需实现它提供的`Builder`和`Resolver`接口，就能完成gRPC客户端负载均衡。\n\n```go\ntype Builder interface {\n\tBuild(target Target, cc ClientConn, opts BuildOption) (Resolver, error)\n\tScheme() string\n}\n```\n`Builder`接口：创建一个`resolver`（本文称之服务发现），用于监视名称解析更新。\n`Build`方法：为给定目标创建一个新的`resolver`，当调用`grpc.Dial()`时执行。\n`Scheme`方法：返回此`resolver`支持的方案，`Scheme`定义可参考：https://github.com/grpc/grpc/blob/master/doc/naming.md\n\n```go\ntype Resolver interface {\n\tResolveNow(ResolveNowOption)\n\tClose()\n}\n```\n`Resolver`接口：监视指定目标的更新，包括地址更新和服务配置更新。\n`ResolveNow`方法：被 gRPC 调用，以尝试再次解析目标名称。只用于提示，可忽略该方法。\n`Close`方法：关闭`resolver`\n\n根据以上两个接口，我们把服务发现的功能写在`Build`方法中，把获取到的负载均衡服务地址返回到客户端，并监视服务更新情况，以修改客户端连接。\n修改服务发现代码，`discovery.go`\n```go\npackage etcdv3\n\nimport (\n\t\"context\"\n\t\"log\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/coreos/etcd/mvcc/mvccpb\"\n\t\"go.etcd.io/etcd/clientv3\"\n\t\"google.golang.org/grpc/resolver\"\n)\n\nconst schema = \"grpclb\"\n\n//ServiceDiscovery 服务发现\ntype ServiceDiscovery struct {\n\tcli *clientv3.Client //etcd client\n\tcc resolver.ClientConn\n\tserverList map[string]resolver.Address //服务列表\n\tlock sync.Mutex\n}\n\n//NewServiceDiscovery 新建发现服务\nfunc NewServiceDiscovery(endpoints []string) resolver.Builder {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: 5 * time.Second,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\treturn &ServiceDiscovery{\n\t\tcli: cli,\n\t}\n}\n\n//Build 为给定目标创建一个新的`resolver`，当调用`grpc.Dial()`时执行\nfunc (s *ServiceDiscovery) Build(target resolver.Target, cc resolver.ClientConn, opts resolver.BuildOption) (resolver.Resolver, error) {\n\tlog.Println(\"Build\")\n\ts.cc = cc\n\ts.serverList = make(map[string]resolver.Address)\n\tprefix := \"/\" + target.Scheme + \"/\" + target.Endpoint + \"/\"\n\t//根据前缀获取现有的key\n\tresp, err := s.cli.Get(context.Background(), prefix, clientv3.WithPrefix())\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfor _, ev := range resp.Kvs {\n\t\ts.SetServiceList(string(ev.Key), string(ev.Value))\n\t}\n\ts.cc.NewAddress(s.getServices())\n\t//监视前缀，修改变更的server\n\tgo s.watcher(prefix)\n\treturn s, nil\n}\n\n// ResolveNow 监视目标更新\nfunc (s *ServiceDiscovery) ResolveNow(rn resolver.ResolveNowOption) {\n\tlog.Println(\"ResolveNow\")\n}\n\n//Scheme return schema\nfunc (s *ServiceDiscovery) Scheme() string {\n\treturn schema\n}\n\n//Close 关闭\nfunc (s *ServiceDiscovery) Close() {\n\tlog.Println(\"Close\")\n\ts.cli.Close()\n}\n\n//watcher 监听前缀\nfunc (s *ServiceDiscovery) watcher(prefix string) {\n\trch := s.cli.Watch(context.Background(), prefix, clientv3.WithPrefix())\n\tlog.Printf(\"watching prefix:%s now...\", prefix)\n\tfor wresp := range rch {\n\t\tfor _, ev := range wresp.Events {\n\t\t\tswitch ev.Type {\n\t\t\tcase mvccpb.PUT: //新增或修改\n\t\t\t\ts.SetServiceList(string(ev.Kv.Key), string(ev.Kv.Value))\n\t\t\tcase mvccpb.DELETE: //删除\n\t\t\t\ts.DelServiceList(string(ev.Kv.Key))\n\t\t\t}\n\t\t}\n\t}\n}\n\n//SetServiceList 新增服务地址\nfunc (s *ServiceDiscovery) SetServiceList(key, val string) {\n\ts.lock.Lock()\n\tdefer s.lock.Unlock()\n\ts.serverList[key] = resolver.Address{Addr: val}\n\ts.cc.NewAddress(s.getServices())\n\tlog.Println(\"put key :\", key, \"val:\", val)\n}\n\n//DelServiceList 删除服务地址\nfunc (s *ServiceDiscovery) DelServiceList(key string) {\n\ts.lock.Lock()\n\tdefer s.lock.Unlock()\n\tdelete(s.serverList, key)\n\ts.cc.NewAddress(s.getServices())\n\tlog.Println(\"del key:\", key)\n}\n\n//GetServices 获取服务地址\nfunc (s *ServiceDiscovery) getServices() []resolver.Address {\n\taddrs := make([]resolver.Address, 0, len(s.serverList))\n\n\tfor _, v := range s.serverList {\n\t\taddrs = append(addrs, v)\n\t}\n\treturn addrs\n}\n```\n\n代码主要修改以下地方：\n\n1. 把获取的服务地址转成`resolver.Address`，供gRPC客户端连接。\n\n2. 根据`schema`的定义规则，修改`key`格式。\n\n服务注册主要修改`key`存储格式，`register.go`\n```go\npackage etcdv3\n\nimport (\n\t\"context\"\n\t\"log\"\n\t\"time\"\n\n\t\"go.etcd.io/etcd/clientv3\"\n)\n\n//ServiceRegister 创建租约注册服务\ntype ServiceRegister struct {\n\tcli *clientv3.Client //etcd client\n\tleaseID clientv3.LeaseID //租约ID\n\t//租约keepalieve相应chan\n\tkeepAliveChan <-chan *clientv3.LeaseKeepAliveResponse\n\tkey string //key\n\tval string //value\n}\n\n//NewServiceRegister 新建注册服务\nfunc NewServiceRegister(endpoints []string, serName, addr string, lease int64) (*ServiceRegister, error) {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: 5 * time.Second,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tser := &ServiceRegister{\n\t\tcli: cli,\n\t\tkey: \"/\" + schema + \"/\" + serName + \"/\" + addr,\n\t\tval: addr,\n\t}\n\n\t//申请租约设置时间keepalive\n\tif err := ser.putKeyWithLease(lease); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn ser, nil\n}\n\n//设置租约\nfunc (s *ServiceRegister) putKeyWithLease(lease int64) error {\n\t//设置租约时间\n\tresp, err := s.cli.Grant(context.Background(), lease)\n\tif err != nil {\n\t\treturn err\n\t}\n\t//注册服务并绑定租约\n\t_, err = s.cli.Put(context.Background(), s.key, s.val, clientv3.WithLease(resp.ID))\n\tif err != nil {\n\t\treturn err\n\t}\n\t//设置续租 定期发送需求请求\n\tleaseRespChan, err := s.cli.KeepAlive(context.Background(), resp.ID)\n\n\tif err != nil {\n\t\treturn err\n\t}\n\ts.leaseID = resp.ID\n\ts.keepAliveChan = leaseRespChan\n\tlog.Printf(\"Put key:%s val:%s success!\", s.key, s.val)\n\treturn nil\n}\n\n//ListenLeaseRespChan 监听 续租情况\nfunc (s *ServiceRegister) ListenLeaseRespChan() {\n\tfor leaseKeepResp := range s.keepAliveChan {\n\t\tlog.Println(\"续约成功\", leaseKeepResp)\n\t}\n\tlog.Println(\"关闭续租\")\n}\n\n// Close 注销服务\nfunc (s *ServiceRegister) Close() error {\n\t//撤销租约\n\tif _, err := s.cli.Revoke(context.Background(), s.leaseID); err != nil {\n\t\treturn err\n\t}\n\tlog.Println(\"撤销租约\")\n\treturn s.cli.Close()\n}\n```\n\n客户端修改gRPC连接服务的部分代码即可：\n```go\nfunc main() {\n\tr := etcdv3.NewServiceDiscovery(EtcdEndpoints)\n\tresolver.Register(r)\n\t// 连接服务器\n\tconn, err := grpc.Dial(r.Scheme()+\"://8.8.8.8/simple_grpc\", grpc.WithBalancerName(\"round_robin\"), grpc.WithInsecure())\n\tif err != nil {\n\t\tlog.Fatalf(\"net.Connect err: %v\", err)\n\t}\n\tdefer conn.Close()\n\n\t// 建立gRPC连接\n\tgrpcClient = pb.NewSimpleClient(conn)\n```\ngRPC内置了简单的负载均衡策略`round_robin`，根据负载均衡地址，以轮询的方式进行调用服务。\n\n服务端启动时，把服务地址注册到`etcd`中即可：\n```go\nfunc main() {\n\t// 监听本地端口\n\tlistener, err := net.Listen(Network, Address)\n\tif err != nil {\n\t\tlog.Fatalf(\"net.Listen err: %v\", err)\n\t}\n\tlog.Println(Address + \" net.Listing...\")\n\t// 新建gRPC服务器实例\n\tgrpcServer := grpc.NewServer()\n\t// 在gRPC服务器注册我们的服务\n\tpb.RegisterSimpleServer(grpcServer, &SimpleService{})\n\t//把服务注册到etcd\n\tser, err := etcdv3.NewServiceRegister(EtcdEndpoints, SerName, Address, 5)\n\tif err != nil {\n\t\tlog.Fatalf(\"register service err: %v\", err)\n\t}\n\tdefer ser.Close()\n\t//用服务器 Serve() 方法以及我们的端口信息区实现阻塞等待，直到进程被杀死或者 Stop() 被调用\n\terr = grpcServer.Serve(listener)\n\tif err != nil {\n\t\tlog.Fatalf(\"grpcServer.Serve err: %v\", err)\n\t}\n}\n```\n\n### 运行效果\n我们先启动并注册三个服务\n\n![](https://img2020.cnblogs.com/blog/1508611/202005/1508611-20200518201520301-2141314089.png)\n\n![](https://img2020.cnblogs.com/blog/1508611/202005/1508611-20200518201526062-1105611810.png)\n\n![](https://img2020.cnblogs.com/blog/1508611/202005/1508611-20200518201529806-1864982377.png)\n\n然后客户端进行调用\n\n![](https://img2020.cnblogs.com/blog/1508611/202005/1508611-20200518201645385-8940133.png)\n\n看服务端接收到的请求\n\n![](https://img2020.cnblogs.com/blog/1508611/202005/1508611-20200518201919646-136721041.png)\n\n![](https://img2020.cnblogs.com/blog/1508611/202005/1508611-20200518201925163-1429636105.png)\n\n![](https://img2020.cnblogs.com/blog/1508611/202005/1508611-20200518201929077-822092499.png)\n\n关闭`localhost:8000`服务，剩余`localhost:8001`和`localhost:8002`服务接收请求\n\n![](https://img2020.cnblogs.com/blog/1508611/202005/1508611-20200518202359155-2143850614.png)\n\n![](https://img2020.cnblogs.com/blog/1508611/202005/1508611-20200518202405272-1990664274.png)\n\n重新打开`localhost:8000`服务\n\n![](https://img2020.cnblogs.com/blog/1508611/202005/1508611-20200518202655967-135791051.png)\n\n![](https://img2020.cnblogs.com/blog/1508611/202005/1508611-20200518202700598-298101288.png)\n\n![](https://img2020.cnblogs.com/blog/1508611/202005/1508611-20200518202703530-602882933.png)\n\n可以看到，gRPC客户端负载均衡运行良好。\n\n### 总结\n本文介绍了gRPC客户端负载均衡的实现，它简单实现了gRPC负载均衡的功能。但在对接其他语言时候比较麻烦，需要每种语言都实现一套服务发现和负载策略，且如果要较为复杂的负载策略，需要修改客户端代码才能完成。\n\n下篇将介绍如何实现官方推荐的负载均衡策略（`External Load Balancing Service`）。\n\n源码地址：https://github.com/Bingjian-Zhu/etcd-example\n\n参考：\n\n* https://segmentfault.com/a/1190000008672912\n\n* https://github.com/wothing/wonaming" }, { "path": "4-etcd-grpclb/client/client.go", "content": "package main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\t\"strconv\"\n\t\"time\"\n\n\t\"google.golang.org/grpc\"\n\t\"google.golang.org/grpc/resolver\"\n\n\t\"etcd-example/4-etcd-grpclb/etcdv3\"\n\tpb \"etcd-example/4-etcd-grpclb/proto\"\n)\n\nvar (\n\t// EtcdEndpoints etcd地址\n\tEtcdEndpoints = []string{\"localhost:2379\"}\n\t// SerName 服务名称\n\tSerName = \"simple_grpc\"\n\tgrpcClient pb.SimpleClient\n)\n\nfunc main() {\n\tr := etcdv3.NewServiceDiscovery(EtcdEndpoints)\n\tresolver.Register(r)\n\t// 连接服务器\n\tconn, err := grpc.Dial(\n\t\tfmt.Sprintf(\"%s:///%s\", r.Scheme(), SerName),\n\t\tgrpc.WithBalancerName(\"round_robin\"),\n\t\tgrpc.WithInsecure(),\n\t)\n\tif err != nil {\n\t\tlog.Fatalf(\"net.Connect err: %v\", err)\n\t}\n\tdefer conn.Close()\n\n\t// 建立gRPC连接\n\tgrpcClient = pb.NewSimpleClient(conn)\n\tfor i := 0; i < 100; i++ {\n\t\troute(i)\n\t\ttime.Sleep(1 * time.Second)\n\t}\n\n}\n\n// route 调用服务端Route方法\nfunc route(i int) {\n\t// 创建发送结构体\n\treq := pb.SimpleRequest{\n\t\tData: \"grpc \" + strconv.Itoa(i),\n\t}\n\t// 调用我们的服务(Route方法)\n\t// 同时传入了一个 context.Context ，在有需要时可以让我们改变RPC的行为，比如超时/取消一个正在运行的RPC\n\tres, err := grpcClient.Route(context.Background(), &req)\n\tif err != nil {\n\t\tlog.Fatalf(\"Call Route err: %v\", err)\n\t}\n\t// 打印返回值\n\tlog.Println(res)\n}\n" }, { "path": "4-etcd-grpclb/etcdv3/discovery.go", "content": "package etcdv3\n\nimport (\n\t\"context\"\n\t\"log\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/coreos/etcd/mvcc/mvccpb\"\n\t\"go.etcd.io/etcd/clientv3\"\n\t\"google.golang.org/grpc/resolver\"\n)\n\nconst schema = \"grpclb\"\n\n//ServiceDiscovery 服务发现\ntype ServiceDiscovery struct {\n\tcli *clientv3.Client //etcd client\n\tcc resolver.ClientConn\n\tserverList sync.Map //服务列表\n}\n\n//NewServiceDiscovery 新建发现服务\nfunc NewServiceDiscovery(endpoints []string) resolver.Builder {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: 5 * time.Second,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\treturn &ServiceDiscovery{\n\t\tcli: cli,\n\t}\n}\n\n//Build 为给定目标创建一个新的`resolver`，当调用`grpc.Dial()`时执行\nfunc (s *ServiceDiscovery) Build(target resolver.Target, cc resolver.ClientConn, opts resolver.BuildOption) (resolver.Resolver, error) {\n\tlog.Println(\"Build\")\n\ts.cc = cc\n\tprefix := \"/\" + target.Scheme + \"/\" + target.Endpoint + \"/\"\n\t//根据前缀获取现有的key\n\tresp, err := s.cli.Get(context.Background(), prefix, clientv3.WithPrefix())\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfor _, ev := range resp.Kvs {\n\t\ts.SetServiceList(string(ev.Key), string(ev.Value))\n\t}\n\ts.cc.UpdateState(resolver.State{Addresses: s.getServices()})\n\t//监视前缀，修改变更的server\n\tgo s.watcher(prefix)\n\treturn s, nil\n}\n\n// ResolveNow 监视目标更新\nfunc (s *ServiceDiscovery) ResolveNow(rn resolver.ResolveNowOption) {\n\tlog.Println(\"ResolveNow\")\n}\n\n//Scheme return schema\nfunc (s *ServiceDiscovery) Scheme() string {\n\treturn schema\n}\n\n//Close 关闭\nfunc (s *ServiceDiscovery) Close() {\n\tlog.Println(\"Close\")\n\ts.cli.Close()\n}\n\n//watcher 监听前缀\nfunc (s *ServiceDiscovery) watcher(prefix string) {\n\trch := s.cli.Watch(context.Background(), prefix, clientv3.WithPrefix())\n\tlog.Printf(\"watching prefix:%s now...\", prefix)\n\tfor wresp := range rch {\n\t\tfor _, ev := range wresp.Events {\n\t\t\tswitch ev.Type {\n\t\t\tcase mvccpb.PUT: //新增或修改\n\t\t\t\ts.SetServiceList(string(ev.Kv.Key), string(ev.Kv.Value))\n\t\t\tcase mvccpb.DELETE: //删除\n\t\t\t\ts.DelServiceList(string(ev.Kv.Key))\n\t\t\t}\n\t\t}\n\t}\n}\n\n//SetServiceList 新增服务地址\nfunc (s *ServiceDiscovery) SetServiceList(key, val string) {\n\ts.serverList.Store(key, resolver.Address{Addr: val})\n\ts.cc.UpdateState(resolver.State{Addresses: s.getServices()})\n\tlog.Println(\"put key :\", key, \"val:\", val)\n}\n\n//DelServiceList 删除服务地址\nfunc (s *ServiceDiscovery) DelServiceList(key string) {\n\ts.serverList.Delete(key)\n\ts.cc.UpdateState(resolver.State{Addresses: s.getServices()})\n\tlog.Println(\"del key:\", key)\n}\n\n//GetServices 获取服务地址\nfunc (s *ServiceDiscovery) getServices() []resolver.Address {\n\taddrs := make([]resolver.Address, 0, 10)\n\ts.serverList.Range(func(k, v interface{}) bool {\n\t\taddrs = append(addrs, v.(resolver.Address))\n\t\treturn true\n\t})\n\treturn addrs\n}\n" }, { "path": "4-etcd-grpclb/etcdv3/register.go", "content": "package etcdv3\n\nimport (\n\t\"context\"\n\t\"log\"\n\t\"time\"\n\n\t\"go.etcd.io/etcd/clientv3\"\n)\n\n//ServiceRegister 创建租约注册服务\ntype ServiceRegister struct {\n\tcli *clientv3.Client //etcd client\n\tleaseID clientv3.LeaseID //租约ID\n\t//租约keepalieve相应chan\n\tkeepAliveChan <-chan *clientv3.LeaseKeepAliveResponse\n\tkey string //key\n\tval string //value\n}\n\n//NewServiceRegister 新建注册服务\nfunc NewServiceRegister(endpoints []string, serName, addr string, lease int64) (*ServiceRegister, error) {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: 5 * time.Second,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tser := &ServiceRegister{\n\t\tcli: cli,\n\t\tkey: \"/\" + schema + \"/\" + serName + \"/\" + addr,\n\t\tval: addr,\n\t}\n\n\t//申请租约设置时间keepalive\n\tif err := ser.putKeyWithLease(lease); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn ser, nil\n}\n\n//设置租约\nfunc (s *ServiceRegister) putKeyWithLease(lease int64) error {\n\t//设置租约时间\n\tresp, err := s.cli.Grant(context.Background(), lease)\n\tif err != nil {\n\t\treturn err\n\t}\n\t//注册服务并绑定租约\n\t_, err = s.cli.Put(context.Background(), s.key, s.val, clientv3.WithLease(resp.ID))\n\tif err != nil {\n\t\treturn err\n\t}\n\t//设置续租 定期发送需求请求\n\tleaseRespChan, err := s.cli.KeepAlive(context.Background(), resp.ID)\n\n\tif err != nil {\n\t\treturn err\n\t}\n\ts.leaseID = resp.ID\n\ts.keepAliveChan = leaseRespChan\n\tlog.Printf(\"Put key:%s val:%s success!\", s.key, s.val)\n\treturn nil\n}\n\n//ListenLeaseRespChan 监听 续租情况\nfunc (s *ServiceRegister) ListenLeaseRespChan() {\n\tfor leaseKeepResp := range s.keepAliveChan {\n\t\tlog.Println(\"续约成功\", leaseKeepResp)\n\t}\n\tlog.Println(\"关闭续租\")\n}\n\n// Close 注销服务\nfunc (s *ServiceRegister) Close() error {\n\t//撤销租约\n\tif _, err := s.cli.Revoke(context.Background(), s.leaseID); err != nil {\n\t\treturn err\n\t}\n\tlog.Println(\"撤销租约\")\n\treturn s.cli.Close()\n}\n" }, { "path": "4-etcd-grpclb/proto/simple.pb.go", "content": "// Code generated by protoc-gen-go. DO NOT EDIT.\n// source: 2-simple_rpc/proto/simple.proto\n\npackage proto\n\nimport (\n\tcontext \"context\"\n\tfmt \"fmt\"\n\tproto \"github.com/golang/protobuf/proto\"\n\tgrpc \"google.golang.org/grpc\"\n\tcodes \"google.golang.org/grpc/codes\"\n\tstatus \"google.golang.org/grpc/status\"\n\tmath \"math\"\n)\n\n// Reference imports to suppress errors if they are not otherwise used.\nvar _ = proto.Marshal\nvar _ = fmt.Errorf\nvar _ = math.Inf\n\n// This is a compile-time assertion to ensure that this generated file\n// is compatible with the proto package it is being compiled against.\n// A compilation error at this line likely means your copy of the\n// proto package needs to be updated.\nconst _ = proto.ProtoPackageIsVersion3 // please upgrade the proto package\n\n// 定义发送请求信息\ntype SimpleRequest struct {\n\t// 定义发送的参数，采用驼峰命名方式，小写加下划线，如：student_name\n\t// 参数类型 参数名 标识号(不可重复)\n\tData string `protobuf:\"bytes,1,opt,name=data,proto3\" json:\"data,omitempty\"`\n\tXXX_NoUnkeyedLiteral struct{} `json:\"-\"`\n\tXXX_unrecognized []byte `json:\"-\"`\n\tXXX_sizecache int32 `json:\"-\"`\n}\n\nfunc (m *SimpleRequest) Reset() { *m = SimpleRequest{} }\nfunc (m *SimpleRequest) String() string { return proto.CompactTextString(m) }\nfunc (*SimpleRequest) ProtoMessage() {}\nfunc (*SimpleRequest) Descriptor() ([]byte, []int) {\n\treturn fileDescriptor_31047b63fe44dee8, []int{0}\n}\n\nfunc (m *SimpleRequest) XXX_Unmarshal(b []byte) error {\n\treturn xxx_messageInfo_SimpleRequest.Unmarshal(m, b)\n}\nfunc (m *SimpleRequest) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {\n\treturn xxx_messageInfo_SimpleRequest.Marshal(b, m, deterministic)\n}\nfunc (m *SimpleRequest) XXX_Merge(src proto.Message) {\n\txxx_messageInfo_SimpleRequest.Merge(m, src)\n}\nfunc (m *SimpleRequest) XXX_Size() int {\n\treturn xxx_messageInfo_SimpleRequest.Size(m)\n}\nfunc (m *SimpleRequest) XXX_DiscardUnknown() {\n\txxx_messageInfo_SimpleRequest.DiscardUnknown(m)\n}\n\nvar xxx_messageInfo_SimpleRequest proto.InternalMessageInfo\n\nfunc (m *SimpleRequest) GetData() string {\n\tif m != nil {\n\t\treturn m.Data\n\t}\n\treturn \"\"\n}\n\n// 定义响应信息\ntype SimpleResponse struct {\n\t// 定义接收的参数\n\t// 参数类型 参数名 标识号(不可重复)\n\tCode int32 `protobuf:\"varint,1,opt,name=code,proto3\" json:\"code,omitempty\"`\n\tValue string `protobuf:\"bytes,2,opt,name=value,proto3\" json:\"value,omitempty\"`\n\tXXX_NoUnkeyedLiteral struct{} `json:\"-\"`\n\tXXX_unrecognized []byte `json:\"-\"`\n\tXXX_sizecache int32 `json:\"-\"`\n}\n\nfunc (m *SimpleResponse) Reset() { *m = SimpleResponse{} }\nfunc (m *SimpleResponse) String() string { return proto.CompactTextString(m) }\nfunc (*SimpleResponse) ProtoMessage() {}\nfunc (*SimpleResponse) Descriptor() ([]byte, []int) {\n\treturn fileDescriptor_31047b63fe44dee8, []int{1}\n}\n\nfunc (m *SimpleResponse) XXX_Unmarshal(b []byte) error {\n\treturn xxx_messageInfo_SimpleResponse.Unmarshal(m, b)\n}\nfunc (m *SimpleResponse) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {\n\treturn xxx_messageInfo_SimpleResponse.Marshal(b, m, deterministic)\n}\nfunc (m *SimpleResponse) XXX_Merge(src proto.Message) {\n\txxx_messageInfo_SimpleResponse.Merge(m, src)\n}\nfunc (m *SimpleResponse) XXX_Size() int {\n\treturn xxx_messageInfo_SimpleResponse.Size(m)\n}\nfunc (m *SimpleResponse) XXX_DiscardUnknown() {\n\txxx_messageInfo_SimpleResponse.DiscardUnknown(m)\n}\n\nvar xxx_messageInfo_SimpleResponse proto.InternalMessageInfo\n\nfunc (m *SimpleResponse) GetCode() int32 {\n\tif m != nil {\n\t\treturn m.Code\n\t}\n\treturn 0\n}\n\nfunc (m *SimpleResponse) GetValue() string {\n\tif m != nil {\n\t\treturn m.Value\n\t}\n\treturn \"\"\n}\n\nfunc init() {\n\tproto.RegisterType((*SimpleRequest)(nil), \"proto.SimpleRequest\")\n\tproto.RegisterType((*SimpleResponse)(nil), \"proto.SimpleResponse\")\n}\n\nfunc init() { proto.RegisterFile(\"2-simple_rpc/proto/simple.proto\", fileDescriptor_31047b63fe44dee8) }\n\nvar fileDescriptor_31047b63fe44dee8 = []byte{\n\t// 158 bytes of a gzipped FileDescriptorProto\n\t0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xe2, 0x92, 0x37, 0xd2, 0x2d, 0xce,\n\t0xcc, 0x2d, 0xc8, 0x49, 0x8d, 0x2f, 0x2a, 0x48, 0xd6, 0x2f, 0x28, 0xca, 0x2f, 0xc9, 0xd7, 0x87,\n\t0x08, 0xe8, 0x81, 0x39, 0x42, 0xac, 0x60, 0x4a, 0x49, 0x99, 0x8b, 0x37, 0x18, 0x2c, 0x1c, 0x94,\n\t0x5a, 0x58, 0x9a, 0x5a, 0x5c, 0x22, 0x24, 0xc4, 0xc5, 0x92, 0x92, 0x58, 0x92, 0x28, 0xc1, 0xa8,\n\t0xc0, 0xa8, 0xc1, 0x19, 0x04, 0x66, 0x2b, 0x59, 0x71, 0xf1, 0xc1, 0x14, 0x15, 0x17, 0xe4, 0xe7,\n\t0x15, 0xa7, 0x82, 0x54, 0x25, 0xe7, 0xa7, 0xa4, 0x82, 0x55, 0xb1, 0x06, 0x81, 0xd9, 0x42, 0x22,\n\t0x5c, 0xac, 0x65, 0x89, 0x39, 0xa5, 0xa9, 0x12, 0x4c, 0x60, 0xad, 0x10, 0x8e, 0x91, 0x03, 0x17,\n\t0x1b, 0x44, 0xaf, 0x90, 0x19, 0x17, 0x6b, 0x50, 0x7e, 0x69, 0x49, 0xaa, 0x90, 0x08, 0xc4, 0x09,\n\t0x7a, 0x28, 0x16, 0x4b, 0x89, 0xa2, 0x89, 0x42, 0x6c, 0x52, 0x62, 0x48, 0x62, 0x03, 0x8b, 0x1b,\n\t0x03, 0x02, 0x00, 0x00, 0xff, 0xff, 0x50, 0xd7, 0x51, 0x6d, 0xd3, 0x00, 0x00, 0x00,\n}\n\n// Reference imports to suppress errors if they are not otherwise used.\nvar _ context.Context\nvar _ grpc.ClientConn\n\n// This is a compile-time assertion to ensure that this generated file\n// is compatible with the grpc package it is being compiled against.\nconst _ = grpc.SupportPackageIsVersion4\n\n// SimpleClient is the client API for Simple service.\n//\n// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://godoc.org/google.golang.org/grpc#ClientConn.NewStream.\ntype SimpleClient interface {\n\tRoute(ctx context.Context, in *SimpleRequest, opts ...grpc.CallOption) (*SimpleResponse, error)\n}\n\ntype simpleClient struct {\n\tcc *grpc.ClientConn\n}\n\nfunc NewSimpleClient(cc *grpc.ClientConn) SimpleClient {\n\treturn &simpleClient{cc}\n}\n\nfunc (c *simpleClient) Route(ctx context.Context, in *SimpleRequest, opts ...grpc.CallOption) (*SimpleResponse, error) {\n\tout := new(SimpleResponse)\n\terr := c.cc.Invoke(ctx, \"/proto.Simple/Route\", in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\n// SimpleServer is the server API for Simple service.\ntype SimpleServer interface {\n\tRoute(context.Context, *SimpleRequest) (*SimpleResponse, error)\n}\n\n// UnimplementedSimpleServer can be embedded to have forward compatible implementations.\ntype UnimplementedSimpleServer struct {\n}\n\nfunc (*UnimplementedSimpleServer) Route(ctx context.Context, req *SimpleRequest) (*SimpleResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method Route not implemented\")\n}\n\nfunc RegisterSimpleServer(s *grpc.Server, srv SimpleServer) {\n\ts.RegisterService(&_Simple_serviceDesc, srv)\n}\n\nfunc _Simple_Route_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(SimpleRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(SimpleServer).Route(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: \"/proto.Simple/Route\",\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(SimpleServer).Route(ctx, req.(*SimpleRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\nvar _Simple_serviceDesc = grpc.ServiceDesc{\n\tServiceName: \"proto.Simple\",\n\tHandlerType: (*SimpleServer)(nil),\n\tMethods: []grpc.MethodDesc{\n\t\t{\n\t\t\tMethodName: \"Route\",\n\t\t\tHandler: _Simple_Route_Handler,\n\t\t},\n\t},\n\tStreams: []grpc.StreamDesc{},\n\tMetadata: \"2-simple_rpc/proto/simple.proto\",\n}\n" }, { "path": "4-etcd-grpclb/proto/simple.proto", "content": "syntax = \"proto3\";// 协议为proto3\n\npackage proto;\n\n// 定义发送请求信息\nmessage SimpleRequest{\n // 定义发送的参数，采用驼峰命名方式，小写加下划线，如：student_name\n // 参数类型 参数名 标识号(不可重复)\n string data = 1;\n}\n\n// 定义响应信息\nmessage SimpleResponse{\n // 定义接收的参数\n // 参数类型 参数名 标识号(不可重复)\n int32 code = 1;\n string value = 2;\n}\n\n// 定义我们的服务（可定义多个服务,每个服务可定义多个接口）\nservice Simple{\n rpc Route (SimpleRequest) returns (SimpleResponse){};\n}" }, { "path": "4-etcd-grpclb/server/server.go", "content": "package main\n\nimport (\n\t\"context\"\n\t\"log\"\n\t\"net\"\n\n\t\"google.golang.org/grpc\"\n\n\t\"etcd-example/4-etcd-grpclb/etcdv3\"\n\tpb \"etcd-example/4-etcd-grpclb/proto\"\n)\n\n// SimpleService 定义我们的服务\ntype SimpleService struct{}\n\nconst (\n\t// Address 监听地址\n\tAddress string = \"localhost:8000\"\n\t// Network 网络通信协议\n\tNetwork string = \"tcp\"\n\t// SerName 服务名称\n\tSerName string = \"simple_grpc\"\n)\n\n// EtcdEndpoints etcd地址\nvar EtcdEndpoints = []string{\"localhost:2379\"}\n\nfunc main() {\n\t// 监听本地端口\n\tlistener, err := net.Listen(Network, Address)\n\tif err != nil {\n\t\tlog.Fatalf(\"net.Listen err: %v\", err)\n\t}\n\tlog.Println(Address + \" net.Listing...\")\n\t// 新建gRPC服务器实例\n\tgrpcServer := grpc.NewServer()\n\t// 在gRPC服务器注册我们的服务\n\tpb.RegisterSimpleServer(grpcServer, &SimpleService{})\n\t//把服务注册到etcd\n\tser, err := etcdv3.NewServiceRegister(EtcdEndpoints, SerName, Address, 5)\n\tif err != nil {\n\t\tlog.Fatalf(\"register service err: %v\", err)\n\t}\n\tdefer ser.Close()\n\t//用服务器 Serve() 方法以及我们的端口信息区实现阻塞等待，直到进程被杀死或者 Stop() 被调用\n\terr = grpcServer.Serve(listener)\n\tif err != nil {\n\t\tlog.Fatalf(\"grpcServer.Serve err: %v\", err)\n\t}\n}\n\n// Route 实现Route方法\nfunc (s *SimpleService) Route(ctx context.Context, req *pb.SimpleRequest) (*pb.SimpleResponse, error) {\n\tlog.Println(\"receive: \" + req.Data)\n\tres := pb.SimpleResponse{\n\t\tCode: 200,\n\t\tValue: \"hello \" + req.Data,\n\t}\n\treturn &res, nil\n}\n" }, { "path": "5-etcd-grpclb-balancer/README.md", "content": "### gRPC负载均衡（自定义负载均衡策略）\n\n### 前言\n上篇文章介绍了如何实现gRPC负载均衡，但目前官方只提供了`pick_first`和`round_robin`两种负载均衡策略，轮询法`round_robin`不能满足因服务器配置不同而承担不同负载量，这篇文章将介绍如何实现自定义负载均衡策略--`加权随机法`。\n\n`加权随机法`可以根据服务器的处理能力而分配不同的权重，从而实现处理能力高的服务器可承担更多的请求，处理能力低的服务器少承担请求。\n\n### 自定义负载均衡策略\n\ngRPC提供了`V2PickerBuilder`和`V2Picker`接口让我们实现自己的负载均衡策略。\n\n```go\ntype V2PickerBuilder interface {\n\tBuild(info PickerBuildInfo) balancer.V2Picker\n}\n```\n`V2PickerBuilder`接口：创建V2版本的子连接选择器。\n\n`Build`方法：返回一个V2选择器，将用于gRPC选择子连接。\n\n```go\ntype V2Picker interface {\n\tPick(info PickInfo) (PickResult, error)\n}\n```\n`V2Picker `接口：用于gRPC选择子连接去发送请求。\n`Pick`方法：子连接选择\n\n问题来了，我们需要把服务器地址的权重添加进去，但是地址`resolver.Address`并没有提供权重的属性。官方给的答复是：把权重存储到地址的元数据`metadata`中。\n\n```go\n// attributeKey is the type used as the key to store AddrInfo in the Attributes\n// field of resolver.Address.\ntype attributeKey struct{}\n\n// AddrInfo will be stored inside Address metadata in order to use weighted balancer.\ntype AddrInfo struct {\n\tWeight int\n}\n\n// SetAddrInfo returns a copy of addr in which the Attributes field is updated\n// with addrInfo.\nfunc SetAddrInfo(addr resolver.Address, addrInfo AddrInfo) resolver.Address {\n\taddr.Attributes = attributes.New()\n\taddr.Attributes = addr.Attributes.WithValues(attributeKey{}, addrInfo)\n\treturn addr\n}\n\n// GetAddrInfo returns the AddrInfo stored in the Attributes fields of addr.\nfunc GetAddrInfo(addr resolver.Address) AddrInfo {\n\tv := addr.Attributes.Value(attributeKey{})\n\tai, _ := v.(AddrInfo)\n\treturn ai\n}\n```\n定义`AddrInfo`结构体并添加权重`Weight`属性，`Set`方法把`Weight`存储到`resolver.Address`中，`Get`方法从`resolver.Address`获取`Weight`。\n\n解决权重存储问题后，接下来我们实现加权随机法负载均衡策略。\n\n首先实现`V2PickerBuilder`接口，返回子连接选择器。\n```go\nfunc (*rrPickerBuilder) Build(info base.PickerBuildInfo) balancer.V2Picker {\n\tgrpclog.Infof(\"weightPicker: newPicker called with info: %v\", info)\n\tif len(info.ReadySCs) == 0 {\n\t\treturn base.NewErrPickerV2(balancer.ErrNoSubConnAvailable)\n\t}\n\tvar scs []balancer.SubConn\n\tfor subConn, addr := range info.ReadySCs {\n\t\tnode := GetAddrInfo(addr.Address)\n\t\tif node.Weight <= 0 {\n\t\t\tnode.Weight = minWeight\n\t\t} else if node.Weight > 5 {\n\t\t\tnode.Weight = maxWeight\n\t\t}\n\t\tfor i := 0; i < node.Weight; i++ {\n\t\t\tscs = append(scs, subConn)\n\t\t}\n\t}\n\treturn &rrPicker{\n\t\tsubConns: scs,\n\t}\n}\n```\n`加权随机法`中，我使用空间换时间的方式，把权重转成地址个数（例如`addr1`的权重是`3`，那么添加`3`个子连接到切片中；`addr2`权重为`1`，则添加`1`个子连接；选择子连接时候，按子连接切片长度生成随机数，以随机数作为下标就是选中的子连接），避免重复计算权重。考虑到内存占用，权重定义从`1`到`5`权重。\n\n接下来实现子连接的选择，获取随机数，选择子连接\n```go\ntype rrPicker struct {\n\tsubConns []balancer.SubConn\n\tmu sync.Mutex\n}\n\nfunc (p *rrPicker) Pick(balancer.PickInfo) (balancer.PickResult, error) {\n\tp.mu.Lock()\n\tindex := rand.Intn(len(p.subConns))\n\tsc := p.subConns[index]\n\tp.mu.Unlock()\n\treturn balancer.PickResult{SubConn: sc}, nil\n}\n```\n\n关键代码完成后，我们把加权随机法负载均衡策略命名为`weight`，并注册到gRPC的负载均衡策略中。\n```go\n// Name is the name of weight balancer.\nconst Name = \"weight\"\n// NewBuilder creates a new weight balancer builder.\nfunc newBuilder() balancer.Builder {\n\treturn base.NewBalancerBuilderV2(Name, &rrPickerBuilder{}, base.Config{HealthCheck: false})\n}\n\nfunc init() {\n\tbalancer.Register(newBuilder())\n}\n```\n\n完整代码[weight.go](https://github.com/Bingjian-Zhu/etcd-example/blob/master/5-etcd-grpclb-balancer/balancer/weight/weight.go)\n\n最后，我们只需要在服务端注册服务时候附带权重，然后客户端在服务发现时把权重`Set`到`resolver.Address`中，最后客户端把负载论衡策略改成`weight`就完成了。\n\n```go\n//SetServiceList 设置服务地址\nfunc (s *ServiceDiscovery) SetServiceList(key, val string) {\n\ts.lock.Lock()\n\tdefer s.lock.Unlock()\n\t//获取服务地址\n\taddr := resolver.Address{Addr: strings.TrimPrefix(key, s.prefix)}\n\t//获取服务地址权重\n\tnodeWeight, err := strconv.Atoi(val)\n\tif err != nil {\n\t\t//非数字字符默认权重为1\n\t\tnodeWeight = 1\n\t}\n\t//把服务地址权重存储到resolver.Address的元数据中\n\taddr = weight.SetAddrInfo(addr, weight.AddrInfo{Weight: nodeWeight})\n\ts.serverList[key] = addr\n\ts.cc.UpdateState(resolver.State{Addresses: s.getServices()})\n\tlog.Println(\"put key :\", key, \"wieght:\", val)\n}\n```\n\n客户端使用`weight`负载均衡策略\n\n```go\nfunc main() {\n\tr := etcdv3.NewServiceDiscovery(EtcdEndpoints)\n\tresolver.Register(r)\n\t// 连接服务器\n\tconn, err := grpc.Dial(\n\t\tfmt.Sprintf(\"%s:///%s\", r.Scheme(), SerName),\n\t\tgrpc.WithBalancerName(\"weight\"),\n\t\tgrpc.WithInsecure(),\n\t)\n\tif err != nil {\n\t\tlog.Fatalf(\"net.Connect err: %v\", err)\n\t}\n\tdefer conn.Close()\n```\n\n运行效果：\n\n运行`服务1`，权重为`1`\n\n![](https://img2020.cnblogs.com/blog/1508611/202005/1508611-20200520162934052-74794177.png)\n\n运行`服务2`，权重为`4`\n\n![](https://img2020.cnblogs.com/blog/1508611/202005/1508611-20200520162941378-1116335906.png)\n\n运行客户端\n\n![](https://img2020.cnblogs.com/blog/1508611/202005/1508611-20200520163515073-1148862720.png)\n\n查看前50次请求在`服务1`和`服务器2`的负载情况。`服务1`分配了`9`次请求，`服务2`分配了`41`次请求，接近权重比值。\n\n![](https://img2020.cnblogs.com/blog/1508611/202005/1508611-20200520163753358-1654741743.png)\n\n![](https://img2020.cnblogs.com/blog/1508611/202005/1508611-20200520163932810-2034341622.png)\n\n断开`服务2`，所有请求流向`服务1`\n\n![](https://img2020.cnblogs.com/blog/1508611/202005/1508611-20200520164432399-923288256.png)\n\n以权重为`4`，重启`服务2`，请求以加权随机法流向两个服务器\n\n![](https://img2020.cnblogs.com/blog/1508611/202005/1508611-20200520164648568-1117742551.png)\n\n### 总结\n\n本篇文章以加权随机法为例，介绍了如何实现gRPC自定义负载均衡策略，以满足我们的需求。\n\n源码地址：https://github.com/Bingjian-Zhu/etcd-example" }, { "path": "5-etcd-grpclb-balancer/balancer/weight/weight.go", "content": "package weight\n\nimport (\n\t\"math/rand\"\n\t\"sync\"\n\n\t\"google.golang.org/grpc/attributes\"\n\t\"google.golang.org/grpc/balancer\"\n\t\"google.golang.org/grpc/balancer/base\"\n\t\"google.golang.org/grpc/grpclog\"\n\t\"google.golang.org/grpc/resolver\"\n)\n\n// Name is the name of weight balancer.\nconst Name = \"weight\"\n\nvar (\n\tminWeight = 1\n\tmaxWeight = 5\n)\n\n// attributeKey is the type used as the key to store AddrInfo in the Attributes\n// field of resolver.Address.\ntype attributeKey struct{}\n\n// AddrInfo will be stored inside Address metadata in order to use weighted balancer.\ntype AddrInfo struct {\n\tWeight int\n}\n\n// SetAddrInfo returns a copy of addr in which the Attributes field is updated\n// with addrInfo.\nfunc SetAddrInfo(addr resolver.Address, addrInfo AddrInfo) resolver.Address {\n\taddr.Attributes = attributes.New()\n\taddr.Attributes = addr.Attributes.WithValues(attributeKey{}, addrInfo)\n\treturn addr\n}\n\n// GetAddrInfo returns the AddrInfo stored in the Attributes fields of addr.\nfunc GetAddrInfo(addr resolver.Address) AddrInfo {\n\tv := addr.Attributes.Value(attributeKey{})\n\tai, _ := v.(AddrInfo)\n\treturn ai\n}\n\n// NewBuilder creates a new weight balancer builder.\nfunc newBuilder() balancer.Builder {\n\treturn base.NewBalancerBuilderV2(Name, &rrPickerBuilder{}, base.Config{HealthCheck: false})\n}\n\nfunc init() {\n\tbalancer.Register(newBuilder())\n}\n\ntype rrPickerBuilder struct{}\n\nfunc (*rrPickerBuilder) Build(info base.PickerBuildInfo) balancer.V2Picker {\n\tgrpclog.Infof(\"weightPicker: newPicker called with info: %v\", info)\n\tif len(info.ReadySCs) == 0 {\n\t\treturn base.NewErrPickerV2(balancer.ErrNoSubConnAvailable)\n\t}\n\tvar scs []balancer.SubConn\n\tfor subConn, addr := range info.ReadySCs {\n\t\tnode := GetAddrInfo(addr.Address)\n\t\tif node.Weight <= 0 {\n\t\t\tnode.Weight = minWeight\n\t\t} else if node.Weight > 5 {\n\t\t\tnode.Weight = maxWeight\n\t\t}\n\t\tfor i := 0; i < node.Weight; i++ {\n\t\t\tscs = append(scs, subConn)\n\t\t}\n\t}\n\treturn &rrPicker{\n\t\tsubConns: scs,\n\t}\n}\n\ntype rrPicker struct {\n\t// subConns is the snapshot of the roundrobin balancer when this picker was\n\t// created. The slice is immutable. Each Get() will do a round robin\n\t// selection from it and return the selected SubConn.\n\tsubConns []balancer.SubConn\n\n\tmu sync.Mutex\n}\n\nfunc (p *rrPicker) Pick(balancer.PickInfo) (balancer.PickResult, error) {\n\tp.mu.Lock()\n\tindex := rand.Intn(len(p.subConns))\n\tsc := p.subConns[index]\n\tp.mu.Unlock()\n\treturn balancer.PickResult{SubConn: sc}, nil\n}\n" }, { "path": "5-etcd-grpclb-balancer/client/client.go", "content": "package main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\t\"strconv\"\n\t\"time\"\n\n\t\"google.golang.org/grpc\"\n\t\"google.golang.org/grpc/resolver\"\n\n\t\"etcd-example/5-etcd-grpclb-balancer/etcdv3\"\n\tpb \"etcd-example/5-etcd-grpclb-balancer/proto\"\n)\n\nvar (\n\t// EtcdEndpoints etcd地址\n\tEtcdEndpoints = []string{\"localhost:2379\"}\n\t// SerName 服务名称\n\tSerName = \"simple_grpc\"\n\tgrpcClient pb.SimpleClient\n)\n\nfunc main() {\n\tr := etcdv3.NewServiceDiscovery(EtcdEndpoints)\n\tresolver.Register(r)\n\t// 连接服务器\n\tconn, err := grpc.Dial(\n\t\tfmt.Sprintf(\"%s:///%s\", r.Scheme(), SerName),\n\t\tgrpc.WithBalancerName(\"weight\"),\n\t\tgrpc.WithInsecure(),\n\t)\n\tif err != nil {\n\t\tlog.Fatalf(\"net.Connect err: %v\", err)\n\t}\n\tdefer conn.Close()\n\n\t// 建立gRPC连接\n\tgrpcClient = pb.NewSimpleClient(conn)\n\tfor i := 0; i < 100; i++ {\n\t\troute(i)\n\t\ttime.Sleep(1 * time.Second)\n\t}\n\n}\n\n// route 调用服务端Route方法\nfunc route(i int) {\n\t// 创建发送结构体\n\treq := pb.SimpleRequest{\n\t\tData: \"grpc \" + strconv.Itoa(i),\n\t}\n\t// 调用我们的服务(Route方法)\n\t// 同时传入了一个 context.Context ，在有需要时可以让我们改变RPC的行为，比如超时/取消一个正在运行的RPC\n\tres, err := grpcClient.Route(context.Background(), &req)\n\tif err != nil {\n\t\tlog.Fatalf(\"Call Route err: %v\", err)\n\t}\n\t// 打印返回值\n\tlog.Println(res)\n}\n" }, { "path": "5-etcd-grpclb-balancer/etcdv3/discovery.go", "content": "package etcdv3\n\nimport (\n\t\"context\"\n\t\"log\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\n\t\"etcd-example/5-etcd-grpclb-balancer/balancer/weight\"\n\n\t\"github.com/coreos/etcd/mvcc/mvccpb\"\n\t\"go.etcd.io/etcd/clientv3\"\n\t\"google.golang.org/grpc/resolver\"\n)\n\nconst schema = \"grpclb\"\n\n//ServiceDiscovery 服务发现\ntype ServiceDiscovery struct {\n\tcli *clientv3.Client //etcd client\n\tcc resolver.ClientConn\n\tserverList sync.Map //服务列表\n\tprefix string //监视的前缀\n}\n\n//NewServiceDiscovery 新建发现服务\nfunc NewServiceDiscovery(endpoints []string) resolver.Builder {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: 5 * time.Second,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\treturn &ServiceDiscovery{\n\t\tcli: cli,\n\t}\n}\n\n//Build 为给定目标创建一个新的`resolver`，当调用`grpc.Dial()`时执行\nfunc (s *ServiceDiscovery) Build(target resolver.Target, cc resolver.ClientConn, opts resolver.BuildOption) (resolver.Resolver, error) {\n\tlog.Println(\"Build\")\n\ts.cc = cc\n\ts.prefix = \"/\" + target.Scheme + \"/\" + target.Endpoint + \"/\"\n\t//根据前缀获取现有的key\n\tresp, err := s.cli.Get(context.Background(), s.prefix, clientv3.WithPrefix())\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tfor _, ev := range resp.Kvs {\n\t\ts.SetServiceList(string(ev.Key), string(ev.Value))\n\t}\n\ts.cc.UpdateState(resolver.State{Addresses: s.getServices()})\n\t//监视前缀，修改变更的server\n\tgo s.watcher()\n\treturn s, nil\n}\n\n// ResolveNow 监视目标更新\nfunc (s *ServiceDiscovery) ResolveNow(rn resolver.ResolveNowOption) {\n\tlog.Println(\"ResolveNow\")\n}\n\n//Scheme return schema\nfunc (s *ServiceDiscovery) Scheme() string {\n\treturn schema\n}\n\n//Close 关闭\nfunc (s *ServiceDiscovery) Close() {\n\tlog.Println(\"Close\")\n\ts.cli.Close()\n}\n\n//watcher 监听前缀\nfunc (s *ServiceDiscovery) watcher() {\n\trch := s.cli.Watch(context.Background(), s.prefix, clientv3.WithPrefix())\n\tlog.Printf(\"watching prefix:%s now...\", s.prefix)\n\tfor wresp := range rch {\n\t\tfor _, ev := range wresp.Events {\n\t\t\tswitch ev.Type {\n\t\t\tcase mvccpb.PUT: //新增或修改\n\t\t\t\ts.SetServiceList(string(ev.Kv.Key), string(ev.Kv.Value))\n\t\t\tcase mvccpb.DELETE: //删除\n\t\t\t\ts.DelServiceList(string(ev.Kv.Key))\n\t\t\t}\n\t\t}\n\t}\n}\n\n//SetServiceList 设置服务地址\nfunc (s *ServiceDiscovery) SetServiceList(key, val string) {\n\t//获取服务地址\n\taddr := resolver.Address{Addr: strings.TrimPrefix(key, s.prefix)}\n\t//获取服务地址权重\n\tnodeWeight, err := strconv.Atoi(val)\n\tif err != nil {\n\t\t//非数字字符默认权重为1\n\t\tnodeWeight = 1\n\t}\n\t//把服务地址权重存储到resolver.Address的元数据中\n\taddr = weight.SetAddrInfo(addr, weight.AddrInfo{Weight: nodeWeight})\n\ts.serverList.Store(key, addr)\n\ts.cc.UpdateState(resolver.State{Addresses: s.getServices()})\n\tlog.Println(\"put key :\", key, \"wieght:\", val)\n}\n\n//DelServiceList 删除服务地址\nfunc (s *ServiceDiscovery) DelServiceList(key string) {\n\ts.serverList.Delete(key)\n\ts.cc.UpdateState(resolver.State{Addresses: s.getServices()})\n\tlog.Println(\"del key:\", key)\n}\n\n//GetServices 获取服务地址\nfunc (s *ServiceDiscovery) getServices() []resolver.Address {\n\taddrs := make([]resolver.Address, 0, 10)\n\ts.serverList.Range(func(k, v interface{}) bool {\n\t\taddrs = append(addrs, v.(resolver.Address))\n\t\treturn true\n\t})\n\treturn addrs\n}\n" }, { "path": "5-etcd-grpclb-balancer/etcdv3/register.go", "content": "package etcdv3\n\nimport (\n\t\"context\"\n\t\"log\"\n\t\"time\"\n\n\t\"go.etcd.io/etcd/clientv3\"\n)\n\n//ServiceRegister 创建租约注册服务\ntype ServiceRegister struct {\n\tcli *clientv3.Client //etcd client\n\tleaseID clientv3.LeaseID //租约ID\n\t//租约keepalieve相应chan\n\tkeepAliveChan <-chan *clientv3.LeaseKeepAliveResponse\n\tkey string //key\n\tweight string //value\n}\n\n//NewServiceRegister 新建注册服务\nfunc NewServiceRegister(endpoints []string, addr, weigit string, lease int64) (*ServiceRegister, error) {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: 5 * time.Second,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tser := &ServiceRegister{\n\t\tcli: cli,\n\t\tkey: \"/\" + schema + \"/\" + addr,\n\t\tweight: weigit,\n\t}\n\n\t//申请租约设置时间keepalive\n\tif err := ser.putKeyWithLease(lease); err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn ser, nil\n}\n\n//设置租约\nfunc (s *ServiceRegister) putKeyWithLease(lease int64) error {\n\t//设置租约时间\n\tresp, err := s.cli.Grant(context.Background(), lease)\n\tif err != nil {\n\t\treturn err\n\t}\n\t//注册服务并绑定租约\n\t_, err = s.cli.Put(context.Background(), s.key, s.weight, clientv3.WithLease(resp.ID))\n\tif err != nil {\n\t\treturn err\n\t}\n\t//设置续租 定期发送需求请求\n\tleaseRespChan, err := s.cli.KeepAlive(context.Background(), resp.ID)\n\n\tif err != nil {\n\t\treturn err\n\t}\n\ts.leaseID = resp.ID\n\ts.keepAliveChan = leaseRespChan\n\tlog.Printf(\"Put key:%s weight:%s success!\", s.key, s.weight)\n\treturn nil\n}\n\n//ListenLeaseRespChan 监听 续租情况\nfunc (s *ServiceRegister) ListenLeaseRespChan() {\n\tfor leaseKeepResp := range s.keepAliveChan {\n\t\tlog.Println(\"续约成功\", leaseKeepResp)\n\t}\n\tlog.Println(\"关闭续租\")\n}\n\n// Close 注销服务\nfunc (s *ServiceRegister) Close() error {\n\t//撤销租约\n\tif _, err := s.cli.Revoke(context.Background(), s.leaseID); err != nil {\n\t\treturn err\n\t}\n\tlog.Println(\"撤销租约\")\n\treturn s.cli.Close()\n}\n" }, { "path": "5-etcd-grpclb-balancer/proto/simple.pb.go", "content": "// Code generated by protoc-gen-go. DO NOT EDIT.\n// source: 2-simple_rpc/proto/simple.proto\n\npackage proto\n\nimport (\n\tcontext \"context\"\n\tfmt \"fmt\"\n\tproto \"github.com/golang/protobuf/proto\"\n\tgrpc \"google.golang.org/grpc\"\n\tcodes \"google.golang.org/grpc/codes\"\n\tstatus \"google.golang.org/grpc/status\"\n\tmath \"math\"\n)\n\n// Reference imports to suppress errors if they are not otherwise used.\nvar _ = proto.Marshal\nvar _ = fmt.Errorf\nvar _ = math.Inf\n\n// This is a compile-time assertion to ensure that this generated file\n// is compatible with the proto package it is being compiled against.\n// A compilation error at this line likely means your copy of the\n// proto package needs to be updated.\nconst _ = proto.ProtoPackageIsVersion3 // please upgrade the proto package\n\n// 定义发送请求信息\ntype SimpleRequest struct {\n\t// 定义发送的参数，采用驼峰命名方式，小写加下划线，如：student_name\n\t// 参数类型 参数名 标识号(不可重复)\n\tData string `protobuf:\"bytes,1,opt,name=data,proto3\" json:\"data,omitempty\"`\n\tXXX_NoUnkeyedLiteral struct{} `json:\"-\"`\n\tXXX_unrecognized []byte `json:\"-\"`\n\tXXX_sizecache int32 `json:\"-\"`\n}\n\nfunc (m *SimpleRequest) Reset() { *m = SimpleRequest{} }\nfunc (m *SimpleRequest) String() string { return proto.CompactTextString(m) }\nfunc (*SimpleRequest) ProtoMessage() {}\nfunc (*SimpleRequest) Descriptor() ([]byte, []int) {\n\treturn fileDescriptor_31047b63fe44dee8, []int{0}\n}\n\nfunc (m *SimpleRequest) XXX_Unmarshal(b []byte) error {\n\treturn xxx_messageInfo_SimpleRequest.Unmarshal(m, b)\n}\nfunc (m *SimpleRequest) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {\n\treturn xxx_messageInfo_SimpleRequest.Marshal(b, m, deterministic)\n}\nfunc (m *SimpleRequest) XXX_Merge(src proto.Message) {\n\txxx_messageInfo_SimpleRequest.Merge(m, src)\n}\nfunc (m *SimpleRequest) XXX_Size() int {\n\treturn xxx_messageInfo_SimpleRequest.Size(m)\n}\nfunc (m *SimpleRequest) XXX_DiscardUnknown() {\n\txxx_messageInfo_SimpleRequest.DiscardUnknown(m)\n}\n\nvar xxx_messageInfo_SimpleRequest proto.InternalMessageInfo\n\nfunc (m *SimpleRequest) GetData() string {\n\tif m != nil {\n\t\treturn m.Data\n\t}\n\treturn \"\"\n}\n\n// 定义响应信息\ntype SimpleResponse struct {\n\t// 定义接收的参数\n\t// 参数类型 参数名 标识号(不可重复)\n\tCode int32 `protobuf:\"varint,1,opt,name=code,proto3\" json:\"code,omitempty\"`\n\tValue string `protobuf:\"bytes,2,opt,name=value,proto3\" json:\"value,omitempty\"`\n\tXXX_NoUnkeyedLiteral struct{} `json:\"-\"`\n\tXXX_unrecognized []byte `json:\"-\"`\n\tXXX_sizecache int32 `json:\"-\"`\n}\n\nfunc (m *SimpleResponse) Reset() { *m = SimpleResponse{} }\nfunc (m *SimpleResponse) String() string { return proto.CompactTextString(m) }\nfunc (*SimpleResponse) ProtoMessage() {}\nfunc (*SimpleResponse) Descriptor() ([]byte, []int) {\n\treturn fileDescriptor_31047b63fe44dee8, []int{1}\n}\n\nfunc (m *SimpleResponse) XXX_Unmarshal(b []byte) error {\n\treturn xxx_messageInfo_SimpleResponse.Unmarshal(m, b)\n}\nfunc (m *SimpleResponse) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {\n\treturn xxx_messageInfo_SimpleResponse.Marshal(b, m, deterministic)\n}\nfunc (m *SimpleResponse) XXX_Merge(src proto.Message) {\n\txxx_messageInfo_SimpleResponse.Merge(m, src)\n}\nfunc (m *SimpleResponse) XXX_Size() int {\n\treturn xxx_messageInfo_SimpleResponse.Size(m)\n}\nfunc (m *SimpleResponse) XXX_DiscardUnknown() {\n\txxx_messageInfo_SimpleResponse.DiscardUnknown(m)\n}\n\nvar xxx_messageInfo_SimpleResponse proto.InternalMessageInfo\n\nfunc (m *SimpleResponse) GetCode() int32 {\n\tif m != nil {\n\t\treturn m.Code\n\t}\n\treturn 0\n}\n\nfunc (m *SimpleResponse) GetValue() string {\n\tif m != nil {\n\t\treturn m.Value\n\t}\n\treturn \"\"\n}\n\nfunc init() {\n\tproto.RegisterType((*SimpleRequest)(nil), \"proto.SimpleRequest\")\n\tproto.RegisterType((*SimpleResponse)(nil), \"proto.SimpleResponse\")\n}\n\nfunc init() { proto.RegisterFile(\"2-simple_rpc/proto/simple.proto\", fileDescriptor_31047b63fe44dee8) }\n\nvar fileDescriptor_31047b63fe44dee8 = []byte{\n\t// 158 bytes of a gzipped FileDescriptorProto\n\t0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xe2, 0x92, 0x37, 0xd2, 0x2d, 0xce,\n\t0xcc, 0x2d, 0xc8, 0x49, 0x8d, 0x2f, 0x2a, 0x48, 0xd6, 0x2f, 0x28, 0xca, 0x2f, 0xc9, 0xd7, 0x87,\n\t0x08, 0xe8, 0x81, 0x39, 0x42, 0xac, 0x60, 0x4a, 0x49, 0x99, 0x8b, 0x37, 0x18, 0x2c, 0x1c, 0x94,\n\t0x5a, 0x58, 0x9a, 0x5a, 0x5c, 0x22, 0x24, 0xc4, 0xc5, 0x92, 0x92, 0x58, 0x92, 0x28, 0xc1, 0xa8,\n\t0xc0, 0xa8, 0xc1, 0x19, 0x04, 0x66, 0x2b, 0x59, 0x71, 0xf1, 0xc1, 0x14, 0x15, 0x17, 0xe4, 0xe7,\n\t0x15, 0xa7, 0x82, 0x54, 0x25, 0xe7, 0xa7, 0xa4, 0x82, 0x55, 0xb1, 0x06, 0x81, 0xd9, 0x42, 0x22,\n\t0x5c, 0xac, 0x65, 0x89, 0x39, 0xa5, 0xa9, 0x12, 0x4c, 0x60, 0xad, 0x10, 0x8e, 0x91, 0x03, 0x17,\n\t0x1b, 0x44, 0xaf, 0x90, 0x19, 0x17, 0x6b, 0x50, 0x7e, 0x69, 0x49, 0xaa, 0x90, 0x08, 0xc4, 0x09,\n\t0x7a, 0x28, 0x16, 0x4b, 0x89, 0xa2, 0x89, 0x42, 0x6c, 0x52, 0x62, 0x48, 0x62, 0x03, 0x8b, 0x1b,\n\t0x03, 0x02, 0x00, 0x00, 0xff, 0xff, 0x50, 0xd7, 0x51, 0x6d, 0xd3, 0x00, 0x00, 0x00,\n}\n\n// Reference imports to suppress errors if they are not otherwise used.\nvar _ context.Context\nvar _ grpc.ClientConn\n\n// This is a compile-time assertion to ensure that this generated file\n// is compatible with the grpc package it is being compiled against.\nconst _ = grpc.SupportPackageIsVersion4\n\n// SimpleClient is the client API for Simple service.\n//\n// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://godoc.org/google.golang.org/grpc#ClientConn.NewStream.\ntype SimpleClient interface {\n\tRoute(ctx context.Context, in *SimpleRequest, opts ...grpc.CallOption) (*SimpleResponse, error)\n}\n\ntype simpleClient struct {\n\tcc *grpc.ClientConn\n}\n\nfunc NewSimpleClient(cc *grpc.ClientConn) SimpleClient {\n\treturn &simpleClient{cc}\n}\n\nfunc (c *simpleClient) Route(ctx context.Context, in *SimpleRequest, opts ...grpc.CallOption) (*SimpleResponse, error) {\n\tout := new(SimpleResponse)\n\terr := c.cc.Invoke(ctx, \"/proto.Simple/Route\", in, out, opts...)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn out, nil\n}\n\n// SimpleServer is the server API for Simple service.\ntype SimpleServer interface {\n\tRoute(context.Context, *SimpleRequest) (*SimpleResponse, error)\n}\n\n// UnimplementedSimpleServer can be embedded to have forward compatible implementations.\ntype UnimplementedSimpleServer struct {\n}\n\nfunc (*UnimplementedSimpleServer) Route(ctx context.Context, req *SimpleRequest) (*SimpleResponse, error) {\n\treturn nil, status.Errorf(codes.Unimplemented, \"method Route not implemented\")\n}\n\nfunc RegisterSimpleServer(s *grpc.Server, srv SimpleServer) {\n\ts.RegisterService(&_Simple_serviceDesc, srv)\n}\n\nfunc _Simple_Route_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {\n\tin := new(SimpleRequest)\n\tif err := dec(in); err != nil {\n\t\treturn nil, err\n\t}\n\tif interceptor == nil {\n\t\treturn srv.(SimpleServer).Route(ctx, in)\n\t}\n\tinfo := &grpc.UnaryServerInfo{\n\t\tServer: srv,\n\t\tFullMethod: \"/proto.Simple/Route\",\n\t}\n\thandler := func(ctx context.Context, req interface{}) (interface{}, error) {\n\t\treturn srv.(SimpleServer).Route(ctx, req.(*SimpleRequest))\n\t}\n\treturn interceptor(ctx, in, info, handler)\n}\n\nvar _Simple_serviceDesc = grpc.ServiceDesc{\n\tServiceName: \"proto.Simple\",\n\tHandlerType: (*SimpleServer)(nil),\n\tMethods: []grpc.MethodDesc{\n\t\t{\n\t\t\tMethodName: \"Route\",\n\t\t\tHandler: _Simple_Route_Handler,\n\t\t},\n\t},\n\tStreams: []grpc.StreamDesc{},\n\tMetadata: \"2-simple_rpc/proto/simple.proto\",\n}\n" }, { "path": "5-etcd-grpclb-balancer/proto/simple.proto", "content": "syntax = \"proto3\";// 协议为proto3\n\npackage proto;\n\n// 定义发送请求信息\nmessage SimpleRequest{\n // 定义发送的参数，采用驼峰命名方式，小写加下划线，如：student_name\n // 参数类型 参数名 标识号(不可重复)\n string data = 1;\n}\n\n// 定义响应信息\nmessage SimpleResponse{\n // 定义接收的参数\n // 参数类型 参数名 标识号(不可重复)\n int32 code = 1;\n string value = 2;\n}\n\n// 定义我们的服务（可定义多个服务,每个服务可定义多个接口）\nservice Simple{\n rpc Route (SimpleRequest) returns (SimpleResponse){};\n}" }, { "path": "5-etcd-grpclb-balancer/server/server.go", "content": "package main\n\nimport (\n\t\"context\"\n\t\"log\"\n\t\"net\"\n\n\t\"google.golang.org/grpc\"\n\n\t\"etcd-example/5-etcd-grpclb-balancer/etcdv3\"\n\tpb \"etcd-example/5-etcd-grpclb-balancer/proto\"\n)\n\n// SimpleService 定义我们的服务\ntype SimpleService struct{}\n\nconst (\n\t// Address 监听地址\n\tAddress string = \"localhost:8000\"\n\t// Network 网络通信协议\n\tNetwork string = \"tcp\"\n\t// SerName 服务名称\n\tSerName string = \"simple_grpc\"\n)\n\n// EtcdEndpoints etcd地址\nvar EtcdEndpoints = []string{\"localhost:2379\"}\n\nfunc main() {\n\t// 监听本地端口\n\tlistener, err := net.Listen(Network, Address)\n\tif err != nil {\n\t\tlog.Fatalf(\"net.Listen err: %v\", err)\n\t}\n\tlog.Println(Address + \" net.Listing...\")\n\t// 新建gRPC服务器实例\n\tgrpcServer := grpc.NewServer()\n\t// 在gRPC服务器注册我们的服务\n\tpb.RegisterSimpleServer(grpcServer, &SimpleService{})\n\t//把服务注册到etcd\n\tser, err := etcdv3.NewServiceRegister(EtcdEndpoints, SerName+\"/\"+Address, \"1\", 5)\n\tif err != nil {\n\t\tlog.Fatalf(\"register service err: %v\", err)\n\t}\n\tdefer ser.Close()\n\t//用服务器 Serve() 方法以及我们的端口信息区实现阻塞等待，直到进程被杀死或者 Stop() 被调用\n\terr = grpcServer.Serve(listener)\n\tif err != nil {\n\t\tlog.Fatalf(\"grpcServer.Serve err: %v\", err)\n\t}\n}\n\n// Route 实现Route方法\nfunc (s *SimpleService) Route(ctx context.Context, req *pb.SimpleRequest) (*pb.SimpleResponse, error) {\n\tlog.Println(\"receive: \" + req.Data)\n\tres := pb.SimpleResponse{\n\t\tCode: 200,\n\t\tValue: \"hello \" + req.Data,\n\t}\n\treturn &res, nil\n}\n" }, { "path": "6-etcd-mutex/README.md", "content": "### etcd分布式锁及事务\n\n### 前言\n\n`分布式锁`是控制分布式系统之间同步访问共享资源的一种方式。在分布式系统中，常常需要协调他们的动作。如果不同的系统或是同一个系统的不同主机之间共享了一个或一组资源，那么访问这些资源的时候，往往需要互斥来防止彼此干扰来保证一致性，在这种情况下，便需要使用到分布式锁。\n\n### etcd分布式锁设计\n\n1. `排他性`：任意时刻，只能有一个机器的一个线程能获取到锁。\n\n通过在etcd中存入key值来实现上锁，删除key实现解锁，参考下面伪代码：\n\n```go\nfunc Lock(key string, cli *clientv3.Client) error {\n //获取key，判断是否存在锁\n\tresp, err := cli.Get(context.Background(), key)\n\tif err != nil {\n\t\treturn err\n\t}\n\t//锁存在，返回上锁失败\n\tif len(resp.Kvs) > 0 {\n\t\treturn errors.New(\"lock fail\")\n\t}\n\t_, err = cli.Put(context.Background(), key, \"lock\")\n\tif err != nil {\n\t\treturn err\n\t}\n\treturn nil\n}\n//删除key，解锁\nfunc UnLock(key string, cli *clientv3.Client) error {\n\t_, err := cli.Delete(context.Background(), key)\n\treturn err\n}\n```\n\n当发现已上锁时，直接返回lock fail。也可以处理成等待解锁，解锁后竞争锁。\n```go\n//等待key删除后再竞争锁\nfunc waitDelete(key string, cli *clientv3.Client) {\n\trch := cli.Watch(context.Background(), key)\n\tfor wresp := range rch {\n\t\tfor _, ev := range wresp.Events {\n\t\t\tswitch ev.Type {\n\t\t\tcase mvccpb.DELETE: //删除\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t}\n}\n```\n\n2. `容错性`：只要分布式锁服务集群节点大部分存活，client就可以进行加锁解锁操作。\n`etcd`基于`Raft`算法，确保集群中数据一致性。\n\n3. `避免死锁`：分布式锁一定能得到释放，即使client在释放之前崩溃。\n上面分布式锁设计有缺陷，假如client获取到锁后程序直接崩了，没有解锁，那其他线程也无法拿到锁，导致死锁出现。\n通过给key设定`leases`来避免死锁，但是`leases`过期时间设多长呢？假如设了30秒，而上锁后的操作比30秒大，会导致以下问题：\n\n* 操作没完成，锁被别人占用了，不安全\n\n* 操作完成后，进行解锁，这时候把别人占用的锁解开了\n\n`解决方案`：给key添加过期时间后，以`Keep leases alive`方式延续`leases`，当client正常持有锁时，锁不会过期；当client程序崩掉后，程序不能执行`Keep leases alive`，从而让锁过期，避免死锁。看以下伪代码：\n\n```go\n//上锁\nfunc Lock(key string, cli *clientv3.Client) error {\n //获取key，判断是否存在锁\n\tresp, err := cli.Get(context.Background(), key)\n\tif err != nil {\n\t\treturn err\n\t}\n\t//锁存在，等待解锁后再竞争锁\n\tif len(resp.Kvs) > 0 {\n\t\twaitDelete(key, cli)\n\t\treturn Lock(key)\n\t}\n //设置key过期时间\n\tresp, err := cli.Grant(context.TODO(), 30)\n\tif err != nil {\n\t\treturn err\n\t}\n\t//设置key并绑定过期时间\n\t_, err = cli.Put(context.Background(), key, \"lock\", clientv3.WithLease(resp.ID))\n\tif err != nil {\n\t\treturn err\n\t}\n\t//延续key的过期时间\n\t_, err = cli.KeepAlive(context.TODO(), resp.ID)\n\tif err != nil {\n\t\treturn err\n\t}\n\treturn nil\n}\n//通过让key值过期来解锁\nfunc UnLock(resp *clientv3.LeaseGrantResponse, cli *clientv3.Client) error {\n\t_, err := cli.Revoke(context.TODO(), resp.ID)\n\treturn err\n}\n```\n\n经过以上步骤，我们初步完成了分布式锁设计。其实官方已经实现了分布式锁，它大致原理和上述有出入，接下来我们看下如何使用官方的分布式锁。\n\n### etcd分布式锁使用\n\n```go\nfunc ExampleMutex_Lock() {\n\tcli, err := clientv3.New(clientv3.Config{Endpoints: endpoints})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close()\n\n\t// create two separate sessions for lock competition\n\ts1, err := concurrency.NewSession(cli)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer s1.Close()\n\tm1 := concurrency.NewMutex(s1, \"/my-lock/\")\n\n\ts2, err := concurrency.NewSession(cli)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer s2.Close()\n\tm2 := concurrency.NewMutex(s2, \"/my-lock/\")\n\n\t// acquire lock for s1\n\tif err := m1.Lock(context.TODO()); err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tfmt.Println(\"acquired lock for s1\")\n\n\tm2Locked := make(chan struct{})\n\tgo func() {\n\t\tdefer close(m2Locked)\n\t\t// wait until s1 is locks /my-lock/\n\t\tif err := m2.Lock(context.TODO()); err != nil {\n\t\t\tlog.Fatal(err)\n\t\t}\n\t}()\n\n\tif err := m1.Unlock(context.TODO()); err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tfmt.Println(\"released lock for s1\")\n\n\t<-m2Locked\n\tfmt.Println(\"acquired lock for s2\")\n\n\t// Output:\n\t// acquired lock for s1\n\t// released lock for s1\n\t// acquired lock for s2\n}\n```\n此代码来源于[官方文档](https://github.com/etcd-io/etcd/blob/master/clientv3/concurrency/example_mutex_test.go)，etcd分布式锁使用起来很方便。\n\n### etcd事务\n\n顺便介绍一下etcd事务，先看这段伪代码：\n\n```go\nTxn(context.TODO()).If(//如果以下判断条件成立\n\tCompare(Value(k1), \"<\", v1),\n\tCompare(Version(k1), \"=\", 2)\n).Then(//则执行Then代码段\n\tOpPut(k2,v2), OpPut(k3,v3)\n).Else(//否则执行Else代码段\n\tOpPut(k4,v4), OpPut(k5,v5)\n).Commit()//最后提交事务\n```\n\n使用例子，代码来自[官方文档](https://github.com/etcd-io/etcd/blob/master/clientv3/example_kv_test.go)：\n\n```go\nfunc ExampleKV_txn() {\n\tcli, err := clientv3.New(clientv3.Config{\n\t\tEndpoints: endpoints,\n\t\tDialTimeout: dialTimeout,\n\t})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close()\n\n\tkvc := clientv3.NewKV(cli)\n\n\t_, err = kvc.Put(context.TODO(), \"key\", \"xyz\")\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tctx, cancel := context.WithTimeout(context.Background(), requestTimeout)\n\t_, err = kvc.Txn(ctx).\n\t\t// txn value comparisons are lexical\n\t\tIf(clientv3.Compare(clientv3.Value(\"key\"), \">\", \"abc\")).\n\t\t// the \"Then\" runs, since \"xyz\" > \"abc\"\n\t\tThen(clientv3.OpPut(\"key\", \"XYZ\")).\n\t\t// the \"Else\" does not run\n\t\tElse(clientv3.OpPut(\"key\", \"ABC\")).\n\t\tCommit()\n\tcancel()\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tgresp, err := kvc.Get(context.TODO(), \"key\")\n\tcancel()\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tfor _, ev := range gresp.Kvs {\n\t\tfmt.Printf(\"%s : %s\\n\", ev.Key, ev.Value)\n\t}\n\t// Output: key : XYZ\n}\n```\n\n### 总结\n\n如果发展到分布式服务阶段，且对数据的可靠性要求很高，选`etcd`实现分布式锁不会错。介于对`ZooKeeper`好感度不强，这里就不介绍`ZooKeeper`分布式锁了。一般的`Redis`分布式锁，可能出现锁丢失的情况（如果你是Java开发者，可以使用Redisson客户端实现分布式锁，据说不会出现锁丢失的情况）。" }, { "path": "6-etcd-mutex/example_mutex_test.go", "content": "// Copyright 2017 The etcd Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage concurrency_test\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\n\t\"github.com/coreos/etcd/clientv3\"\n\t\"github.com/coreos/etcd/clientv3/concurrency\"\n)\n\nfunc ExampleMutex_Lock() {\n\tcli, err := clientv3.New(clientv3.Config{Endpoints: endpoints})\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer cli.Close()\n\n\t// create two separate sessions for lock competition\n\ts1, err := concurrency.NewSession(cli)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer s1.Close()\n\tm1 := concurrency.NewMutex(s1, \"/my-lock/\")\n\n\ts2, err := concurrency.NewSession(cli)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer s2.Close()\n\tm2 := concurrency.NewMutex(s2, \"/my-lock/\")\n\n\t// acquire lock for s1\n\tif err := m1.Lock(context.TODO()); err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tfmt.Println(\"acquired lock for s1\")\n\n\tm2Locked := make(chan struct{})\n\tgo func() {\n\t\tdefer close(m2Locked)\n\t\t// wait until s1 is locks /my-lock/\n\t\tif err := m2.Lock(context.TODO()); err != nil {\n\t\t\tlog.Fatal(err)\n\t\t}\n\t}()\n\n\tif err := m1.Unlock(context.TODO()); err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tfmt.Println(\"released lock for s1\")\n\n\t<-m2Locked\n\tfmt.Println(\"acquired lock for s2\")\n\n\t// Output:\n\t// acquired lock for s1\n\t// released lock for s1\n\t// acquired lock for s2\n}\n" }, { "path": "LICENSE", "content": " Apache License\n Version 2.0, January 2004\n http://www.apache.org/licenses/\n\n TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n 1. Definitions.\n\n \"License\" shall mean the terms and conditions for use, reproduction,\n and distribution as defined by Sections 1 through 9 of this document.\n\n \"Licensor\" shall mean the copyright owner or entity authorized by\n the copyright owner that is granting the License.\n\n \"Legal Entity\" shall mean the union of the acting entity and all\n other entities that control, are controlled by, or are under common\n control with that entity. For the purposes of this definition,\n \"control\" means (i) the power, direct or indirect, to cause the\n direction or management of such entity, whether by contract or\n otherwise, or (ii) ownership of fifty percent (50%) or more of the\n outstanding shares, or (iii) beneficial ownership of such entity.\n\n \"You\" (or \"Your\") shall mean an individual or Legal Entity\n exercising permissions granted by this License.\n\n \"Source\" form shall mean the preferred form for making modifications,\n including but not limited to software source code, documentation\n source, and configuration files.\n\n \"Object\" form shall mean any form resulting from mechanical\n transformation or translation of a Source form, including but\n not limited to compiled object code, generated documentation,\n and conversions to other media types.\n\n \"Work\" shall mean the work of authorship, whether in Source or\n Object form, made available under the License, as indicated by a\n copyright notice that is included in or attached to the work\n (an example is provided in the Appendix below).\n\n \"Derivative Works\" shall mean any work, whether in Source or Object\n form, that is based on (or derived from) the Work and for which the\n editorial revisions, annotations, elaborations, or other modifications\n represent, as a whole, an original work of authorship. For the purposes\n of this License, Derivative Works shall not include works that remain\n separable from, or merely link (or bind by name) to the interfaces of,\n the Work and Derivative Works thereof.\n\n \"Contribution\" shall mean any work of authorship, including\n the original version of the Work and any modifications or additions\n to that Work or Derivative Works thereof, that is intentionally\n submitted to Licensor for inclusion in the Work by the copyright owner\n or by an individual or Legal Entity authorized to submit on behalf of\n the copyright owner. For the purposes of this definition, \"submitted\"\n means any form of electronic, verbal, or written communication sent\n to the Licensor or its representatives, including but not limited to\n communication on electronic mailing lists, source code control systems,\n and issue tracking systems that are managed by, or on behalf of, the\n Licensor for the purpose of discussing and improving the Work, but\n excluding communication that is conspicuously marked or otherwise\n designated in writing by the copyright owner as \"Not a Contribution.\"\n\n \"Contributor\" shall mean Licensor and any individual or Legal Entity\n on behalf of whom a Contribution has been received by Licensor and\n subsequently incorporated within the Work.\n\n 2. Grant of Copyright License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n copyright license to reproduce, prepare Derivative Works of,\n publicly display, publicly perform, sublicense, and distribute the\n Work and such Derivative Works in Source or Object form.\n\n 3. Grant of Patent License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n (except as stated in this section) patent license to make, have made,\n use, offer to sell, sell, import, and otherwise transfer the Work,\n where such license applies only to those patent claims licensable\n by such Contributor that are necessarily infringed by their\n Contribution(s) alone or by combination of their Contribution(s)\n with the Work to which such Contribution(s) was submitted. If You\n institute patent litigation against any entity (including a\n cross-claim or counterclaim in a lawsuit) alleging that the Work\n or a Contribution incorporated within the Work constitutes direct\n or contributory patent infringement, then any patent licenses\n granted to You under this License for that Work shall terminate\n as of the date such litigation is filed.\n\n 4. Redistribution. You may reproduce and distribute copies of the\n Work or Derivative Works thereof in any medium, with or without\n modifications, and in Source or Object form, provided that You\n meet the following conditions:\n\n (a) You must give any other recipients of the Work or\n Derivative Works a copy of this License; and\n\n (b) You must cause any modified files to carry prominent notices\n stating that You changed the files; and\n\n (c) You must retain, in the Source form of any Derivative Works\n that You distribute, all copyright, patent, trademark, and\n attribution notices from the Source form of the Work,\n excluding those notices that do not pertain to any part of\n the Derivative Works; and\n\n (d) If the Work includes a \"NOTICE\" text file as part of its\n distribution, then any Derivative Works that You distribute must\n include a readable copy of the attribution notices contained\n within such NOTICE file, excluding those notices that do not\n pertain to any part of the Derivative Works, in at least one\n of the following places: within a NOTICE text file distributed\n as part of the Derivative Works; within the Source form or\n documentation, if provided along with the Derivative Works; or,\n within a display generated by the Derivative Works, if and\n wherever such third-party notices normally appear. The contents\n of the NOTICE file are for informational purposes only and\n do not modify the License. You may add Your own attribution\n notices within Derivative Works that You distribute, alongside\n or as an addendum to the NOTICE text from the Work, provided\n that such additional attribution notices cannot be construed\n as modifying the License.\n\n You may add Your own copyright statement to Your modifications and\n may provide additional or different license terms and conditions\n for use, reproduction, or distribution of Your modifications, or\n for any such Derivative Works as a whole, provided Your use,\n reproduction, and distribution of the Work otherwise complies with\n the conditions stated in this License.\n\n 5. Submission of Contributions. Unless You explicitly state otherwise,\n any Contribution intentionally submitted for inclusion in the Work\n by You to the Licensor shall be under the terms and conditions of\n this License, without any additional terms or conditions.\n Notwithstanding the above, nothing herein shall supersede or modify\n the terms of any separate license agreement you may have executed\n with Licensor regarding such Contributions.\n\n 6. Trademarks. This License does not grant permission to use the trade\n names, trademarks, service marks, or product names of the Licensor,\n except as required for reasonable and customary use in describing the\n origin of the Work and reproducing the content of the NOTICE file.\n\n 7. Disclaimer of Warranty. Unless required by applicable law or\n agreed to in writing, Licensor provides the Work (and each\n Contributor provides its Contributions) on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n implied, including, without limitation, any warranties or conditions\n of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n PARTICULAR PURPOSE. You are solely responsible for determining the\n appropriateness of using or redistributing the Work and assume any\n risks associated with Your exercise of permissions under this License.\n\n 8. Limitation of Liability. In no event and under no legal theory,\n whether in tort (including negligence), contract, or otherwise,\n unless required by applicable law (such as deliberate and grossly\n negligent acts) or agreed to in writing, shall any Contributor be\n liable to You for damages, including any direct, indirect, special,\n incidental, or consequential damages of any character arising as a\n result of this License or out of the use or inability to use the\n Work (including but not limited to damages for loss of goodwill,\n work stoppage, computer failure or malfunction, or any and all\n other commercial damages or losses), even if such Contributor\n has been advised of the possibility of such damages.\n\n 9. Accepting Warranty or Additional Liability. While redistributing\n the Work or Derivative Works thereof, You may choose to offer,\n and charge a fee for, acceptance of support, warranty, indemnity,\n or other liability obligations and/or rights consistent with this\n License. However, in accepting such obligations, You may act only\n on Your own behalf and on Your sole responsibility, not on behalf\n of any other Contributor, and only if You agree to indemnify,\n defend, and hold each Contributor harmless for any liability\n incurred by, or claims asserted against, such Contributor by reason\n of your accepting any such warranty or additional liability.\n\n END OF TERMS AND CONDITIONS\n\n APPENDIX: How to apply the Apache License to your work.\n\n To apply the Apache License to your work, attach the following\n boilerplate notice, with the fields enclosed by brackets \"[]\"\n replaced with your own identifying information. (Don't include\n the brackets!) The text should be enclosed in the appropriate\n comment syntax for the file format. We also recommend that a\n file or class name and description of purpose be included on the\n same \"printed page\" as the copyright notice for easier\n identification within third-party archives.\n\n Copyright [yyyy] [name of copyright owner]\n\n Licensed under the Apache License, Version 2.0 (the \"License\");\n you may not use this file except in compliance with the License.\n You may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\n Unless required by applicable law or agreed to in writing, software\n distributed under the License is distributed on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n See the License for the specific language governing permissions and\n limitations under the License.\n" }, { "path": "README.md", "content": "### 教程\n\n* [etcd环境安装与使用](https://www.cnblogs.com/FireworksEasyCool/p/12858570.html)\n\n* [etcd实现服务发现](https://www.cnblogs.com/FireworksEasyCool/p/12890649.html)\n\n* [gRPC负载均衡（客户端负载均衡）--基于etcd服务发现](https://www.cnblogs.com/FireworksEasyCool/p/12912839.html)\n\n* [gRPC负载均衡（自定义负载均衡策略）--基于etcd服务发现](https://www.cnblogs.com/FireworksEasyCool/p/12924701.html)\n\n* [etcd分布式锁及事务](https://www.cnblogs.com/FireworksEasyCool/p/12937882.html)" }, { "path": "go.mod", "content": "module etcd-example\n\ngo 1.13\n\nrequire (\n\tgithub.com/coreos/etcd v3.3.20+incompatible\n\tgithub.com/coreos/go-semver v0.3.0 // indirect\n\tgithub.com/coreos/go-systemd v0.0.0-20191104093116-d3cd4ed1dbcf // indirect\n\tgithub.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f // indirect\n\tgithub.com/gogo/protobuf v1.3.1 // indirect\n\tgithub.com/golang/protobuf v1.4.1\n\tgithub.com/google/uuid v1.1.1 // indirect\n\tgithub.com/grpc-ecosystem/go-grpc-prometheus v1.2.0\n\tgithub.com/prometheus/client_golang v1.6.0\n\tgo.etcd.io/etcd v3.3.20+incompatible\n\tgo.uber.org/zap v1.15.0 // indirect\n\tgolang.org/x/net v0.0.0-20200506145744-7e3656a0809f // indirect\n\tgolang.org/x/sys v0.0.0-20200511232937-7e40ca221e25 // indirect\n\tgolang.org/x/text v0.3.2 // indirect\n\tgoogle.golang.org/genproto v0.0.0-20200511104702-f5ebc3bea380 // indirect\n\tgoogle.golang.org/grpc v1.29.1\n)\n\nreplace google.golang.org/grpc => google.golang.org/grpc v1.26.0\n" }, { "path": "go.sum", "content": "cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=\ngithub.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=\ngithub.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=\ngithub.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=\ngithub.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=\ngithub.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=\ngithub.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=\ngithub.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=\ngithub.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=\ngithub.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=\ngithub.com/bingjian-zhu/etcd-example v0.0.0-20200509084147-ecf5c76cccde h1:sa9ttAnwsJ+GgzqpHQSs4Iq7h/BJgrOxQAEJweaw4Tc=\ngithub.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=\ngithub.com/cespare/xxhash/v2 v2.1.1 h1:6MnRN8NT7+YBpUIWxHtefFZOKTAPgGjpQSxqLNn0+qY=\ngithub.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=\ngithub.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=\ngithub.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=\ngithub.com/coreos/etcd v3.3.20+incompatible h1:jIrdkuJDHmyh6VZsxQQ3LQGfOrwgJx6sILz/lxzXsGw=\ngithub.com/coreos/etcd v3.3.20+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=\ngithub.com/coreos/go-semver v0.3.0 h1:wkHLiw0WNATZnSG7epLsujiMCgPAc9xhjJ4tgnAxmfM=\ngithub.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=\ngithub.com/coreos/go-systemd v0.0.0-20191104093116-d3cd4ed1dbcf h1:iW4rZ826su+pqaw19uhpSCzhj44qo35pNgKFGqzDKkU=\ngithub.com/coreos/go-systemd v0.0.0-20191104093116-d3cd4ed1dbcf/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=\ngithub.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f h1:lBNOc5arjvs8E5mO2tbpBpLoyyu8B6e44T7hJy6potg=\ngithub.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA=\ngithub.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=\ngithub.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=\ngithub.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=\ngithub.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=\ngithub.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=\ngithub.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=\ngithub.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=\ngithub.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=\ngithub.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE=\ngithub.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk=\ngithub.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=\ngithub.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=\ngithub.com/gogo/protobuf v1.3.1 h1:DqDEcV5aeaTmdFBePNpYsp3FlcVH/2ISVVM9Qf8PSls=\ngithub.com/gogo/protobuf v1.3.1/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o=\ngithub.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=\ngithub.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=\ngithub.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=\ngithub.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=\ngithub.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=\ngithub.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=\ngithub.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=\ngithub.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=\ngithub.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=\ngithub.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=\ngithub.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=\ngithub.com/golang/protobuf v1.4.1 h1:ZFgWrT+bLgsYPirOnRfKLYJLvssAegOj/hgyMFdJZe0=\ngithub.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=\ngithub.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=\ngithub.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=\ngithub.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=\ngithub.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=\ngithub.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=\ngithub.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=\ngithub.com/google/uuid v1.1.1 h1:Gkbcsh/GbpXz7lPftLA3P6TYMwjCLYm83jiFQZF/3gY=\ngithub.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=\ngithub.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 h1:Ovs26xHkKqVztRpIrF/92BcuyuQ/YW4NSIpoGtfXNho=\ngithub.com/grpc-ecosystem/go-grpc-prometheus v1.2.0/go.mod h1:8NvIoxWQoOIhqOTXgfV/d3M/q6VIi02HzZEHgUlZvzk=\ngithub.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=\ngithub.com/json-iterator/go v1.1.9/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=\ngithub.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=\ngithub.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00=\ngithub.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=\ngithub.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=\ngithub.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=\ngithub.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=\ngithub.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=\ngithub.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=\ngithub.com/matttproud/golang_protobuf_extensions v1.0.1 h1:4hp9jkHxhMHkqkrB3Ix0jegS5sx/RkqARlsWZ6pIwiU=\ngithub.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=\ngithub.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=\ngithub.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=\ngithub.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=\ngithub.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=\ngithub.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=\ngithub.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=\ngithub.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=\ngithub.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=\ngithub.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=\ngithub.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo=\ngithub.com/prometheus/client_golang v1.6.0 h1:YVPodQOcK15POxhgARIvnDRVpLcuK8mglnMrWfyrw6A=\ngithub.com/prometheus/client_golang v1.6.0/go.mod h1:ZLOG9ck3JLRdB5MgO8f+lLTe83AXG6ro35rLTxvnIl4=\ngithub.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=\ngithub.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=\ngithub.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=\ngithub.com/prometheus/client_model v0.2.0 h1:uq5h0d+GuxiXLJLNABMgp2qUWDPiLvgCzz2dUR+/W/M=\ngithub.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=\ngithub.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=\ngithub.com/prometheus/common v0.9.1 h1:KOMtN28tlbam3/7ZKEYKHhKoJZYYj3gMH4uc62x7X7U=\ngithub.com/prometheus/common v0.9.1/go.mod h1:yhUN8i9wzaXS3w1O07YhxHEBxD+W35wd8bs7vj7HSQ4=\ngithub.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=\ngithub.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=\ngithub.com/prometheus/procfs v0.0.11 h1:DhHlBtkHWPYi8O2y31JkK0TF+DGM+51OopZjH/Ia5qI=\ngithub.com/prometheus/procfs v0.0.11/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU=\ngithub.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=\ngithub.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=\ngithub.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=\ngithub.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=\ngithub.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=\ngithub.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=\ngithub.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=\ngithub.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=\ngo.etcd.io/etcd v3.3.20+incompatible h1:EyOVslCepyFB2JcbYXvqcYdBTh7cyBKU2NYdKfgTSC0=\ngo.etcd.io/etcd v3.3.20+incompatible/go.mod h1:yaeTdrJi5lOmYerz05bd8+V7KubZs8YSFZfzsF9A6aI=\ngo.uber.org/atomic v1.6.0 h1:Ezj3JGmsOnG1MoRWQkPBsKLe9DwWD9QeXzTRzzldNVk=\ngo.uber.org/atomic v1.6.0/go.mod h1:sABNBOSYdrvTF6hTgEIbc7YasKWGhgEQZyfxyTvoXHQ=\ngo.uber.org/multierr v1.5.0 h1:KCa4XfM8CWFCpxXRGok+Q0SS/0XBhMDbHHGABQLvD2A=\ngo.uber.org/multierr v1.5.0/go.mod h1:FeouvMocqHpRaaGuG9EjoKcStLC43Zu/fmqdUMPcKYU=\ngo.uber.org/tools v0.0.0-20190618225709-2cfd321de3ee/go.mod h1:vJERXedbb3MVM5f9Ejo0C68/HhF8uaILCdgjnY+goOA=\ngo.uber.org/zap v1.15.0 h1:ZZCA22JRF2gQE5FoNmhmrf7jeJJ2uhqDUNRYKm8dvmM=\ngo.uber.org/zap v1.15.0/go.mod h1:Mb2vm2krFEG5DV0W9qcHBYFtp/Wku1cvYaqPsS/WYfc=\ngolang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=\ngolang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=\ngolang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=\ngolang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=\ngolang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=\ngolang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=\ngolang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=\ngolang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=\ngolang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=\ngolang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=\ngolang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=\ngolang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=\ngolang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20190620200207-3b0461eec859 h1:R/3boaszxrf1GEUWTVDzSKVwLmSJpwZ1yqXm8j0v2QI=\ngolang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=\ngolang.org/x/net v0.0.0-20200506145744-7e3656a0809f h1:QBjCr1Fz5kw158VqdE9JfI9cJnl/ymnJWAdMuinqL7Y=\ngolang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=\ngolang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=\ngolang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=\ngolang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=\ngolang.org/x/sys v0.0.0-20190412213103-97732733099d h1:+R4KGOnez64A81RvjARKc4UT5/tI9ujCIVX+P5KiHuI=\ngolang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200106162015-b016eb3dc98e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200420163511-1957bb5e6d1f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/sys v0.0.0-20200511232937-7e40ca221e25 h1:OKbAoGs4fGM5cPLlVQLZGYkFC8OnOfgo6tt0Smf9XhM=\ngolang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=\ngolang.org/x/text v0.3.0 h1:g61tztE5qeGQ89tm6NTjjM9VPIm088od1l6aSorWRWg=\ngolang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=\ngolang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs=\ngolang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=\ngolang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=\ngolang.org/x/tools v0.0.0-20181030221726-6c7e314b6563/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=\ngolang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=\ngolang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=\ngolang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=\ngolang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=\ngolang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=\ngolang.org/x/tools v0.0.0-20191029041327-9cc4af7d6b2c/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/tools v0.0.0-20191029190741-b9c20aec41a5/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=\ngolang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\ngolang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=\ngoogle.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=\ngoogle.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=\ngoogle.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=\ngoogle.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=\ngoogle.golang.org/genproto v0.0.0-20200511104702-f5ebc3bea380 h1:xriR1EgvKfkKxIoU2uUvrMVl+H26359loFFUleSMXFo=\ngoogle.golang.org/genproto v0.0.0-20200511104702-f5ebc3bea380/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=\ngoogle.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=\ngoogle.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=\ngoogle.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=\ngoogle.golang.org/grpc v1.26.0 h1:2dTRdpdFEEhJYQD8EMLB61nnrzSCTbG38PhqdhvOltg=\ngoogle.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=\ngoogle.golang.org/grpc v1.27.0 h1:rRYRFMVgRv6E0D70Skyfsr28tDXIuuPZyWGMPdMcnXg=\ngoogle.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=\ngoogle.golang.org/grpc v1.29.1 h1:EC2SB8S04d2r73uptxphDSUG+kTKVgjRPF+N3xpxRB4=\ngoogle.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3IjizoKk=\ngoogle.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=\ngoogle.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=\ngoogle.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=\ngoogle.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=\ngoogle.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=\ngoogle.golang.org/protobuf v1.22.0 h1:cJv5/xdbk1NnMPR1VP9+HU6gupuG9MLBoH1r6RHZ2MY=\ngoogle.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=\ngopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=\ngopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=\ngopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=\ngopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=\ngopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=\ngopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\ngopkg.in/yaml.v2 v2.2.5/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=\nhonnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=\nhonnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=\nhonnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=\n" } ]