Showing preview only (379K chars total). Download the full file or copy to clipboard to get everything.
Repository: cloudwego/netpoll
Branch: main
Commit: fcc5e9d814c8
Files: 92
Total size: 356.8 KB
Directory structure:
gitextract_ncyzmgn6/
├── .github/
│ ├── CODEOWNERS
│ ├── ISSUE_TEMPLATE/
│ │ ├── bug_report.md
│ │ └── feature_request.md
│ ├── PULL_REQUEST_TEMPLATE.md
│ └── workflows/
│ └── pr-check.yml
├── .gitignore
├── .golangci.yaml
├── .licenserc.yaml
├── CODE_OF_CONDUCT.md
├── CONTRIBUTING.md
├── CREDITS
├── LICENSE
├── NOTICE
├── README.md
├── README_CN.md
├── _typos.toml
├── connection.go
├── connection_errors.go
├── connection_errors_test.go
├── connection_impl.go
├── connection_lock.go
├── connection_onevent.go
├── connection_reactor.go
├── connection_test.go
├── docs/
│ ├── guide/
│ │ ├── guide_cn.md
│ │ └── guide_en.md
│ └── reference/
│ ├── design_cn.md
│ ├── design_en.md
│ └── explain.md
├── eventloop.go
├── fd_operator.go
├── fd_operator_cache.go
├── fd_operator_cache_test.go
├── go.mod
├── go.sum
├── internal/
│ └── runner/
│ ├── runner.go
│ └── runner_test.go
├── lint.sh
├── mux/
│ ├── mux_test.go
│ ├── shard_queue.go
│ └── shard_queue_test.go
├── net_dialer.go
├── net_dialer_test.go
├── net_io.go
├── net_listener.go
├── net_listener_test.go
├── net_netfd.go
├── net_netfd_conn.go
├── net_polldesc.go
├── net_polldesc_test.go
├── net_sock.go
├── net_tcpsock.go
├── net_unixsock.go
├── netpoll_config.go
├── netpoll_options.go
├── netpoll_server.go
├── netpoll_unix.go
├── netpoll_unix_test.go
├── netpoll_windows.go
├── nocopy.go
├── nocopy_linkbuffer.go
├── nocopy_linkbuffer_norace.go
├── nocopy_linkbuffer_race.go
├── nocopy_linkbuffer_test.go
├── nocopy_readwriter.go
├── nocopy_readwriter_test.go
├── poll.go
├── poll_default.go
├── poll_default_bsd.go
├── poll_default_bsd_norace.go
├── poll_default_bsd_race.go
├── poll_default_linux.go
├── poll_default_linux_norace.go
├── poll_default_linux_race.go
├── poll_default_linux_test.go
├── poll_loadbalance.go
├── poll_manager.go
├── poll_manager_test.go
├── poll_test.go
├── sys_epoll_linux.go
├── sys_epoll_linux_arm64.go
├── sys_epoll_linux_loong64.go
├── sys_exec.go
├── sys_exec_test.go
├── sys_keepalive_darwin.go
├── sys_keepalive_openbsd.go
├── sys_keepalive_unix.go
├── sys_sendmsg_bsd.go
├── sys_sendmsg_linux.go
├── sys_sockopt_bsd.go
├── sys_sockopt_linux.go
└── test_conns.sh
================================================
FILE CONTENTS
================================================
================================================
FILE: .github/CODEOWNERS
================================================
# For more information, please refer to https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners
* @cloudwego/netpoll-reviewers @cloudwego/netpoll-approvers @cloudwego/netpoll-maintainers
================================================
FILE: .github/ISSUE_TEMPLATE/bug_report.md
================================================
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: ''
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Desktop (please complete the following information):**
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
**Smartphone (please complete the following information):**
- Device: [e.g. iPhone6]
- OS: [e.g. iOS8.1]
- Browser [e.g. stock browser, safari]
- Version [e.g. 22]
**Additional context**
Add any other context about the problem here.
================================================
FILE: .github/ISSUE_TEMPLATE/feature_request.md
================================================
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: ''
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.
================================================
FILE: .github/PULL_REQUEST_TEMPLATE.md
================================================
================================================
FILE: .github/workflows/pr-check.yml
================================================
name: Push and Pull Request Check
on: [ push, pull_request ]
jobs:
compatibility-test:
strategy:
matrix:
go: [ 1.18, 1.24 ]
os: [ ubuntu-latest, ubuntu-24.04-arm, macos-latest ]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go }}
- name: Unit Test
run: go test -timeout=2m -race ./...
- name: Benchmark
run: go test -bench=. -benchmem -run=none ./... -benchtime=100ms
windows-test:
runs-on: windows-latest
steps:
- uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: stable
- name: Build Test
run: go vet ./...
compliant:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Check License Header
uses: apache/skywalking-eyes/header@v0.4.0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Check Spell
uses: crate-ci/typos@v1.13.14
golangci-lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: stable
# for self-hosted, the cache path is shared across projects
# and it works well without the cache of github actions
# Enable it if we're going to use Github only
cache: false
- name: Golangci Lint
# https://golangci-lint.run/
uses: golangci/golangci-lint-action@v6
with:
version: latest
only-new-issues: true
================================================
FILE: .gitignore
================================================
# Binaries for programs and plugins
*.exe
*.exe~
*.dll
*.so
*.dylib
# Test binary, built with `go test -c`
*.test
# Output of the go coverage tool, specifically when used with LiteIDE
*.out
# Dependency directories (remove the comment below to include it)
# vendor/
.idea/
================================================
FILE: .golangci.yaml
================================================
# Options for analysis running.
run:
timeout: 3m
linters: # https://golangci-lint.run/usage/linters/
disable-all: true
enable:
- gosimple
- govet
- ineffassign
- staticcheck
- unused
- unconvert
- goimports
- gofumpt
# Refer to https://golangci-lint.run/usage/linters
linters-settings:
gofumpt:
# Choose whether to use the extra rules.
# Default: false
extra-rules: true
goimports:
# Put imports beginning with prefix after 3rd-party packages.
# It's a comma-separated list of prefixes.
local-prefixes: github.com/cloudwego/netpoll
issues:
exclude-use-default: true
================================================
FILE: .licenserc.yaml
================================================
header:
license:
spdx-id: Apache-2.0
copyright-owner: CloudWeGo Authors
paths:
- '**/*.go'
- '**/*.s'
paths-ignore:
- 'net_netfd.go'
- 'net_sock.go'
- 'net_tcpsock.go'
- 'net_unixsock.go'
- 'sys_sockopt_bsd.go'
- 'sys_sockopt_linux.go'
comment: on-failure
================================================
FILE: CODE_OF_CONDUCT.md
================================================
# Contributor Covenant Code of Conduct
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, religion, or sexual identity
and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming,
diverse, inclusive, and healthy community.
## Our Standards
Examples of behavior that contributes to a positive environment for our
community include:
* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
* Focusing on what is best not just for us as individuals, but for the
overall community
Examples of unacceptable behavior include:
* The use of sexualized language or imagery, and sexual attention or
advances of any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email
address, without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Enforcement Responsibilities
Community leaders are responsible for clarifying and enforcing our standards of
acceptable behavior and will take appropriate and fair corrective action in
response to any behavior that they deem inappropriate, threatening, offensive,
or harmful.
Community leaders have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are
not aligned to this Code of Conduct, and will communicate reasons for moderation
decisions when appropriate.
## Scope
This Code of Conduct applies within all community spaces, and also applies when
an individual is officially representing the community in public spaces.
Examples of representing our community include using an official e-mail address,
posting via an official social media account, or acting as an appointed
representative at an online or offline event.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at
conduct@cloudwego.io.
All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the
reporter of any incident.
## Enforcement Guidelines
Community leaders will follow these Community Impact Guidelines in determining
the consequences for any action they deem in violation of this Code of Conduct:
### 1. Correction
**Community Impact**: Use of inappropriate language or other behavior deemed
unprofessional or unwelcome in the community.
**Consequence**: A private, written warning from community leaders, providing
clarity around the nature of the violation and an explanation of why the
behavior was inappropriate. A public apology may be requested.
### 2. Warning
**Community Impact**: A violation through a single incident or series
of actions.
**Consequence**: A warning with consequences for continued behavior. No
interaction with the people involved, including unsolicited interaction with
those enforcing the Code of Conduct, for a specified period of time. This
includes avoiding interactions in community spaces as well as external channels
like social media. Violating these terms may lead to a temporary or
permanent ban.
### 3. Temporary Ban
**Community Impact**: A serious violation of community standards, including
sustained inappropriate behavior.
**Consequence**: A temporary ban from any sort of interaction or public
communication with the community for a specified period of time. No public or
private interaction with the people involved, including unsolicited interaction
with those enforcing the Code of Conduct, is allowed during this period.
Violating these terms may lead to a permanent ban.
### 4. Permanent Ban
**Community Impact**: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior, harassment of an
individual, or aggression toward or disparagement of classes of individuals.
**Consequence**: A permanent ban from any sort of public interaction within
the community.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 2.0, available at
https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
Community Impact Guidelines were inspired by [Mozilla's code of conduct
enforcement ladder](https://github.com/mozilla/diversity).
[homepage]: https://www.contributor-covenant.org
For answers to common questions about this code of conduct, see the FAQ at
https://www.contributor-covenant.org/faq. Translations are available at
https://www.contributor-covenant.org/translations.
================================================
FILE: CONTRIBUTING.md
================================================
# How to Contribute
## Your First Pull Request
We use github for our codebase. You can start by reading [How To Pull Request](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/about-pull-requests).
## Without Semantic Versioning
We keep the stable code in branch `main` like `golang.org/x`. Development base on branch `develop`. And we promise the **Forward Compatibility** by adding new package directory with suffix `v2/v3` when code has break changes.
## Branch Organization
We use [git-flow](https://nvie.com/posts/a-successful-git-branching-model/) as our branch organization, as known as [FDD](https://en.wikipedia.org/wiki/Feature-driven_development)
## Bugs
### 1. How to Find Known Issues
We are using [Github Issues](https://github.com/cloudwego/netpoll/issues) for our public bugs. We keep a close eye on this and try to make it clear when we have an internal fix in progress. Before filing a new task, try to make sure your problem doesn’t already exist.
### 2. Reporting New Issues
Providing a reduced test code is a recommended way for reporting issues. Then can be placed in:
- Just in issues
- [Golang Playground](https://play.golang.org/)
### 3. Security Bugs
Please do not report the safe disclosure of bugs to public issues. Contact us by [Support Email](mailto:conduct@cloudwego.io)
## How to Get in Touch
- [Email](mailto:conduct@cloudwego.io)
## Submit a Pull Request
Before you submit your Pull Request (PR) consider the following guidelines:
1. Search [GitHub](https://github.com/cloudwego/netpoll/pulls) for an open or closed PR that relates to your submission. You don't want to duplicate existing efforts.
2. Be sure that an issue describes the problem you're fixing, or documents the design for the feature you'd like to add. Discussing the design upfront helps to ensure that we're ready to accept your work.
3. [Fork](https://docs.github.com/en/github/getting-started-with-github/fork-a-repo) the cloudwego/netpoll repo.
4. In your forked repository, make your changes in a new git branch:
```
git checkout -b my-fix-branch main
```
5. Create your patch, including appropriate test cases.
6. Follow our [Style Guides](#code-style-guides).
7. Commit your changes using a descriptive commit message that follows [AngularJS Git Commit Message Conventions](https://docs.google.com/document/d/1QrDFcIiPjSLDn3EL15IJygNPiHORgU1_OOAqWjiDU5Y/edit).
Adherence to these conventions is necessary because release notes are automatically generated from these messages.
8. Push your branch to GitHub:
```
git push origin my-fix-branch
```
9. In GitHub, send a pull request to `netpoll:main`
## Contribution Prerequisites
- Our development environment keeps up with [Go Official](https://golang.org/project/).
- You need to fully check with lint tools before submitting your pull request. [gofmt](https://golang.org/pkg/cmd/gofmt/) and [golangci-lint](https://github.com/golangci/golangci-lint)
- You are familiar with [Github](https://github.com)
- Maybe you need to be familiar with [Actions](https://github.com/features/actions)(our default workflow tool).
## Code Style Guides
- [Effective Go](https://golang.org/doc/effective_go)
- [Go Code Review Comments](https://github.com/golang/go/wiki/CodeReviewComments)
================================================
FILE: CREDITS
================================================
================================================
FILE: LICENSE
================================================
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
================================================
FILE: NOTICE
================================================
CloudWeGO
Copyright 2022 CloudWeGO authors.
Go
Copyright (c) 2009 The Go Authors.
================================================
FILE: README.md
================================================
# CloudWeGo-Netpoll
[中文](README_CN.md)
[](https://github.com/cloudwego/netpoll/releases)
[](https://www.cloudwego.io/)
[](https://github.com/cloudwego/netpoll/blob/main/LICENSE)
[](https://goreportcard.com/report/github.com/cloudwego/netpoll)
[](https://github.com/cloudwego/netpoll/issues)
[](https://github.com/cloudwego/netpoll/issues?q=is%3Aissue+is%3Aclosed)
## Introduction
[Netpoll][Netpoll] is a high-performance non-blocking I/O networking framework, which
focused on RPC scenarios, developed by [ByteDance][ByteDance].
RPC is usually heavy on processing logic and therefore cannot handle I/O serially. But Go's standard
library [net][net] is designed for blocking I/O APIs, so that the RPC framework can
only follow the One Conn One Goroutine design. It will waste a lot of cost for context switching, due to a large number
of goroutines under high concurrency. Besides, [net.Conn][net.Conn] has
no API to check Alive, so it is difficult to make an efficient connection pool for RPC framework, because there may be a
large number of failed connections in the pool.
On the other hand, the open source community currently lacks Go network libraries that focus on RPC scenarios. Similar
repositories such as: [evio][evio], [gnet][gnet], etc., are all
focus on scenarios like [Redis][Redis], [HAProxy][HAProxy].
But now, [Netpoll][Netpoll] was born and solved the above problems. It draws inspiration
from the design of [evio][evio] and [netty][netty], has
excellent [Performance](#performance), and is more suitable for microservice architecture.
Also [Netpoll][Netpoll] provides a number of [Features](#features), and it is recommended
to replace [net][net] in some RPC scenarios.
We developed the RPC framework [Kitex][Kitex] and HTTP framework [Hertz][Hertz]
based on [Netpoll][Netpoll], both with industry-leading performance.
[Examples][netpoll-examples] show how to build RPC client and server
using [Netpoll][Netpoll].
For more information, please refer to [Document](#document).
## Features
* **Already**
- [LinkBuffer][LinkBuffer] provides nocopy API for streaming reading and writing
- [gopool][gopool] provides high-performance goroutine pool
- [mcache][mcache] provides efficient memory reuse
- `IsActive` supports checking whether the connection is alive
- `Dialer` supports building clients
- `EventLoop` supports building a server
- TCP, Unix Domain Socket
- Linux, macOS (operating system)
* **Unsupported**
- Windows (operating system)
## Performance
Benchmark should meet the requirements of industrial use.
In the RPC scenario, concurrency and timeout are necessary support items.
We provide the [netpoll-benchmark][netpoll-benchmark] project to track and compare
the performance of [Netpoll][Netpoll] and other frameworks under different conditions for reference.
More benchmarks reference [kitex-benchmark][kitex-benchmark] and [hertz-benchmark][hertz-benchmark].
## Reference
* [Official Website](https://www.cloudwego.io)
* [Getting Started](docs/guide/guide_en.md)
* [Design](docs/reference/design_en.md)
* [Why DATA RACE](docs/reference/explain.md)
[Netpoll]: https://github.com/cloudwego/netpoll
[net]: https://github.com/golang/go/tree/master/src/net
[net.Conn]: https://github.com/golang/go/blob/master/src/net/net.go
[evio]: https://github.com/tidwall/evio
[gnet]: https://github.com/panjf2000/gnet
[netty]: https://github.com/netty/netty
[Kitex]: https://github.com/cloudwego/kitex
[Hertz]: https://github.com/cloudwego/hertz
[netpoll-benchmark]: https://github.com/cloudwego/netpoll-benchmark
[kitex-benchmark]: https://github.com/cloudwego/kitex-benchmark
[hertz-benchmark]: https://github.com/cloudwego/hertz-benchmark
[netpoll-examples]:https://github.com/cloudwego/netpoll-examples
[ByteDance]: https://www.bytedance.com
[Redis]: https://redis.io
[HAProxy]: http://www.haproxy.org
[LinkBuffer]: nocopy_linkbuffer.go
[gopool]: https://github.com/bytedance/gopkg/tree/develop/util/gopool
[mcache]: https://github.com/bytedance/gopkg/tree/develop/lang/mcache
================================================
FILE: README_CN.md
================================================
# CloudWeGo-Netpoll
[English](README.md)
[](https://github.com/cloudwego/netpoll/releases)
[](https://www.cloudwego.io/)
[](https://github.com/cloudwego/netpoll/blob/main/LICENSE)
[](https://goreportcard.com/report/github.com/cloudwego/netpoll)
[](https://github.com/cloudwego/netpoll/issues)
[](https://github.com/cloudwego/netpoll/issues?q=is%3Aissue+is%3Aclosed)
## 简介
[Netpoll][Netpoll] 是由 [字节跳动][ByteDance] 开发的高性能 NIO(Non-blocking I/O)
网络库，专注于 RPC 场景。
RPC 通常有较重的处理逻辑，因此无法串行处理 I/O。而 Go 的标准库 [net][net] 设计了 BIO(Blocking I/O) 模式的
API，使得 RPC 框架设计上只能为每个连接都分配一个 goroutine。 这在高并发下，会产生大量的
goroutine，大幅增加调度开销。此外，[net.Conn][net.Conn] 没有提供检查连接活性的 API，因此 RPC
框架很难设计出高效的连接池，池中的失效连接无法及时清理。
另一方面，开源社区目前缺少专注于 RPC 方案的 Go 网络库。类似的项目如：[evio][evio]
, [gnet][gnet] 等，均面向 [Redis][Redis], [HAProxy][HAProxy] 这样的场景。
因此 [Netpoll][Netpoll] 应运而生，它借鉴了 [evio][evio]
和 [netty][netty] 的优秀设计，具有出色的 [性能](#性能)，更适用于微服务架构。
同时，[Netpoll][Netpoll] 还提供了一些 [特性](#特性)，推荐在 RPC 设计中替代
[net][net] 。
基于 [Netpoll][Netpoll] 开发的 RPC 框架 [Kitex][Kitex] 和 HTTP 框架 [Hertz][Hertz]，性能均业界领先。
[范例][netpoll-examples] 展示了如何使用 [Netpoll][Netpoll]
构建 RPC Client 和 Server。
更多信息请参阅 [文档](#文档)。
## 特性
* **已经支持**
- [LinkBuffer][LinkBuffer] 提供可以流式读写的 nocopy API
- [gopool][gopool] 提供高性能的 goroutine 池
- [mcache][mcache] 提供高效的内存复用
- `IsActive` 支持检查连接是否存活
- `Dialer` 支持构建 client
- `EventLoop` 支持构建 server
- 支持 TCP，Unix Domain Socket
- 支持 Linux，macOS（操作系统）
* **不被支持**
- Windows（操作系统）
## 性能
性能测试应满足工业级使用要求，在 RPC 场景下，并发请求、等待超时是必要的支持项。
我们提供了 [netpoll-benchmark][netpoll-benchmark] 项目用来长期追踪和比较 [Netpoll][Netpoll] 与其他框架在不同情况下的性能数据以供参考。
更多测试参考 [kitex-benchmark][kitex-benchmark] 和 [hertz-benchmark][hertz-benchmark]
## 参考
* [官方网站](https://www.cloudwego.io)
* [使用文档](docs/guide/guide_cn.md)
* [设计文档](docs/reference/design_cn.md)
* [DATA RACE 说明](docs/reference/explain.md)
[Netpoll]: https://github.com/cloudwego/netpoll
[net]: https://github.com/golang/go/tree/master/src/net
[net.Conn]: https://github.com/golang/go/blob/master/src/net/net.go
[evio]: https://github.com/tidwall/evio
[gnet]: https://github.com/panjf2000/gnet
[netty]: https://github.com/netty/netty
[Kitex]: https://github.com/cloudwego/kitex
[Hertz]: https://github.com/cloudwego/hertz
[netpoll-benchmark]: https://github.com/cloudwego/netpoll-benchmark
[kitex-benchmark]: https://github.com/cloudwego/kitex-benchmark
[hertz-benchmark]: https://github.com/cloudwego/hertz-benchmark
[netpoll-examples]:https://github.com/cloudwego/netpoll-examples
[ByteDance]: https://www.bytedance.com
[Redis]: https://redis.io
[HAProxy]: http://www.haproxy.org
[LinkBuffer]: nocopy_linkbuffer.go
[gopool]: https://github.com/bytedance/gopkg/tree/develop/util/gopool
[mcache]: https://github.com/bytedance/gopkg/tree/develop/lang/mcache
================================================
FILE: _typos.toml
================================================
# Typo check: https://github.com/crate-ci/typos
[files]
extend-exclude = ["go.mod", "go.sum"]
[default.extend-identifiers]
# *sigh* this just isn't worth the cost of fixing
nd = "nd"
write_datas = "write_datas"
================================================
FILE: connection.go
================================================
// Copyright 2022 CloudWeGo Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package netpoll
import (
"net"
"time"
)
// CloseCallback will be called after the connection is closed.
// Return: error is unused which will be ignored directly.
type CloseCallback func(connection Connection) error
// Connection supports reading and writing simultaneously,
// but does not support simultaneous reading or writing by multiple goroutines.
// It maintains its own input/output buffer, and provides nocopy API for reading and writing.
type Connection interface {
// Connection extends net.Conn, just for interface compatibility.
// It's not recommended to use net.Conn API except for io.Closer.
net.Conn
// The recommended API for nocopy reading and writing.
// Reader will return nocopy buffer data, or error after timeout which set by SetReadTimeout.
Reader() Reader
// Writer will write data to the connection by NIO mode,
// so it will return an error only when the connection isn't Active.
Writer() Writer
// IsActive checks whether the connection is active or not.
IsActive() bool
// SetReadTimeout sets the timeout for future Read calls wait.
// A zero value for timeout means Reader will not timeout.
SetReadTimeout(timeout time.Duration) error
// SetWriteTimeout sets the timeout for future Write calls wait.
// A zero value for timeout means Writer will not timeout.
SetWriteTimeout(timeout time.Duration) error
// SetIdleTimeout sets the idle timeout of connections by enabling TCP KeepAlive
// and setting the KeepAlive interval to the given timeout duration.
// NOTE: Despite its name, this does not track application-level idle time.
// It configures OS-level TCP KeepAlive to detect dead peers on idle connections.
// The name is kept for backward compatibility.
SetIdleTimeout(timeout time.Duration) error
// SetOnRequest can set or replace the OnRequest method for a connection, but can't be set to nil.
// Although SetOnRequest avoids data race, it should still be used before transmitting data.
// Replacing OnRequest while processing data may cause unexpected behavior and results.
// Generally, the server side should uniformly set the OnRequest method for each connection via NewEventLoop,
// which is set when the connection is initialized.
// On the client side, if necessary, make sure that OnRequest is set before sending data.
SetOnRequest(on OnRequest) error
// AddCloseCallback can add hangup callback for a connection, which will be called when connection closing.
// This is very useful for cleaning up idle connections. For instance, you can use callbacks to clean up
// the local resources, which bound to the idle connection, when hangup by the peer. No need another goroutine
// to polling check connection status.
AddCloseCallback(callback CloseCallback) error
}
// Conn extends net.Conn, but supports getting the conn's fd.
type Conn interface {
net.Conn
// Fd return conn's fd, used by poll
Fd() (fd int)
}
// Listener extends net.Listener, but supports getting the listener's fd.
type Listener interface {
net.Listener
// Fd return listener's fd, used by poll.
Fd() (fd int)
}
// Dialer extends net.Dialer's API, just for interface compatibility.
// DialConnection is recommended, but of course all functions are practically the same.
// The returned net.Conn can be directly asserted as Connection if error is nil.
type Dialer interface {
DialConnection(network, address string, timeout time.Duration) (connection Connection, err error)
DialTimeout(network, address string, timeout time.Duration) (conn net.Conn, err error)
}
================================================
FILE: connection_errors.go
================================================
// Copyright 2022 CloudWeGo Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package netpoll
import (
"fmt"
"net"
"syscall"
)
// extends syscall.Errno, the range is set to 0x100-0x1FF
const (
// The connection closed when in use.
ErrConnClosed = syscall.Errno(0x101)
// Read I/O buffer timeout, called by Connection.Reader
ErrReadTimeout = syscall.Errno(0x102)
// Dial timeout
ErrDialTimeout = syscall.Errno(0x103)
// Calling dialer without timeout.
ErrDialNoDeadline = syscall.Errno(0x104) // TODO: no-deadline support in future
// The calling function not support.
ErrUnsupported = syscall.Errno(0x105)
// Same as io.EOF
ErrEOF = syscall.Errno(0x106)
// Write I/O buffer timeout, calling by Connection.Writer
ErrWriteTimeout = syscall.Errno(0x107)
// Concurrent connection access error
ErrConcurrentAccess = syscall.Errno(0x108)
)
const ErrnoMask = 0xFF
// wrap Errno, implement xerrors.Wrapper
func Exception(err error, suffix string) error {
no, ok := err.(syscall.Errno)
if !ok {
if suffix == "" {
return err
}
return fmt.Errorf("%s %s", err.Error(), suffix)
}
return &exception{no: no, suffix: suffix}
}
var _ net.Error = (*exception)(nil)
type exception struct {
no syscall.Errno
suffix string
}
func (e *exception) Error() string {
var s string
if int(e.no)&0x100 != 0 {
s = errnos[int(e.no)&ErrnoMask]
}
if s == "" {
s = e.no.Error()
}
if e.suffix != "" {
s += " " + e.suffix
}
return s
}
func (e *exception) Is(target error) bool {
if e == target {
return true
}
if e.no == target {
return true
}
// TODO: ErrConnClosed contains ErrEOF
if e.no == ErrEOF && target == ErrConnClosed {
return true
}
return e.no.Is(target)
}
func (e *exception) Unwrap() error {
return e.no
}
func (e *exception) Timeout() bool {
switch e.no {
case ErrDialTimeout, ErrReadTimeout, ErrWriteTimeout:
return true
}
return e.no.Timeout()
}
func (e *exception) Temporary() bool {
return e.no.Temporary()
}
// Errors defined in netpoll
var errnos = [...]string{
ErrnoMask & ErrConnClosed: "connection has been closed",
ErrnoMask & ErrReadTimeout: "connection read timeout",
ErrnoMask & ErrDialTimeout: "dial wait timeout",
ErrnoMask & ErrDialNoDeadline: "dial no deadline",
ErrnoMask & ErrUnsupported: "netpoll does not support",
ErrnoMask & ErrEOF: "EOF",
ErrnoMask & ErrWriteTimeout: "connection write timeout",
ErrnoMask & ErrConcurrentAccess: "concurrent connection access",
}
================================================
FILE: connection_errors_test.go
================================================
// Copyright 2022 CloudWeGo Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//go:build !windows
// +build !windows
package netpoll
import (
"errors"
"syscall"
"testing"
)
func TestErrno(t *testing.T) {
var err1 error = Exception(ErrConnClosed, "when next")
MustTrue(t, errors.Is(err1, ErrConnClosed))
Equal(t, err1.Error(), "connection has been closed when next")
t.Logf("error1=%s", err1)
var err2 error = Exception(syscall.EPIPE, "when flush")
MustTrue(t, errors.Is(err2, syscall.EPIPE))
Equal(t, err2.Error(), "broken pipe when flush")
t.Logf("error2=%s", err2)
}
================================================
FILE: connection_impl.go
================================================
// Copyright 2022 CloudWeGo Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//go:build !windows
// +build !windows
package netpoll
import (
"sync"
"sync/atomic"
"syscall"
"time"
)
type connState = int32
const (
connStateNone = 0
connStateConnected = 1
connStateDisconnected = 2
)
// connection is the implementation of Connection
type connection struct {
netFD
onEvent
locker
operator *FDOperator
readTimeout time.Duration
readDeadline int64 // UnixNano(). it overwrites readTimeout. 0 if not set.
readTimer *time.Timer
readTrigger chan error
waitReadSize int64
writeTimeout time.Duration
writeDeadline int64 // UnixNano(). it overwrites writeTimeout. 0 if not set.
writeTimer *time.Timer
writeTrigger chan error
inputBuffer *LinkBuffer
outputBuffer *LinkBuffer
outputBarrier *barrier
maxSize int // The maximum size of data between two Release().
bookSize int // The size of data that can be read at once.
state connState // Connection state should be changed sequentially.
}
var (
_ Connection = &connection{}
_ Reader = &connection{}
_ Writer = &connection{}
)
// Reader implements Connection.
func (c *connection) Reader() Reader {
return c
}
// Writer implements Connection.
func (c *connection) Writer() Writer {
return c
}
// IsActive implements Connection.
func (c *connection) IsActive() bool {
return c.isCloseBy(none)
}
// SetIdleTimeout implements Connection.
func (c *connection) SetIdleTimeout(timeout time.Duration) error {
if timeout > 0 {
return c.SetKeepAlive(int(timeout.Seconds()))
}
return nil
}
// SetReadTimeout implements Connection.
func (c *connection) SetReadTimeout(timeout time.Duration) error {
if timeout >= 0 {
c.readTimeout = timeout
}
c.readDeadline = 0
return nil
}
// SetWriteTimeout implements Connection.
func (c *connection) SetWriteTimeout(timeout time.Duration) error {
if timeout >= 0 {
c.writeTimeout = timeout
}
c.writeDeadline = 0
return nil
}
// SetDeadline implements net.Conn.SetDeadline
func (c *connection) SetDeadline(t time.Time) error {
v := int64(0)
if !t.IsZero() {
v = t.UnixNano()
}
c.readDeadline = v
c.writeDeadline = v
return nil
}
// SetReadDeadline implements net.Conn.SetReadDeadline
func (c *connection) SetReadDeadline(t time.Time) error {
if t.IsZero() {
c.readDeadline = 0
} else {
c.readDeadline = t.UnixNano()
}
return nil
}
// SetWriteDeadline implements net.Conn.SetWriteDeadline
func (c *connection) SetWriteDeadline(t time.Time) error {
if t.IsZero() {
c.writeDeadline = 0
} else {
c.writeDeadline = t.UnixNano()
}
return nil
}
// ------------------------------------------ implement zero-copy reader ------------------------------------------
// Next implements Connection.
func (c *connection) Next(n int) (p []byte, err error) {
if err = c.waitRead(n); err != nil {
return p, err
}
return c.inputBuffer.Next(n)
}
// Peek implements Connection.
func (c *connection) Peek(n int) (buf []byte, err error) {
if err = c.waitRead(n); err != nil {
return buf, err
}
return c.inputBuffer.Peek(n)
}
// Skip implements Connection.
func (c *connection) Skip(n int) (err error) {
if err = c.waitRead(n); err != nil {
return err
}
return c.inputBuffer.Skip(n)
}
// Release implements Connection.
func (c *connection) Release() (err error) {
// Check inputBuffer length first to reduce contention in mux situation.
// c.operator.do competes with c.inputs/c.inputAck
if c.inputBuffer.Len() == 0 && c.operator.do() {
maxSize := c.inputBuffer.calcMaxSize()
// Set the maximum value of maxsize equal to mallocMax to prevent GC pressure.
if maxSize > mallocMax {
maxSize = mallocMax
}
if maxSize > c.maxSize {
c.maxSize = maxSize
}
// Double check length to reset tail node
if c.inputBuffer.Len() == 0 {
c.inputBuffer.resetTail(c.maxSize)
}
c.operator.done()
}
return c.inputBuffer.Release()
}
// Slice implements Connection.
func (c *connection) Slice(n int) (r Reader, err error) {
if err = c.waitRead(n); err != nil {
return nil, err
}
return c.inputBuffer.Slice(n)
}
// Len implements Connection.
func (c *connection) Len() (length int) {
return c.inputBuffer.Len()
}
// Until implements Connection.
func (c *connection) Until(delim byte) (line []byte, err error) {
var n, l int
for {
if err = c.waitRead(n + 1); err != nil {
// return all the data in the buffer
line, _ = c.inputBuffer.Next(c.inputBuffer.Len())
return
}
l = c.inputBuffer.Len()
i := c.inputBuffer.indexByte(delim, n)
if i < 0 {
n = l // skip all exists bytes
continue
}
return c.Next(i + 1)
}
}
// ReadString implements Connection.
func (c *connection) ReadString(n int) (s string, err error) {
if err = c.waitRead(n); err != nil {
return s, err
}
return c.inputBuffer.ReadString(n)
}
// ReadBinary implements Connection.
func (c *connection) ReadBinary(n int) (p []byte, err error) {
if err = c.waitRead(n); err != nil {
return p, err
}
return c.inputBuffer.ReadBinary(n)
}
// ReadByte implements Connection.
func (c *connection) ReadByte() (b byte, err error) {
if err = c.waitRead(1); err != nil {
return b, err
}
return c.inputBuffer.ReadByte()
}
// ------------------------------------------ implement zero-copy writer ------------------------------------------
// Malloc implements Connection.
func (c *connection) Malloc(n int) (buf []byte, err error) {
if !c.IsActive() {
return nil, Exception(ErrConnClosed, "when malloc")
}
return c.outputBuffer.Malloc(n)
}
// MallocLen implements Connection.
func (c *connection) MallocLen() (length int) {
return c.outputBuffer.MallocLen()
}
// Flush will send all malloc data to the peer,
// so must confirm that the allocated bytes have been correctly assigned.
//
// Flush first checks whether the out buffer is empty.
// If empty, it will call syscall.Write to send data directly,
// otherwise the buffer will be sent asynchronously by the epoll trigger.
func (c *connection) Flush() error {
if !c.IsActive() {
return Exception(ErrConnClosed, "when flush")
}
if !c.lock(flushing) {
return Exception(ErrConcurrentAccess, "when flush")
}
defer c.unlock(flushing)
c.outputBuffer.Flush()
return c.flush()
}
// MallocAck implements Connection.
func (c *connection) MallocAck(n int) (err error) {
if !c.IsActive() {
return Exception(ErrConnClosed, "when malloc ack")
}
return c.outputBuffer.MallocAck(n)
}
// Append implements Connection.
func (c *connection) Append(w Writer) (err error) {
if !c.IsActive() {
return Exception(ErrConnClosed, "when append")
}
return c.outputBuffer.Append(w)
}
// WriteString implements Connection.
func (c *connection) WriteString(s string) (n int, err error) {
if !c.IsActive() {
return 0, Exception(ErrConnClosed, "when write string")
}
return c.outputBuffer.WriteString(s)
}
// WriteBinary implements Connection.
func (c *connection) WriteBinary(b []byte) (n int, err error) {
if !c.IsActive() {
return 0, Exception(ErrConnClosed, "when write binary")
}
return c.outputBuffer.WriteBinary(b)
}
// WriteDirect implements Connection.
func (c *connection) WriteDirect(p []byte, remainCap int) (err error) {
if !c.IsActive() {
return Exception(ErrConnClosed, "when write direct")
}
return c.outputBuffer.WriteDirect(p, remainCap)
}
// WriteByte implements Connection.
func (c *connection) WriteByte(b byte) (err error) {
if !c.IsActive() {
return Exception(ErrConnClosed, "when write byte")
}
return c.outputBuffer.WriteByte(b)
}
// ------------------------------------------ implement net.Conn ------------------------------------------
// Read behavior is the same as net.Conn, it will return io.EOF if buffer is empty.
func (c *connection) Read(p []byte) (n int, err error) {
if len(p) == 0 {
return 0, nil
}
if err = c.waitRead(1); err != nil {
return 0, err
}
return c.inputBuffer.readCopy(p), nil
}
// Write will Flush soon.
func (c *connection) Write(p []byte) (n int, err error) {
if !c.IsActive() {
return 0, Exception(ErrConnClosed, "when write")
}
if !c.lock(flushing) {
return 0, Exception(ErrConcurrentAccess, "when write")
}
defer c.unlock(flushing)
dst, _ := c.outputBuffer.Malloc(len(p))
n = copy(dst, p)
c.outputBuffer.Flush()
err = c.flush()
return n, err
}
// Close implements Connection.
func (c *connection) Close() error {
return c.onClose()
}
// Detach detaches the connection from poller but doesn't close it.
func (c *connection) Detach() error {
c.detaching = true
return c.onClose()
}
// ------------------------------------------ private ------------------------------------------
var barrierPool = sync.Pool{
New: func() interface{} {
return &barrier{
bs: make([][]byte, barriercap),
ivs: make([]syscall.Iovec, barriercap),
}
},
}
// init initializes the connection with options
func (c *connection) init(conn Conn, opts *options) (err error) {
// init buffer, barrier, finalizer
c.readTrigger = make(chan error, 1)
c.writeTrigger = make(chan error, 1)
c.bookSize, c.maxSize = defaultLinkBufferSize, defaultLinkBufferSize
c.inputBuffer, c.outputBuffer = NewLinkBuffer(defaultLinkBufferSize), NewLinkBuffer()
c.outputBarrier = barrierPool.Get().(*barrier)
c.state = connStateNone
c.initNetFD(conn) // conn must be *netFD{}
c.initFDOperator()
c.initFinalizer()
syscall.SetNonblock(c.fd, true)
// enable TCP_NODELAY by default
switch c.network {
case "tcp", "tcp4", "tcp6":
setTCPNoDelay(c.fd, true)
}
// connection initialized and prepare options
return c.onPrepare(opts)
}
func (c *connection) initNetFD(conn Conn) {
if nfd, ok := conn.(*netFD); ok {
c.netFD = *nfd
return
}
c.netFD = netFD{
fd: conn.Fd(),
localAddr: conn.LocalAddr(),
remoteAddr: conn.RemoteAddr(),
}
}
func (c *connection) initFDOperator() {
poll := pollmanager.Pick()
op := poll.Alloc()
op.FD = c.fd
op.OnRead, op.OnWrite, op.OnHup = nil, nil, c.onHup
op.Inputs, op.InputAck = c.inputs, c.inputAck
op.Outputs, op.OutputAck = c.outputs, c.outputAck
c.operator = op
}
func (c *connection) initFinalizer() {
c.AddCloseCallback(func(connection Connection) (err error) {
c.stop(flushing)
c.operator.Free()
if err = c.netFD.Close(); err != nil {
logger.Printf("NETPOLL: netFD close failed: %v", err)
}
c.closeBuffer()
return nil
})
}
func (c *connection) triggerRead(err error) {
select {
case c.readTrigger <- err:
default:
}
}
func (c *connection) triggerWrite(err error) {
select {
case c.writeTrigger <- err:
default:
}
}
// waitRead will wait full n bytes.
func (c *connection) waitRead(n int) (err error) {
if n <= c.inputBuffer.Len() {
return nil
}
atomic.StoreInt64(&c.waitReadSize, int64(n))
defer atomic.StoreInt64(&c.waitReadSize, 0)
if dl := c.readDeadline; dl > 0 {
timeout := time.Duration(dl - time.Now().UnixNano())
if timeout <= 0 {
return Exception(ErrReadTimeout, c.remoteAddr.String())
}
return c.waitReadWithTimeout(n, timeout)
} else if c.readTimeout > 0 {
return c.waitReadWithTimeout(n, c.readTimeout)
}
// wait full n
for c.inputBuffer.Len() < n {
switch c.status(closing) {
case poller:
return Exception(ErrEOF, "wait read")
case user:
return Exception(ErrConnClosed, "wait read")
default:
err = <-c.readTrigger
if err != nil {
return err
}
}
}
return nil
}
// waitReadWithTimeout will wait full n bytes or until timeout.
func (c *connection) waitReadWithTimeout(n int, timeout time.Duration) (err error) {
if c.readTimer == nil {
c.readTimer = time.NewTimer(timeout)
} else {
c.readTimer.Reset(timeout)
}
for c.inputBuffer.Len() < n {
switch c.status(closing) {
case poller:
// cannot return directly, stop timer first!
err = Exception(ErrEOF, "wait read")
goto RET
case user:
// cannot return directly, stop timer first!
err = Exception(ErrConnClosed, "wait read")
goto RET
default:
select {
case <-c.readTimer.C:
// double check if there is enough data to be read
if c.inputBuffer.Len() >= n {
return nil
}
return Exception(ErrReadTimeout, c.remoteAddr.String())
case err = <-c.readTrigger:
if err != nil {
goto RET
}
continue
}
}
}
RET:
// clean timer.C
if !c.readTimer.Stop() {
<-c.readTimer.C
}
return err
}
// flush writes data directly.
func (c *connection) flush() error {
if c.outputBuffer.IsEmpty() {
return nil
}
bs := c.outputBuffer.GetBytes(c.outputBarrier.bs)
n, err := sendmsg(c.fd, bs, c.outputBarrier.ivs, false)
if err != nil && err != syscall.EAGAIN {
return Exception(err, "when flush")
}
if n > 0 {
err = c.outputBuffer.Skip(n)
c.outputBuffer.Release()
if err != nil {
return Exception(err, "when flush")
}
}
// return if write all buffer.
if c.outputBuffer.IsEmpty() {
return nil
}
err = c.operator.Control(PollR2RW)
if err != nil {
return Exception(err, "when flush")
}
return c.waitFlush()
}
func (c *connection) waitFlush() (err error) {
timeout := c.writeTimeout
if dl := c.writeDeadline; dl > 0 {
timeout = time.Duration(dl - time.Now().UnixNano())
if timeout <= 0 {
return Exception(ErrWriteTimeout, c.remoteAddr.String())
}
}
if timeout == 0 {
return <-c.writeTrigger
}
// set write timeout
if c.writeTimer == nil {
c.writeTimer = time.NewTimer(timeout)
} else {
c.writeTimer.Reset(timeout)
}
select {
case err = <-c.writeTrigger:
if !c.writeTimer.Stop() { // clean timer
<-c.writeTimer.C
}
return err
case <-c.writeTimer.C:
select {
// try fetch writeTrigger if both cases fires
case err = <-c.writeTrigger:
return err
default:
}
// if timeout, remove write event from poller
// we cannot flush it again, since we don't if the poller is still process outputBuffer
c.operator.Control(PollRW2R)
return Exception(ErrWriteTimeout, c.remoteAddr.String())
}
}
func (c *connection) getState() connState {
return atomic.LoadInt32(&c.state)
}
func (c *connection) setState(newState connState) {
atomic.StoreInt32(&c.state, newState)
}
func (c *connection) changeState(from, to connState) bool {
return atomic.CompareAndSwapInt32(&c.state, from, to)
}
================================================
FILE: connection_lock.go
================================================
// Copyright 2022 CloudWeGo Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package netpoll
import (
"runtime"
"sync/atomic"
)
type who = int32
const (
none who = iota
user
poller
)
type key int32
/* State Diagram
+--------------+ +--------------+
| processing |-------->| flushing |
+-------+------+ +-------+------+
|
| +--------------+
+--------------->| closing |
+--------------+
- "processing" locks onRequest handler, and doesn't exist in dialer.
- "flushing" locks outputBuffer
- "closing" should wait for flushing finished and call the closeCallback after that.
*/
const (
closing key = iota
connecting
processing
flushing
// total must be at the bottom.
total
)
type locker struct {
// keychain used for lock/unlock/stop operation by who.
// 0 means unlock, 1 means locked, 2 means stop.
keychain [total]int32
}
func (l *locker) closeBy(w who) (success bool) {
return atomic.CompareAndSwapInt32(&l.keychain[closing], 0, w)
}
func (l *locker) isCloseBy(w who) (yes bool) {
return atomic.LoadInt32(&l.keychain[closing]) == w
}
func (l *locker) status(k key) int32 {
return atomic.LoadInt32(&l.keychain[k])
}
func (l *locker) force(k key, v int32) {
atomic.StoreInt32(&l.keychain[k], v)
}
func (l *locker) lock(k key) (success bool) {
return atomic.CompareAndSwapInt32(&l.keychain[k], 0, 1)
}
func (l *locker) unlock(k key) {
atomic.StoreInt32(&l.keychain[k], 0)
}
func (l *locker) stop(k key) {
for !atomic.CompareAndSwapInt32(&l.keychain[k], 0, 2) && atomic.LoadInt32(&l.keychain[k]) != 2 {
runtime.Gosched()
}
}
func (l *locker) isUnlock(k key) bool {
return atomic.LoadInt32(&l.keychain[k]) == 0
}
================================================
FILE: connection_onevent.go
================================================
// Copyright 2022 CloudWeGo Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//go:build !windows
// +build !windows
package netpoll
import (
"context"
"sync/atomic"
"github.com/cloudwego/netpoll/internal/runner"
)
// ------------------------------------ implement OnPrepare, OnRequest, CloseCallback ------------------------------------
type gracefulExit interface {
isIdle() (yes bool)
Close() (err error)
}
// onEvent is the collection of event processing.
// OnPrepare, OnRequest, CloseCallback share the lock processing,
// which is a CAS lock and can only be cleared by OnRequest.
type onEvent struct {
ctx context.Context
onConnectCallback atomic.Value
onDisconnectCallback atomic.Value
onRequestCallback atomic.Value
closeCallbacks atomic.Value // value is latest *callbackNode
}
type callbackNode struct {
fn CloseCallback
pre *callbackNode
}
// SetOnConnect set the OnConnect callback.
func (c *connection) SetOnConnect(onConnect OnConnect) error {
if onConnect != nil {
c.onConnectCallback.Store(onConnect)
}
return nil
}
// SetOnDisconnect set the OnDisconnect callback.
func (c *connection) SetOnDisconnect(onDisconnect OnDisconnect) error {
if onDisconnect != nil {
c.onDisconnectCallback.Store(onDisconnect)
}
return nil
}
// SetOnRequest initialize ctx when setting OnRequest.
func (c *connection) SetOnRequest(onRequest OnRequest) error {
if onRequest == nil {
return nil
}
c.onRequestCallback.Store(onRequest)
// fix: trigger OnRequest if there is already input data.
if !c.inputBuffer.IsEmpty() {
c.onRequest()
}
return nil
}
// AddCloseCallback adds a CloseCallback to this connection.
func (c *connection) AddCloseCallback(callback CloseCallback) error {
if callback == nil {
return nil
}
cb := &callbackNode{}
cb.fn = callback
if pre := c.closeCallbacks.Load(); pre != nil {
cb.pre = pre.(*callbackNode)
}
c.closeCallbacks.Store(cb)
return nil
}
// onPrepare supports close connection, but not read/write data.
// connection will be registered by this call after preparing.
func (c *connection) onPrepare(opts *options) (err error) {
if opts != nil {
c.SetOnConnect(opts.onConnect)
c.SetOnDisconnect(opts.onDisconnect)
c.SetOnRequest(opts.onRequest)
c.SetReadTimeout(opts.readTimeout)
c.SetWriteTimeout(opts.writeTimeout)
c.SetIdleTimeout(opts.idleTimeout)
// calling prepare first and then register.
if opts.onPrepare != nil {
c.ctx = opts.onPrepare(c)
}
}
if c.ctx == nil {
c.ctx = context.Background()
}
// prepare may close the connection.
if c.IsActive() {
return c.register()
}
return nil
}
// onConnect is responsible for executing onRequest if there is new data coming after onConnect callback finished.
func (c *connection) onConnect() {
onConnect, _ := c.onConnectCallback.Load().(OnConnect)
if onConnect == nil {
c.changeState(connStateNone, connStateConnected)
return
}
if !c.lock(connecting) {
// it never happens because onDisconnect will not lock connecting if c.connected == 0
return
}
onRequest, _ := c.onRequestCallback.Load().(OnRequest)
c.onProcess(onConnect, onRequest)
}
// when onDisconnect called, c.IsActive() must return false
func (c *connection) onDisconnect() {
onDisconnect, _ := c.onDisconnectCallback.Load().(OnDisconnect)
if onDisconnect == nil {
return
}
onConnect, _ := c.onConnectCallback.Load().(OnConnect)
if onConnect == nil {
// no need lock if onConnect is nil
// it's ok to force set state to disconnected since onConnect is nil
c.setState(connStateDisconnected)
onDisconnect(c.ctx, c)
return
}
// check if OnConnect finished when onConnect != nil && onDisconnect != nil
if c.getState() != connStateNone && c.lock(connecting) { // means OnConnect already finished
// protect onDisconnect run once
// if CAS return false, means OnConnect already helps to run onDisconnect
if c.changeState(connStateConnected, connStateDisconnected) {
onDisconnect(c.ctx, c)
}
c.unlock(connecting)
return
}
// OnConnect is not finished yet, return and let onConnect helps to call onDisconnect
}
// onRequest is responsible for executing the closeCallbacks after the connection has been closed.
func (c *connection) onRequest() (needTrigger bool) {
onRequest, ok := c.onRequestCallback.Load().(OnRequest)
if !ok {
return true
}
// wait onConnect finished first
if c.getState() == connStateNone && c.onConnectCallback.Load() != nil {
// let onConnect to call onRequest
return
}
processed := c.onProcess(nil, onRequest)
// if not processed, should trigger read
return !processed
}
// onProcess is responsible for executing the onConnect/onRequest function serially,
// and make sure the connection has been closed correctly if user call c.Close() in onConnect/onRequest function.
func (c *connection) onProcess(onConnect OnConnect, onRequest OnRequest) (processed bool) {
// task already exists
if !c.lock(processing) {
return false
}
task := func() {
panicked := true
defer func() {
if !panicked {
return
}
// cannot use recover() here, since we don't want to break the panic stack
c.unlock(processing)
if c.IsActive() {
c.Close()
} else {
c.closeCallback(false, false)
}
}()
// trigger onConnect first
if onConnect != nil && c.changeState(connStateNone, connStateConnected) {
c.ctx = onConnect(c.ctx, c)
if !c.IsActive() && c.changeState(connStateConnected, connStateDisconnected) {
// since we hold connecting lock, so we should help to call onDisconnect here
onDisconnect, _ := c.onDisconnectCallback.Load().(OnDisconnect)
if onDisconnect != nil {
onDisconnect(c.ctx, c)
}
}
c.unlock(connecting)
}
START:
// The `onRequest` must be executed at least once if conn have any readable data,
// which is in order to cover the `send & close by peer` case.
if onRequest != nil && c.Reader().Len() > 0 {
_ = onRequest(c.ctx, c)
}
// The processing loop must ensure that the connection meets `IsActive`.
// `onRequest` must either eventually read all the input data or actively Close the connection,
// otherwise the goroutine will fall into a dead loop.
var closedBy who
for {
closedBy = c.status(closing)
// close by user or not processable
if closedBy == user || onRequest == nil || c.Reader().Len() == 0 {
break
}
_ = onRequest(c.ctx, c)
}
// handling callback if connection has been closed.
if closedBy != none {
// if closed by user when processing, it "may" needs detach
needDetach := closedBy == user
// Here is a corner case that operator will be detached twice:
// If server closed the connection(client OnHup will detach op first and closeBy=poller),
// and then client's OnRequest function also closed the connection(closeBy=user).
// But operator already prevent that detach twice will not cause any problem
c.closeCallback(false, needDetach)
panicked = false
return
}
c.unlock(processing)
// Note: Poller's closeCallback call will try to get processing lock failed but here already near to unlock processing.
// So here we need to check connection state again, to avoid connection leak
// double check close state
if c.status(closing) != 0 && c.lock(processing) {
// poller will get the processing lock failed, here help poller do closeCallback
// fd must already detach by poller
c.closeCallback(false, false)
panicked = false
return
}
// double check is processable
if onRequest != nil && c.Reader().Len() > 0 && c.lock(processing) {
goto START
}
// task exits
panicked = false
} // end of task closure func
// add new task
runner.RunTask(c.ctx, task)
return true
}
// closeCallback .
// It can be confirmed that closeCallback and onRequest will not be executed concurrently.
// If onRequest is still running, it will trigger closeCallback on exit.
func (c *connection) closeCallback(needLock, needDetach bool) (err error) {
if needLock && !c.lock(processing) {
return nil
}
if needDetach && c.operator.poll != nil { // If Close is called during OnPrepare, poll is not registered.
// PollDetach only happen when user call conn.Close() or poller detect error
if err := c.operator.Control(PollDetach); err != nil {
logger.Printf("NETPOLL: closeCallback[%v,%v] detach operator failed: %v", needLock, needDetach, err)
}
}
latest := c.closeCallbacks.Load()
if latest == nil {
return nil
}
for callback := latest.(*callbackNode); callback != nil; callback = callback.pre {
callback.fn(c)
}
return nil
}
// register only use for connection register into poll.
func (c *connection) register() (err error) {
err = c.operator.Control(PollReadable)
if err != nil {
logger.Printf("NETPOLL: connection register failed: %v", err)
c.Close()
return Exception(ErrConnClosed, err.Error())
}
return nil
}
// isIdle implements gracefulExit.
func (c *connection) isIdle() (yes bool) {
return c.isUnlock(processing) &&
c.inputBuffer.IsEmpty() &&
c.outputBuffer.IsEmpty()
}
================================================
FILE: connection_reactor.go
================================================
// Copyright 2022 CloudWeGo Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//go:build !windows
// +build !windows
package netpoll
import (
"sync/atomic"
)
// ------------------------------------------ implement FDOperator ------------------------------------------
// onHup means close by poller.
func (c *connection) onHup(p Poll) error {
if !c.closeBy(poller) {
return nil
}
c.triggerRead(Exception(ErrEOF, "peer close"))
c.triggerWrite(Exception(ErrConnClosed, "peer close"))
// call Disconnect callback first
c.onDisconnect()
// It depends on closing by user if OnConnect and OnRequest is nil, otherwise it needs to be released actively.
// It can be confirmed that the OnRequest goroutine has been exited before closeCallback executing,
// and it is safe to close the buffer at this time.
onConnect := c.onConnectCallback.Load()
onRequest := c.onRequestCallback.Load()
needCloseByUser := onConnect == nil && onRequest == nil
if !needCloseByUser {
// already PollDetach when call OnHup
c.closeCallback(true, false)
}
return nil
}
// onClose means close by user.
func (c *connection) onClose() error {
// user code close the connection
if c.closeBy(user) {
c.triggerRead(Exception(ErrConnClosed, "self close"))
c.triggerWrite(Exception(ErrConnClosed, "self close"))
// Detach from poller when processing finished, otherwise it will cause race
c.closeCallback(true, true)
return nil
}
// closed by poller
// still need to change closing status to `user` since OnProcess should not be processed again
c.force(closing, user)
// user code should actively close the connection to recycle resources.
// poller already detached operator
return c.closeCallback(true, false)
}
// closeBuffer recycle input & output LinkBuffer.
func (c *connection) closeBuffer() {
onConnect, _ := c.onConnectCallback.Load().(OnConnect)
onRequest, _ := c.onRequestCallback.Load().(OnRequest)
// if client close the connection, we cannot ensure that the poller is not process the buffer,
// so we need to check the buffer length, and if it's an "unclean" close operation, let's give up to reuse the buffer
if c.inputBuffer.Len() == 0 || onConnect != nil || onRequest != nil {
c.inputBuffer.Close()
}
if c.outputBuffer.Len() == 0 || onConnect != nil || onRequest != nil {
c.outputBuffer.Close()
barrierPool.Put(c.outputBarrier)
}
}
// inputs implements FDOperator.
func (c *connection) inputs(vs [][]byte) (rs [][]byte) {
vs[0] = c.inputBuffer.book(c.bookSize, c.maxSize)
return vs[:1]
}
// inputAck implements FDOperator.
func (c *connection) inputAck(n int) (err error) {
if n <= 0 {
c.inputBuffer.bookAck(0)
return nil
}
// Auto size bookSize.
if n == c.bookSize && c.bookSize < mallocMax {
c.bookSize <<= 1
}
length, _ := c.inputBuffer.bookAck(n)
if c.maxSize < length {
c.maxSize = length
}
if c.maxSize > mallocMax {
c.maxSize = mallocMax
}
needTrigger := true
if length == n { // first start onRequest
needTrigger = c.onRequest()
}
if needTrigger && length >= int(atomic.LoadInt64(&c.waitReadSize)) {
c.triggerRead(nil)
}
return nil
}
// outputs implements FDOperator.
func (c *connection) outputs(vs [][]byte) (rs [][]byte, _ bool) {
if c.outputBuffer.IsEmpty() {
c.rw2r()
return rs, false
}
rs = c.outputBuffer.GetBytes(vs)
return rs, false
}
// outputAck implements FDOperator.
func (c *connection) outputAck(n int) (err error) {
if n > 0 {
c.outputBuffer.Skip(n)
c.outputBuffer.Release()
}
if c.outputBuffer.IsEmpty() {
c.rw2r()
}
return nil
}
// rw2r removed the monitoring of write events.
func (c *connection) rw2r() {
c.operator.Control(PollRW2R)
c.triggerWrite(nil)
}
================================================
FILE: connection_test.go
================================================
// Copyright 2022 CloudWeGo Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//go:build !windows
// +build !windows
package netpoll
import (
"context"
"errors"
"fmt"
"net"
"os"
"runtime"
"strings"
"sync"
"sync/atomic"
"syscall"
"testing"
"time"
)
func BenchmarkConnectionIO(b *testing.B) {
dataSize := 1024 * 16
writeBuffer := make([]byte, dataSize)
rfd, wfd := GetSysFdPairs()
rconn, wconn := new(connection), new(connection)
rconn.init(&netFD{fd: rfd}, &options{onRequest: func(ctx context.Context, connection Connection) error {
read, _ := connection.Reader().Next(dataSize)
_ = wconn.Reader().Release()
_, _ = connection.Writer().WriteBinary(read)
_ = connection.Writer().Flush()
return nil
}})
wconn.init(&netFD{fd: wfd}, new(options))
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
_, _ = wconn.WriteBinary(writeBuffer)
_ = wconn.Flush()
_, _ = wconn.Reader().Next(dataSize)
_ = wconn.Reader().Release()
}
}
func TestConnectionWrite(t *testing.T) {
cycle, caps := 10000, 256
msg, buf := make([]byte, caps), make([]byte, caps)
var wg sync.WaitGroup
wg.Add(1)
var count int32
expect := int32(cycle * caps)
opts := &options{}
opts.onRequest = func(ctx context.Context, connection Connection) error {
n, err := connection.Read(buf)
MustNil(t, err)
if atomic.AddInt32(&count, int32(n)) >= expect {
wg.Done()
}
return nil
}
r, w := GetSysFdPairs()
rconn, wconn := &connection{}, &connection{}
rconn.init(&netFD{fd: r}, opts)
wconn.init(&netFD{fd: w}, opts)
for i := 0; i < cycle; i++ {
n, err := wconn.Write(msg)
MustNil(t, err)
Equal(t, n, len(msg))
}
wg.Wait()
Equal(t, atomic.LoadInt32(&count), expect)
rconn.Close()
}
func TestConnectionLargeWrite(t *testing.T) {
// ci machine don't have 4GB memory, so skip test
t.Skipf("skip large write test for ci job")
totalSize := 1024 * 1024 * 1024 * 4
var wg sync.WaitGroup
wg.Add(1)
opts := &options{}
opts.onRequest = func(ctx context.Context, connection Connection) error {
if connection.Reader().Len() < totalSize {
return nil
}
_, err := connection.Reader().Next(totalSize)
MustNil(t, err)
err = connection.Reader().Release()
MustNil(t, err)
wg.Done()
return nil
}
r, w := GetSysFdPairs()
rconn, wconn := &connection{}, &connection{}
rconn.init(&netFD{fd: r}, opts)
wconn.init(&netFD{fd: w}, opts)
msg := make([]byte, totalSize/4)
for i := 0; i < 4; i++ {
_, err := wconn.Writer().WriteBinary(msg)
MustNil(t, err)
}
wg.Wait()
rconn.Close()
}
func TestConnectionRead(t *testing.T) {
r, w := GetSysFdPairs()
rconn, wconn := &connection{}, &connection{}
err := rconn.init(&netFD{fd: r}, nil)
MustNil(t, err)
err = wconn.init(&netFD{fd: w}, nil)
MustNil(t, err)
size := 256
cycleTime := 1000
msg := make([]byte, size)
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
for i := 0; i < cycleTime; i++ {
buf, err := rconn.Reader().Next(size)
MustNil(t, err)
Equal(t, len(buf), size)
rconn.Reader().Release()
}
}()
for i := 0; i < cycleTime; i++ {
n, err := wconn.Write(msg)
MustNil(t, err)
Equal(t, n, len(msg))
}
wg.Wait()
rconn.Close()
}
// TestConnectionIOReader tests the io.Reader Read method which uses readCopy internally.
// Verifies that Read after Peek preserves exposed buffer until Release.
func TestConnectionIOReader(t *testing.T) {
r, w := GetSysFdPairs()
rconn := &connection{}
rconn.init(&netFD{fd: r}, nil)
msg := make([]byte, 64)
for i := range msg {
msg[i] = byte(i)
}
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
// Peek exposes the underlying buffer
pk, err := rconn.Peek(16)
MustNil(t, err)
Equal(t, len(pk), 16)
// Read copies without exposing
buf := make([]byte, 64)
n, err := rconn.Read(buf)
MustNil(t, err)
Equal(t, n, 64)
for i := 0; i < 64; i++ {
Equal(t, buf[i], byte(i))
}
// Peek data still valid before Release
for i := 0; i < 16; i++ {
Equal(t, pk[i], byte(i))
}
rconn.Release()
}()
syscall.Write(w, msg)
wg.Wait()
rconn.Close()
syscall.Close(w)
}
func TestConnectionReadAfterClosed(t *testing.T) {
r, w := GetSysFdPairs()
rconn := &connection{}
rconn.init(&netFD{fd: r}, nil)
size := 256
msg := make([]byte, size)
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
buf, err := rconn.Reader().Next(size)
MustNil(t, err)
Equal(t, len(buf), size)
}()
time.Sleep(time.Millisecond)
syscall.Write(w, msg)
syscall.Close(w)
wg.Wait()
}
func TestConnectionWaitReadHalfPacket(t *testing.T) {
r, w := GetSysFdPairs()
rconn := &connection{}
rconn.init(&netFD{fd: r}, nil)
size := pagesize * 2
msg := make([]byte, size)
// write half packet
syscall.Write(w, msg[:size/2])
// wait poller reads buffer
for rconn.inputBuffer.Len() <= 0 {
runtime.Gosched()
}
// wait read full packet
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
buf, err := rconn.Reader().Next(size)
Equal(t, atomic.LoadInt64(&rconn.waitReadSize), int64(0))
MustNil(t, err)
Equal(t, len(buf), size)
}()
// write left half packet
for atomic.LoadInt64(&rconn.waitReadSize) <= 0 {
runtime.Gosched()
}
Equal(t, atomic.LoadInt64(&rconn.waitReadSize), int64(size))
syscall.Write(w, msg[size/2:])
wg.Wait()
}
func TestReadTimer(t *testing.T) {
read := time.NewTimer(time.Second)
MustTrue(t, read.Stop())
time.Sleep(time.Millisecond)
Equal(t, len(read.C), 0)
}
func TestReadTrigger(t *testing.T) {
trigger := make(chan int, 1)
select {
case trigger <- 0:
default:
}
Equal(t, len(trigger), 1)
}
func writeAll(fd int, buf []byte) error {
for len(buf) > 0 {
n, err := syscall.Write(fd, buf)
if n < 0 {
return err
}
buf = buf[n:]
}
return nil
}
func createTestTCPListener(t *testing.T) net.Listener {
ln, err := net.Listen("tcp", "127.0.0.1:0")
MustNil(t, err)
return ln
}
// Large packet write test. The socket buffer is 2MB by default, here to verify
// whether Connection.Close can be executed normally after socket output buffer is full.
func TestLargeBufferWrite(t *testing.T) {
ln := createTestTCPListener(t)
defer ln.Close()
address := ln.Addr().String()
ln, err := ConvertListener(ln)
MustNil(t, err)
trigger := make(chan int)
defer close(trigger)
go func() {
for {
conn, err := ln.Accept()
if conn == nil && err == nil {
continue
}
trigger <- conn.(*netFD).fd
<-trigger
err = ln.Close()
MustNil(t, err)
return
}
}()
conn, err := DialConnection("tcp", address, time.Second)
MustNil(t, err)
rfd := <-trigger
var wg sync.WaitGroup
wg.Add(1)
bufferSize := 2 * 1024 * 1024 // 2MB
round := 128
// start large buffer writing
go func() {
defer wg.Done()
for i := 1; i <= round+1; i++ {
_, err := conn.Writer().Malloc(bufferSize)
MustNil(t, err)
err = conn.Writer().Flush()
if i <= round {
MustNil(t, err)
}
}
}()
// wait socket buffer full
time.Sleep(time.Millisecond * 100)
buf := make([]byte, 1024)
for received := 0; received < round*bufferSize; {
n, _ := syscall.Read(rfd, buf)
received += n
}
// close success
err = conn.Close()
MustNil(t, err)
wg.Wait()
trigger <- 1
}
func TestConnectionTimeout(t *testing.T) {
ln, err := net.Listen("tcp", "127.0.0.1:0")
MustNil(t, err)
defer ln.Close()
const (
bufsz = 1 << 20
interval = 10 * time.Millisecond
)
calcRate := func(n int32) int32 {
v := n / int32(time.Second/interval)
if v > bufsz {
panic(v)
}
if v < 1 {
return 1
}
return v
}
wn := int32(1) // for each Read, must <= bufsz
setServerWriteRate := func(n int32) {
atomic.StoreInt32(&wn, calcRate(n))
}
rn := int32(1) // for each Write, must <= bufsz
setServerReadRate := func(n int32) {
atomic.StoreInt32(&rn, calcRate(n))
}
go func() {
for {
conn, err := ln.Accept()
if err != nil {
return
}
// set small SO_SNDBUF/SO_RCVBUF buffer for better control timeout test
tcpconn := conn.(*net.TCPConn)
tcpconn.SetReadBuffer(512)
tcpconn.SetWriteBuffer(512)
go func() {
buf := make([]byte, bufsz)
for {
n := atomic.LoadInt32(&rn)
_, err := conn.Read(buf[:int(n)])
if err != nil {
conn.Close()
return
}
time.Sleep(interval)
}
}()
go func() {
buf := make([]byte, bufsz)
for {
n := atomic.LoadInt32(&wn)
_, err := conn.Write(buf[:int(n)])
if err != nil {
conn.Close()
return
}
time.Sleep(interval)
}
}()
}
}()
newConn := func() Connection {
conn, err := DialConnection("tcp", ln.Addr().String(), time.Second)
MustNil(t, err)
fd := conn.(Conn).Fd()
// set small SO_SNDBUF/SO_RCVBUF buffer for better control timeout test
err = syscall.SetsockoptInt(fd, syscall.SOL_SOCKET, syscall.SO_SNDBUF, 512)
MustNil(t, err)
err = syscall.SetsockoptInt(fd, syscall.SOL_SOCKET, syscall.SO_RCVBUF, 512)
MustNil(t, err)
return conn
}
mallocAndFlush := func(conn Connection, sz int) error {
_, err := conn.Writer().Malloc(sz)
MustNil(t, err)
return conn.Writer().Flush()
}
t.Run("TestWriteTimeout", func(t *testing.T) {
setServerReadRate(10 << 10) // 10KB/s
conn := newConn()
defer conn.Close()
// write 1KB without timeout
err := mallocAndFlush(conn, 1<<10) // ~100ms
MustNil(t, err)
// write 50ms timeout
_ = conn.SetWriteTimeout(50 * time.Millisecond)
err = mallocAndFlush(conn, 1<<20)
MustTrue(t, errors.Is(err, ErrWriteTimeout))
})
t.Run("TestReadTimeout", func(t *testing.T) {
setServerWriteRate(10 << 10) // 10KB/s
conn := newConn()
defer conn.Close()
// read 1KB without timeout
_, err := conn.Reader().Next(1 << 10) // ~100ms
MustNil(t, err)
// read 20KB ~ 2s, 50ms timeout
_ = conn.SetReadTimeout(50 * time.Millisecond)
_, err = conn.Reader().Next(20 << 10)
MustTrue(t, errors.Is(err, ErrReadTimeout))
})
t.Run("TestWriteDeadline", func(t *testing.T) {
setServerReadRate(10 << 10) // 10KB/s
conn := newConn()
defer conn.Close()
// write 1KB without deadline
err := conn.SetWriteDeadline(time.Now())
MustNil(t, err)
err = conn.SetDeadline(time.Time{})
MustNil(t, err)
err = mallocAndFlush(conn, 1<<10) // ~100ms
MustNil(t, err)
// write with deadline
err = conn.SetWriteDeadline(time.Now().Add(50 * time.Millisecond))
MustNil(t, err)
t0 := time.Now()
err = mallocAndFlush(conn, 1<<20)
MustTrue(t, errors.Is(err, ErrWriteTimeout))
MustTrue(t, time.Since(t0)-50*time.Millisecond < 20*time.Millisecond)
// write deadline exceeded
t1 := time.Now()
err = mallocAndFlush(conn, 10<<10)
MustTrue(t, errors.Is(err, ErrWriteTimeout))
MustTrue(t, time.Since(t1) < 20*time.Millisecond)
})
t.Run("TestReadDeadline", func(t *testing.T) {
setServerWriteRate(20 << 10) // 20KB/s
conn := newConn()
defer conn.Close()
// read 1KB without deadline
err := conn.SetReadDeadline(time.Now())
MustNil(t, err)
err = conn.SetDeadline(time.Time{})
MustNil(t, err)
_, err = conn.Reader().Next(1 << 10)
MustNil(t, err)
// read 100KB with deadline
err = conn.SetReadDeadline(time.Now().Add(50 * time.Millisecond))
MustNil(t, err)
t0 := time.Now()
_, err = conn.Reader().Next(100 << 10)
MustTrue(t, errors.Is(err, ErrReadTimeout))
MustTrue(t, time.Since(t0)-50*time.Millisecond < 20*time.Millisecond)
// read 10KB, deadline exceeded
t1 := time.Now()
_, err = conn.Reader().Next(10 << 10)
MustTrue(t, errors.Is(err, ErrReadTimeout))
MustTrue(t, time.Since(t1) < 20*time.Millisecond)
})
}
// TestConnectionLargeMemory is used to verify the memory usage in the large package scenario.
func TestConnectionLargeMemory(t *testing.T) {
var start, end runtime.MemStats
runtime.GC()
runtime.ReadMemStats(&start)
r, w := GetSysFdPairs()
rconn := &connection{}
rconn.init(&netFD{fd: r}, nil)
var wg sync.WaitGroup
rn, wn := 1024, 1*1024*1024
wg.Add(1)
go func() {
defer wg.Done()
_, err := rconn.Reader().Next(wn)
MustNil(t, err)
}()
msg := make([]byte, rn)
for i := 0; i < wn/rn; i++ {
n, err := syscall.Write(w, msg)
if err != nil {
MustNil(t, err)
}
Equal(t, n, rn)
}
runtime.ReadMemStats(&end)
alloc := end.TotalAlloc - start.TotalAlloc
limit := uint64(4 * 1024 * 1024)
Assert(t, alloc <= limit, fmt.Sprintf("alloc[%d] out of memory %d", alloc, limit))
}
// TestSetTCPNoDelay is used to verify the connection initialization set the TCP_NODELAY correctly
func TestSetTCPNoDelay(t *testing.T) {
fd, err := sysSocket(syscall.AF_INET, syscall.SOCK_STREAM, 0)
MustNil(t, err)
conn := &connection{}
conn.init(&netFD{network: "tcp", fd: fd}, nil)
n, _ := syscall.GetsockoptInt(fd, syscall.IPPROTO_TCP, syscall.TCP_NODELAY)
MustTrue(t, n > 0)
err = setTCPNoDelay(fd, false)
MustNil(t, err)
n, _ = syscall.GetsockoptInt(fd, syscall.IPPROTO_TCP, syscall.TCP_NODELAY)
MustTrue(t, n == 0)
}
func TestConnectionUntil(t *testing.T) {
r, w := GetSysFdPairs()
rconn, wconn := &connection{}, &connection{}
rconn.init(&netFD{fd: r}, nil)
wconn.init(&netFD{fd: w}, nil)
loopSize := 10000
msg := make([]byte, 1002)
msg[500], msg[1001] = '\n', '\n'
go func() {
for i := 0; i < loopSize; i++ {
n, err := wconn.Write(msg)
MustNil(t, err)
MustTrue(t, n == len(msg))
}
wconn.Write(msg[:100])
wconn.Close()
}()
for i := 0; i < loopSize*2; i++ {
buf, err := rconn.Reader().Until('\n')
MustNil(t, err)
Equal(t, len(buf), 501)
rconn.Reader().Release()
}
buf, err := rconn.Reader().Until('\n')
Equal(t, len(buf), 100)
Assert(t, errors.Is(err, ErrEOF), err)
}
func TestBookSizeLargerThanMaxSize(t *testing.T) {
r, w := GetSysFdPairs()
rconn, wconn := &connection{}, &connection{}
err := rconn.init(&netFD{fd: r}, nil)
MustNil(t, err)
err = wconn.init(&netFD{fd: w}, nil)
MustNil(t, err)
// prepare data
maxSize := 1024 * 1024 * 128
origin := make([][]byte, 0)
for size := maxSize; size > 0; size = size >> 1 {
ch := 'a' + byte(size%26)
origin = append(origin, make([]byte, size))
for i := 0; i < size; i++ {
origin[len(origin)-1][i] = ch
}
}
// read
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
idx := 0
for size := maxSize; size > 0; size = size >> 1 {
buf, err := rconn.Reader().Next(size)
MustNil(t, err)
Equal(t, string(buf), string(origin[idx]))
err = rconn.Reader().Release()
MustNil(t, err)
idx++
}
}()
// write
for i := 0; i < len(origin); i++ {
n, err := wconn.Write(origin[i])
MustNil(t, err)
Equal(t, n, len(origin[i]))
}
wg.Wait()
rconn.Close()
wconn.Close()
}
func TestConnDetach(t *testing.T) {
ln := createTestTCPListener(t)
defer ln.Close()
address := ln.Addr().String()
// accept => read => write
var wg sync.WaitGroup
go func() {
for {
conn, err := ln.Accept()
if err != nil {
return
}
if conn == nil {
continue
}
wg.Add(1)
go func() {
defer wg.Done()
buf := make([]byte, 1024)
// slow read
_, err := conn.Read(buf)
if err != nil {
return
}
time.Sleep(10 * time.Millisecond)
_, err = conn.Write(buf)
if err != nil {
return
}
}()
}
}()
// dial => detach => write => read
c, err := DialConnection("tcp", address, time.Second)
MustNil(t, err)
conn := c.(*TCPConnection)
err = conn.Detach()
MustNil(t, err)
f := os.NewFile(uintptr(conn.fd), "netpoll-connection")
defer f.Close()
gonetconn, err := net.FileConn(f)
MustNil(t, err)
buf := make([]byte, 1024)
_, err = gonetconn.Write(buf)
MustNil(t, err)
_, err = gonetconn.Read(buf)
MustNil(t, err)
err = gonetconn.Close()
MustNil(t, err)
err = ln.Close()
MustNil(t, err)
err = c.Close()
MustNil(t, err)
wg.Wait()
}
func TestParallelShortConnection(t *testing.T) {
ln := createTestTCPListener(t)
defer ln.Close()
address := ln.Addr().String()
var received int64
el, err := NewEventLoop(func(ctx context.Context, connection Connection) error {
data, err := connection.Reader().Next(connection.Reader().Len())
atomic.AddInt64(&received, int64(len(data)))
if err != nil {
return err
}
// t.Logf("conn[%s] received: %d, active: %v", connection.RemoteAddr(), len(data), connection.IsActive())
return nil
})
MustNil(t, err)
go func() {
el.Serve(ln)
}()
defer el.Shutdown(context.Background())
conns := 100
sizePerConn := 1024
totalSize := conns * sizePerConn
var wg sync.WaitGroup
for i := 0; i < conns; i++ {
wg.Add(1)
go func() {
defer wg.Done()
conn, err := DialConnection("tcp", address, time.Second)
MustNil(t, err)
n, err := conn.Writer().WriteBinary(make([]byte, sizePerConn))
MustNil(t, err)
MustTrue(t, n == sizePerConn)
err = conn.Writer().Flush()
MustNil(t, err)
err = conn.Close()
MustNil(t, err)
}()
}
wg.Wait()
t0 := time.Now()
for atomic.LoadInt64(&received) < int64(totalSize) {
time.Sleep(time.Millisecond)
if time.Since(t0) > 100*time.Millisecond { // max wait 100ms
break
}
}
Equal(t, atomic.LoadInt64(&received), int64(totalSize))
}
func TestConnectionServerClose(t *testing.T) {
ln := createTestTCPListener(t)
defer ln.Close()
address := ln.Addr().String()
/*
Client Server
- Client --- connect --> Server
- Client <-- [ping] --- Server
- Client --- [pong] --> Server
- Client <-- close --- Server
- Client --- close --> Server
*/
const PING, PONG = "ping", "pong"
var wg sync.WaitGroup
el, err := NewEventLoop(
func(ctx context.Context, connection Connection) error {
t.Logf("server.OnRequest: addr=%s", connection.RemoteAddr())
defer wg.Done()
buf, err := connection.Reader().Next(len(PONG)) // pong
Equal(t, string(buf), PONG)
MustNil(t, err)
err = connection.Reader().Release()
MustNil(t, err)
err = connection.Close()
MustNil(t, err)
return err
},
WithOnConnect(func(ctx context.Context, connection Connection) context.Context {
t.Logf("server.OnConnect: addr=%s", connection.RemoteAddr())
defer wg.Done()
// check OnPrepare
v := ctx.Value("prepare").(string)
Equal(t, v, "true")
_, err := connection.Writer().WriteBinary([]byte(PING))
MustNil(t, err)
err = connection.Writer().Flush()
MustNil(t, err)
connection.AddCloseCallback(func(connection Connection) error {
t.Logf("server.CloseCallback: addr=%s", connection.RemoteAddr())
wg.Done()
return nil
})
return ctx
}),
WithOnPrepare(func(connection Connection) context.Context {
t.Logf("server.OnPrepare: addr=%s", connection.RemoteAddr())
defer wg.Done()
//nolint:staticcheck // SA1029 no built-in type string as key
return context.WithValue(context.Background(), "prepare", "true")
}),
)
MustNil(t, err)
defer el.Shutdown(context.Background())
go func() {
err := el.Serve(ln)
if err != nil {
t.Logf("service end with error: %v", err)
}
}()
var clientOnRequest OnRequest = func(ctx context.Context, connection Connection) error {
t.Logf("client.OnRequest: addr=%s", connection.LocalAddr())
defer wg.Done()
buf, err := connection.Reader().Next(len(PING))
MustNil(t, err)
Equal(t, string(buf), PING)
_, err = connection.Writer().WriteBinary([]byte(PONG))
MustNil(t, err)
err = connection.Writer().Flush()
MustNil(t, err)
_, err = connection.Reader().Next(1) // server will not send any data, just wait for server close
MustTrue(t, errors.Is(err, ErrEOF)) // should get EOF when server close
return connection.Close()
}
conns := 10
// server: OnPrepare, OnConnect, OnRequest, CloseCallback
// client: OnRequest, CloseCallback
wg.Add(conns * 6)
for i := 0; i < conns; i++ {
go func() {
conn, err := DialConnection("tcp", address, time.Second)
MustNil(t, err)
err = conn.SetOnRequest(clientOnRequest)
MustNil(t, err)
conn.AddCloseCallback(func(connection Connection) error {
t.Logf("client.CloseCallback: addr=%s", connection.LocalAddr())
defer wg.Done()
return nil
})
}()
}
wg.Wait()
}
func TestWriterAfterClose(t *testing.T) {
r, w := GetSysFdPairs()
rconn, wconn := &connection{}, &connection{}
rconn.init(&netFD{fd: r}, nil)
wconn.init(&netFD{fd: w}, nil)
err := wconn.Close()
MustNil(t, err)
for wconn.IsActive() {
runtime.Gosched()
}
methods := []struct {
name string
fn func() error
}{
{"Malloc", func() error { _, err := wconn.Malloc(1); return err }},
{"MallocAck", func() error { return wconn.MallocAck(0) }},
{"WriteBinary", func() error { _, err := wconn.WriteBinary([]byte("hi")); return err }},
{"WriteString", func() error { _, err := wconn.WriteString("hi"); return err }},
{"WriteByte", func() error { return wconn.WriteByte('a') }},
{"WriteDirect", func() error { return wconn.WriteDirect([]byte("hi"), 0) }},
{"Flush", func() error { return wconn.Flush() }},
}
for _, tc := range methods {
t.Run(tc.name, func(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Fatalf("Writer.%s panicked after Close: %v", tc.name, r)
}
}()
err := tc.fn()
Assert(t, err != nil, fmt.Sprintf("Writer.%s should return error after Close", tc.name))
})
}
rconn.Close()
}
func TestConnectionDailTimeoutAndClose(t *testing.T) {
ln := createTestTCPListener(t)
defer ln.Close()
go func() {
for {
conn, err := ln.Accept()
if err != nil {
return
}
time.Sleep(time.Millisecond)
conn.Close()
}
}()
var wg sync.WaitGroup
for i := 0; i < 100; i++ {
wg.Add(1)
go func() {
defer wg.Done()
conn, err := DialConnection("tcp", ln.Addr().String(), time.Millisecond)
Assert(t, err == nil || strings.Contains(err.Error(), "i/o timeout"), err)
if err == nil { // XXX: conn is always not nil ...
conn.Close()
}
}()
}
wg.Wait()
}
================================================
FILE: docs/guide/guide_cn.md
================================================
# 快速开始
本教程通过一些简单的 [示例][Examples] 帮助您开始使用 [Netpoll][Netpoll]，包括如何使用 [Server](#1-使用-sever)、[Client](#2-使用-dialer) 和 [nocopy API](#3-使用-nocopy-api)。
## 1. 使用 Sever
[这里][server-example] 是一个简单的 server 例子，接下来我们会解释它是如何构建的。
### 1.1 创建 Listener
首先我们需要一个 `Listener`，它可以是 `net.Listener` 或者 `netpoll.Listener`，两者都可以，依据你的代码情况自由选择。
创建 `Listener` 的过程如下：
```go
package main
import "net"
func main() {
listener, err := net.Listen(network, address)
if err != nil {
panic("create net listener failed")
}
...
}
```
或者
```go
package main
import "github.com/cloudwego/netpoll"
func main() {
listener, err := netpoll.CreateListener(network, address)
if err != nil {
panic("create netpoll listener failed")
}
...
}
```
### 1.2 创建 EventLoop
`EventLoop` 是一个事件驱动的调度器，一个真正的 NIO Server，负责连接管理、事件调度等。
参数说明:
* `OnRequest` 是用户应该自己实现来处理业务逻辑的接口。 [注释][netpoll.go] 详细描述了它的行为。
* `Option` 用于自定义 `EventLoop` 创建时的配置，下面的例子展示了它的用法。更多详情请参考 [options][netpoll_options.go]。
创建过程如下：
```go
package main
import (
"time"
"github.com/cloudwego/netpoll"
)
var eventLoop netpoll.EventLoop
func main() {
...
eventLoop, _ = netpoll.NewEventLoop(
handle,
netpoll.WithOnPrepare(prepare),
netpoll.WithReadTimeout(time.Second),
)
...
}
```
### 1.3 运行 Server
`EventLoop` 通过绑定 `Listener` 来提供服务，如下所示。`Serve` 方法为阻塞式调用，直到发生 `panic` 等错误，或者由用户主动调用 `Shutdown` 时触发退出。
```go
package main
import (
"github.com/cloudwego/netpoll"
)
var eventLoop netpoll.EventLoop
func main() {
...
// start listen loop ...
eventLoop.Serve(listener)
}
```
### 1.4 关闭 Server
`EventLoop` 提供了 `Shutdown` 功能，用于优雅地停止服务器。用法如下：
```go
package main
import (
"context"
"time"
"github.com/cloudwego/netpoll"
)
var eventLoop netpoll.EventLoop
func main() {
// stop server ...
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
eventLoop.Shutdown(ctx)
}
```
## 2. 使用 Dialer
[Netpoll][Netpoll] 也支持在 Client 端使用，提供了 `Dialer`，类似于 `net.Dialer`。同样的，[这里][client-example] 展示了一个简单的 Client 端示例，接下来我们详细介绍一下：
### 2.1 快速方式
与 [Net][net] 类似，[Netpoll][Netpoll] 提供了几个用于直接建立连接的公共方法，可以直接调用。 如：
```go
DialConnection(network, address string, timeout time.Duration) (connection Connection, err error)
DialTCP(ctx context.Context, network string, laddr, raddr *TCPAddr) (*TCPConnection, error)
DialUnix(network string, laddr, raddr *UnixAddr) (*UnixConnection, error)
```
### 2.2 创建 Dialer
[Netpoll][Netpoll] 还定义了`Dialer` 接口。 用法如下：（通常推荐使用上一节的快速方式）
```go
package main
import (
"github.com/cloudwego/netpoll"
)
func main() {
// Dial a connection with Dialer.
dialer := netpoll.NewDialer()
conn, err := dialer.DialConnection(network, address, timeout)
if err != nil {
panic("dial netpoll connection failed")
}
...
}
```
## 3. 使用 Nocopy API
`Connection` 提供了 Nocopy API —— `Reader` 和 `Writer`，以避免频繁复制。下面介绍一下它们的简单用法。
```go
package main
type Connection interface {
// Recommended nocopy APIs
Reader() Reader
Writer() Writer
... // see code comments for more details
}
```
### 3.1 简单用法
Nocopy API 设计为两步操作。
使用 `Reader` 时，通过 `Next`、`Peek`、`ReadString` 等方法读取数据后，还需要主动调用 `Release` 方法释放 buffer（`Nocopy` 读取 buffer 的原地址，所以您必须主动再次确认 buffer 已经不再使用）。
同样，使用 `Writer` 时，首先需要分配一个 `[]byte` 来写入数据，然后调用 `Flush` 确认所有数据都已经写入。`Writer` 还提供了丰富的 API 来分配 buffer，例如 `Malloc`、`WriteString` 等。
下面是一些简单的读写数据的例子。 更多详情请参考 [说明][nocopy.go]。
```go
package main
import (
"github.com/cloudwego/netpoll"
)
func main() {
var conn netpoll.Connection
var reader, writer = conn.Reader(), conn.Writer()
// reading
buf, _ := reader.Next(n)
... parse the read data ...
reader.Release()
// writing
var write_data []byte
... make the write data ...
alloc, _ := writer.Malloc(len(write_data))
copy(alloc, write_data) // write data
writer.Flush()
}
```
### 3.2 高阶用法
如果你想使用单个连接来发送（或接收）多组数据（如连接多路复用），那么你将面临数据打包和分包。在 [net][net] 上，这种工作一般都是通过复制来完成的。一个例子如下：
```go
package main
import (
"net"
)
func main() {
var conn net.Conn
var buf = make([]byte, 8192)
// reading
for {
n, _ := conn.Read(buf)
... unpacking & handling ...
var i int
for i = 0; i <= n-pkgsize; i += pkgsize {
pkg := append([]byte{}, buf[i:i+pkgsize]...)
go func() {
... handling pkg ...
}
}
buf = append(buf[:0], buf[i:n]...)
}
// writing
var write_datas <-chan []byte
... packing write ...
for {
pkg := <-write_datas
conn.Write(pkg)
}
}
```
但是，[Netpoll][Netpoll] 不需要这样做，nocopy APIs 支持对 buffer 进行原地址操作（原地址组包和分包），并通过引用计数实现资源的自动回收和重用。
示例如下（使用方法 `Reader.Slice` 和 `Writer.Append`）：
```go
package main
import (
"github.com/cloudwego/netpoll"
)
func main() {
var conn netpoll.Connection
// reading
reader := conn.Reader()
for {
... unpacking & handling ...
pkg, _ := reader.Slice(pkgsize)
go func() {
... handling pkg ...
pkg.Release()
}
}
// writing
var write_datas <-chan netpoll.Writer
... packing write ...
writer := conn.Writer()
for {
select {
case pkg := <-write_datas:
writer.Append(pkg)
default:
if writer.MallocLen() > 0 {
writer.Flush()
}
}
}
}
```
# 常见用法
## 1. 如何配置 poller 的数量 ？
`NumLoops` 表示 [Netpoll][Netpoll] 创建的 `epoll` 的数量，默认已经根据P的数量自动调整(`runtime.GOMAXPROCS(0)`)，用户一般不需要关心。
但是如果你的服务有大量的 I/O，你可能需要如下配置：
```go
package main
import (
"runtime"
"github.com/cloudwego/netpoll"
)
func init() {
netpoll.SetNumLoops(runtime.GOMAXPROCS(0))
}
```
## 2. 如何配置 poller 的连接负载均衡 ？
当 [Netpoll][Netpoll] 中有多个 poller 时，服务进程中的连接会负载均衡到每个 poller。
现在支持以下策略：
1. Random
* 新连接将分配给随机选择的轮询器。
2. RoundRobin
* 新连接将按顺序分配给轮询器。
[Netpoll][Netpoll] 默认使用 `RoundRobin`，用户可以通过以下方式更改：
```go
package main
import (
"github.com/cloudwego/netpoll"
)
func init() {
netpoll.SetLoadBalance(netpoll.Random)
// or
netpoll.SetLoadBalance(netpoll.RoundRobin)
}
```
## 3. 如何配置 [gopool][gopool] ？
[Netpoll][Netpoll] 默认使用 [gopool][gopool] 作为 goroutine 池来优化 `栈扩张` 问题（RPC 服务常见问题）。
[gopool][gopool] 项目中已经详细解释了如何自定义配置，这里不再赘述。
当然，如果你的项目没有 `栈扩张` 问题，建议最好关闭 [gopool][gopool]，关闭方式如下：
```go
package main
import (
"github.com/cloudwego/netpoll"
)
func init() {
netpoll.DisableGopool()
}
```
## 4. 如何初始化新的连接 ？
Client 和 Server 端通过不同的方式初始化新连接。
1. 在 Server 端，定义了 `OnPrepare` 来初始化新链接，同时支持返回一个 `context`，可以传递给后续的业务处理并复用。`WithOnPrepare` 提供方法注册。当 Server 接收新连接时，会自动执行注册的 `OnPrepare` 方法来完成准备工作。示例如下：
```go
package main
import (
"context"
"github.com/cloudwego/netpoll"
)
func main() {
// register OnPrepare
var onPrepare netpoll.OnPrepare = prepare
evl, _ := netpoll.NewEventLoop(handler, netpoll.WithOnPrepare(onPrepare))
...
}
func prepare(connection netpoll.Connection) (ctx context.Context) {
... prepare connection ...
return
}
```
2. 在 Client 端，连接初始化需要由用户自行完成。 一般来说，`Dialer` 创建的新连接是可以由用户自行控制的，这与 Server 端被动接收连接不同。因此，用户不需要依赖触发器，可以自行初始化，如下所示：
```go
package main
import (
"context"
"github.com/cloudwego/netpoll"
)
func main() {
conn, err := netpoll.DialConnection(network, address, timeout)
if err != nil {
panic("dial netpoll connection failed")
}
... prepare here directly ...
prepare(conn)
...
}
func prepare(connection netpoll.Connection) (ctx context.Context) {
... prepare connection ...
return
}
```
## 5. 如何配置连接超时 ？
[Netpoll][Netpoll] 现在支持两种类型的超时配置：
1. 读超时（`ReadTimeout`）
* 为了保持与 `net.Conn` 相同的操作风格，`Connection.Reader` 也被设计为阻塞读取。 所以提供了读取超时（`ReadTimeout`）。
* 读超时（`ReadTimeout`）没有默认值（默认无限等待），可以通过 `Connection` 或 `EventLoop.Option` 进行配置，例如：
```go
package main
import (
"github.com/cloudwego/netpoll"
)
func main() {
var conn netpoll.Connection
// 1. setting by Connection
conn.SetReadTimeout(timeout)
// or
// 2. setting with Option
netpoll.NewEventLoop(handler, netpoll.WithReadTimeout(timeout))
...
}
```
2. 空闲超时（`IdleTimeout`）
* 空闲超时（`IdleTimeout`）利用 `TCP KeepAlive` 机制来踢出死连接并减少维护开销。使用 [Netpoll][Netpoll] 时，一般不需要频繁创建和关闭连接，所以通常来说，空闲连接影响不大。当连接长时间处于非活动状态时，为了防止出现假死、对端挂起、异常断开等造成的死连接，在空闲超时（`IdleTimeout`）后，netpoll 会主动关闭连接。
* 空闲超时（`IdleTimeout`）的默认配置为 `10min`，可以通过 `Connection` API 或 `EventLoop.Option` 进行配置，例如：
```go
package main
import (
"github.com/cloudwego/netpoll"
)
func main() {
var conn netpoll.Connection
// 1. setting by Connection
conn.SetIdleTimeout(timeout)
// or
// 2. setting with Option
netpoll.NewEventLoop(handler, netpoll.WithIdleTimeout(timeout))
...
}
```
## 6. 如何配置连接的读事件回调 ？
`OnRequest` 是指连接上发生读事件时 [Netpoll][Netpoll] 触发的回调。在 Server 端，在创建 `EventLoop` 时，可以注册一个`OnRequest`，在每次连接数据到达时触发，进行业务处理。Client端默认没有 `OnRequest`，需要时可以通过 API 设置。例如：
```go
package main
import (
"context"
"github.com/cloudwego/netpoll"
)
func main() {
var onRequest netpoll.OnRequest = handler
// 1. on server side
evl, _ := netpoll.NewEventLoop(onRequest, opts...)
...
// 2. on client side
conn, _ := netpoll.DialConnection(network, address, timeout)
conn.SetOnRequest(handler)
...
}
func handler(ctx context.Context, connection netpoll.Connection) (err error) {
... handling ...
return nil
}
```
## 7. 如何配置连接的关闭回调 ？
`CloseCallback` 是指连接关闭时 [Netpoll][Netpoll] 触发的回调，用于在连接关闭后进行额外的处理。
[Netpoll][Netpoll] 能够感知连接状态。当连接被对端关闭或被自己清理时，会主动触发 `CloseCallback`，而不是由下一次调用 `Read` 或 `Write` 时返回错误（`net.Conn` 的方式）。
`Connection` 提供了添加 `CloseCallback` 的 API，已经添加的回调无法删除，支持多个回调。
```go
package main
import (
"github.com/cloudwego/netpoll"
)
func main() {
var conn netpoll.Connection
// add close callback
var cb netpoll.CloseCallback = callback
conn.AddCloseCallback(cb)
...
}
func callback(connection netpoll.Connection) error {
return nil
}
```
# 注意事项
## 1. 错误设置 NumLoops
如果你的服务器运行在物理机上，Go 进程创建的 P 个数就等于机器的 CPU 核心数。 但是 Server 可能不会使用这么多核心。在这种情况下，过多的 poller 会导致性能下降。
这里提供了以下几种解决方案：
1. 使用 `taskset` 命令来限制 CPU 个数，例如：
```shell
taskset -c 0-3 $run_your_server
```
2. 主动设置 P 的个数，例如：
```go
package main
import (
"runtime"
)
func init() {
runtime.GOMAXPROCS(num_you_want)
}
```
3. 主动设置 poller 的个数，例如：
```go
package main
import (
"github.com/cloudwego/netpoll"
)
func init() {
netpoll.SetNumLoops(num_you_want)
}
```
[Netpoll]: https://github.com/cloudwego/netpoll
[net]: https://github.com/golang/go/tree/master/src/net
[gopool]: https://github.com/bytedance/gopkg/tree/develop/util/gopool
[Examples]: https://github.com/cloudwego/netpoll-examples
[server-example]: https://github.com/cloudwego/netpoll-examples/blob/main/server.go
[client-example]: https://github.com/cloudwego/netpoll-examples/blob/main/client.go
[netpoll.go]: https://github.com/cloudwego/netpoll/blob/main/netpoll.go
[netpoll_options.go]: https://github.com/cloudwego/netpoll/blob/main/netpoll_options.go
[nocopy.go]: https://github.com/cloudwego/netpoll/blob/main/nocopy.go
================================================
FILE: docs/guide/guide_en.md
================================================
# Tutorial
This tutorial gets you started with [Netpoll][Netpoll] through some simple [examples][Examples], includes how to
use [Server](#1-use-sever), [Client](#2-use-dialer) and [nocopy APIs](#3-use-nocopy-api).
## 1. Use Server
[Here][server-example] is a simple server demo, we will explain how it is constructed next.
### 1.1 Create Listener
First we need to get a `Listener`, it can be `net.Listener` or `netpoll.Listener`, which is no difference for server
usage. Create a `Listener` as shown below:
```go
package main
import "net"
func main() {
listener, err := net.Listen(network, address)
if err != nil {
panic("create net listener failed")
}
...
}
```
or
```go
package main
import "github.com/cloudwego/netpoll"
func main() {
listener, err := netpoll.CreateListener(network, address)
if err != nil {
panic("create netpoll listener failed")
}
...
}
```
### 1.2 New EventLoop
`EventLoop` is an event-driven scheduler, a real NIO Server, responsible for connection management, event scheduling,
etc.
params:
* `OnRequest` is an interface that users should implement by themselves to process business
logic. [Code Comment][netpoll.go] describes its behavior in detail.
* `Option` is used to customize the configuration when creating `EventLoop`, and the following example shows its usage.
For more details, please refer to [options][netpoll_options.go].
The creation process is as follows:
```go
package main
import (
"time"
"github.com/cloudwego/netpoll"
)
var eventLoop netpoll.EventLoop
func main() {
...
eventLoop, _ := netpoll.NewEventLoop(
handle,
netpoll.WithOnPrepare(prepare),
netpoll.WithReadTimeout(time.Second),
)
...
}
```
### 1.3 Run Server
`EventLoop` provides services by binding `Listener`, as shown below.
`Serve` function will block until an error occurs, such as a panic or the user actively calls `Shutdown`.
```go
package main
import (
"github.com/cloudwego/netpoll"
)
var eventLoop netpoll.EventLoop
func main() {
...
// start listen loop ...
eventLoop.Serve(listener)
}
```
### 1.4 Shutdown Server
`EventLoop` provides the `Shutdown` function, which is used to stop the server gracefully. The usage is as follows.
```go
package main
import (
"context"
"time"
"github.com/cloudwego/netpoll"
)
var eventLoop netpoll.EventLoop
func main() {
// stop server ...
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
eventLoop.Shutdown(ctx)
}
```
## 2. Use Dialer
[Netpoll][Netpoll] also has the ability to be used on the Client side. It provides `Dialer`, similar to `net.Dialer`.
Again, [here][client-example] is a simple client demo, and then we introduce it in detail.
### 2.1 The Fast Way
Similar to [Net][net], [Netpoll][Netpoll] provides several public functions for directly dialing a connection. such as:
```go
DialConnection(network, address string, timeout time.Duration) (connection Connection, err error)
DialTCP(ctx context.Context, network string, laddr, raddr *TCPAddr) (*TCPConnection, error)
DialUnix(network string, laddr, raddr *UnixAddr) (*UnixConnection, error)
```
### 2.2 Create Dialer
[Netpoll][Netpoll] also defines the `Dialer` interface. The usage is as follows:
(of course, you can usually use the fast way)
```go
package main
import (
"github.com/cloudwego/netpoll"
)
func main() {
// Dial a connection with Dialer.
dialer := netpoll.NewDialer()
conn, err := dialer.DialConnection(network, address, timeout)
if err != nil {
panic("dial netpoll connection failed")
}
...
}
```
## 3. Use Nocopy API
`Connection` provides Nocopy APIs - `Reader` and `Writer`, to avoid frequent copying. Let’s introduce their simple
usage.
```go
package main
type Connection interface {
// Recommended nocopy APIs
Reader() Reader
Writer() Writer
... // see code comments for more details
}
```
### 3.1 Simple Usage
Nocopy APIs is designed as a two-step operation.
On `Reader`, after reading data through `Next`, `Peek`, `ReadString`, etc., you still have to actively call `Release` to
release the buffer(`Nocopy` reads the original address of the buffer, so you must take the initiative to confirm that
the buffer is no longer used).
Similarly, on `Writer`, you first need to allocate a buffer to write data, and then call `Flush` to confirm that all
data has been written.
`Writer` also provides rich APIs to allocate buffers, such as `Malloc`, `WriteString` and so on.
The following shows some simple examples of reading and writing data. For more details, please refer to
the [code comments][nocopy.go].
```go
package main
import (
"github.com/cloudwego/netpoll"
)
func main() {
var conn netpoll.Connection
var reader, writer = conn.Reader(), conn.Writer()
// reading
buf, _ := reader.Next(n)
... parse the read data ...
reader.Release()
// writing
var write_data []byte
... make the write data ...
alloc, _ := writer.Malloc(len(write_data))
copy(alloc, write_data) // write data
writer.Flush()
}
```
### 3.2 Advanced Usage
If you want to use the connection to send (or receive) multiple sets of data, then you will face the work of packing and
unpacking the data.
On [net][net], this kind of work is generally done by copying. An example is as follows:
```go
package main
import (
"net"
)
func main() {
var conn net.Conn
var buf = make([]byte, 8192)
// reading
for {
n, _ := conn.Read(buf)
... unpacking & handling ...
var i int
for i = 0; i <= n-pkgsize; i += pkgsize {
pkg := append([]byte{}, buf[i:i+pkgsize]...)
go func() {
... handling pkg ...
}
}
buf = append(buf[:0], buf[i:n]...)
}
// writing
var write_datas <-chan []byte
... packing write ...
for {
pkg := <-write_datas
conn.Write(pkg)
}
}
```
But, this is not necessary in [Netpoll][Netpoll], nocopy APIs supports operations on the original address of the buffer,
and realizes automatic recycling and reuse of resources through reference counting.
Examples are as follows(use function `Reader.Slice` and `Writer.Append`):
```go
package main
import (
"github.com/cloudwego/netpoll"
)
func main() {
var conn netpoll.Connection
// reading
reader := conn.Reader()
for {
... unpacking & handling ...
pkg, _ := reader.Slice(pkgsize)
go func() {
... handling pkg ...
pkg.Release()
}
}
// writing
var write_datas <-chan netpoll.Writer
... packing write ...
writer := conn.Writer()
for {
select {
case pkg := <-write_datas:
writer.Append(pkg)
default:
if writer.MallocLen() > 0 {
writer.Flush()
}
}
}
}
```
# How To
## 1. How to configure the number of pollers ?
`NumLoops` represents the number of `epoll` created by [Netpoll][Netpoll], which has been automatically adjusted
according to the number of P (`runtime.GOMAXPROCS(0)`) by default, and users generally don't need to care.
But if your service has heavy I/O, you may need the following configuration:
```go
package main
import (
"runtime"
"github.com/cloudwego/netpoll"
)
func init() {
netpoll.SetNumLoops(runtime.GOMAXPROCS(0))
}
```
## 2. How to configure poller's connection loadbalance ?
When there are multiple pollers in [Netpoll][Netpoll], the connections in the service process will be loadbalanced to
each poller.
The following strategies are supported now:
1. Random
* The new connection will be assigned to a randomly picked poller.
2. RoundRobin
* The new connection will be assigned to the poller in order.
[Netpoll][Netpoll] uses `RoundRobin` by default, and users can change it in the following ways:
```go
package main
import (
"github.com/cloudwego/netpoll"
)
func init() {
netpoll.SetLoadBalance(netpoll.Random)
// or
netpoll.SetLoadBalance(netpoll.RoundRobin)
}
```
## 3. How to configure [gopool][gopool] ?
[Netpoll][Netpoll] uses [gopool][gopool] as the goroutine pool by default to optimize the `stack growth` problem that
generally occurs in RPC services.
In the project [gopool][gopool], it explains how to change its configuration, so won't repeat it here.
Of course, if your project does not have a `stack growth` problem, it is best to close [gopool][gopool] as follows:
```go
package main
import (
"github.com/cloudwego/netpoll"
)
func init() {
netpoll.DisableGopool()
}
```
## 4. How to prepare a new connection ?
There are different ways to prepare a new connection on the client and server.
1. On the server side, `OnPrepare` is defined to prepare for the new connection, and it also supports returning
a `context`, which can be reused in subsequent business processing.
`WithOnPrepare` provides this registration. When the server accepts a new connection, it will automatically execute
the registered `OnPrepare` function to complete the preparation work. The example is as follows:
```go
package main
import (
"context"
"github.com/cloudwego/netpoll"
)
func main() {
// register OnPrepare
var onPrepare netpoll.OnPrepare = prepare
evl, _ := netpoll.NewEventLoop(handler, netpoll.WithOnPrepare(onPrepare))
...
}
func prepare(connection netpoll.Connection) (ctx context.Context) {
... prepare connection ...
return
}
```
2. On the client side, the connection preparation needs to be completed by the user. Generally speaking, the connection
created by `Dialer` can be controlled by the user, which is different from passively accepting the connection on the
server side. Therefore, the user not relying on the trigger, just prepare a new connection like this:
```go
package main
import (
"context"
"github.com/cloudwego/netpoll"
)
func main() {
conn, err := netpoll.DialConnection(network, address, timeout)
if err != nil {
panic("dial netpoll connection failed")
}
... prepare here directly ...
prepare(conn)
...
}
func prepare(connection netpoll.Connection) (ctx context.Context) {
... prepare connection ...
return
}
```
## 5. How to configure connection timeout ?
[Netpoll][Netpoll] now supports two timeout configurations:
1. `Read Timeout`
* In order to maintain the same operating style as `net.Conn`, `Connection.Reader` is also designed to block
reading. So provide `Read Timeout`.
* `Read Timeout` has no default value(wait infinitely), it can be configured via `Connection` or `EventLoop.Option`,
for example:
```go
package main
import (
"github.com/cloudwego/netpoll"
)
func main() {
var conn netpoll.Connection
// 1. setting by Connection
conn.SetReadTimeout(timeout)
// or
// 2. setting with Option
netpoll.NewEventLoop(handler, netpoll.WithReadTimeout(timeout))
...
}
```
2. `Idle Timeout`
* `Idle Timeout` utilizes the `TCP KeepAlive` mechanism to kick out dead connections and reduce maintenance
overhead. When using [Netpoll][Netpoll], there is generally no need to create and close connections frequently,
and idle connections have little effect. When the connection is inactive for a long time, in order to prevent dead
connection caused by suspended animation, hang of the opposite end, abnormal disconnection, etc., the connection
will be actively closed after the `Idle Timeout`.
* The default minimum value of `Idle Timeout` is `10min`, which can be configured through `Connection` API
or `EventLoop.Option`, for example:
```go
package main
import (
"github.com/cloudwego/netpoll"
)
func main() {
var conn netpoll.Connection
// 1. setting by Connection
conn.SetIdleTimeout(timeout)
// or
// 2. setting with Option
netpoll.NewEventLoop(handler, netpoll.WithIdleTimeout(timeout))
...
}
```
## 6. How to configure connection read event callback ?
`OnRequest` refers to the callback triggered by [Netpoll][Netpoll] when a read event occurs on the connection. On the
Server side, when creating the `EventLoop`, you can register an `OnRequest`, which will be triggered when each
connection data arrives and perform business processing. On the Client side, there is no `OnRequest` by default, and it
can be set via API when needed. E.g:
```go
package main
import (
"context"
"github.com/cloudwego/netpoll"
)
func main() {
var onRequest netpoll.OnRequest = handler
// 1. on server side
evl, _ := netpoll.NewEventLoop(onRequest, opts...)
...
// 2. on client side
conn, _ := netpoll.DialConnection(network, address, timeout)
conn.SetOnRequest(handler)
...
}
func handler(ctx context.Context, connection netpoll.Connection) (err error) {
... handling ...
return nil
}
```
## 7. How to configure the connection close callback ?
`CloseCallback` refers to the callback triggered by [Netpoll][Netpoll] when the connection is closed, which is used to
perform additional processing after the connection is closed.
[Netpoll][Netpoll] is able to perceive the connection status. When the connection is closed by peer or cleaned up by
self, it will actively trigger `CloseCallback` instead of returning an error on the next `Read` or `Write`(the way
of `net.Conn`).
`Connection` provides API for adding `CloseCallback`, callbacks that have been added cannot be removed, and multiple
callbacks are supported.
```go
package main
import (
"github.com/cloudwego/netpoll"
)
func main() {
var conn netpoll.Connection
// add close callback
var cb netpoll.CloseCallback = callback
conn.AddCloseCallback(cb)
...
}
func callback(connection netpoll.Connection) error {
return nil
}
```
# Attention
## 1. Wrong setting of NumLoops
If your server is running on a physical machine, the number of P created by the Go process is equal to the number of
CPUs of the machine. But the server may not use so many cores. In this case, too many pollers will cause performance
degradation.
There are several solutions:
1. Use the `taskset` command to limit CPU usage, such as:
```shell
taskset -c 0-3 $run_your_server
```
2. Actively set the number of P, for instance:
```go
package main
import (
"runtime"
)
func init() {
runtime.GOMAXPROCS(num_you_want)
}
```
3. Actively set the number of pollers, e.g:
```go
package main
import (
"github.com/cloudwego/netpoll"
)
func init() {
netpoll.SetNumLoops(num_you_want)
}
```
[Netpoll]: https://github.com/cloudwego/netpoll
[net]: https://github.com/golang/go/tree/master/src/net
[gopool]: https://github.com/bytedance/gopkg/tree/develop/util/gopool
[Examples]: https://github.com/cloudwego/netpoll-examples
[server-example]: https://github.com/cloudwego/netpoll-examples/blob/main/server.go
[client-example]: https://github.com/cloudwego/netpoll-examples/blob/main/client.go
[netpoll.go]: https://github.com/cloudwego/netpoll/blob/main/netpoll.go
[netpoll_options.go]: https://github.com/cloudwego/netpoll/blob/main/netpoll_options.go
[nocopy.go]: https://github.com/cloudwego/netpoll/blob/main/nocopy.go
================================================
FILE: docs/reference/design_cn.md
================================================
# TODO
================================================
FILE: docs/reference/design_en.md
================================================
# TODO
================================================
FILE: docs/reference/explain.md
================================================
# DATA RACE EXPLAIN
`Netpoll` declare different files by `//+build !race` and `//+build race` to avoid `DATA RACE` detection in some code.
The reason is that the `epoll` uses `unsafe.Pointer` to access the struct pointer, in order
to improve performance. This operation is beyond the detection range of the `race detector`,
so it is mistaken for data race, but not code bug actually.
================================================
FILE: eventloop.go
================================================
// Copyright 2022 CloudWeGo Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package netpoll
import (
"context"
"net"
)
// A EventLoop is a network server.
type EventLoop interface {
// Serve registers a listener and runs blockingly to provide services, including listening to ports,
// accepting connections and processing trans data. When an exception occurs or Shutdown is invoked,
// Serve will return an error which describes the specific reason.
Serve(ln net.Listener) error
// Shutdown is used to graceful exit.
// It will close all idle connections on the server, but will not change the underlying pollers.
//
// Argument: ctx set the waiting deadline, after which an error will be returned,
// but will not force the closing of connections in progress.
Shutdown(ctx context.Context) error
}
/* The Connection Callback Sequence Diagram
| Connection State | Callback Function | Notes
| Connected but not initialized | OnPrepare | Conn is not registered into poller
| Connected and initialized | OnConnect | Conn is ready for read or write
| Read first byte | OnRequest | Conn is ready for read or write
| Peer closed but conn is active | OnDisconnect | Conn access will race with OnRequest function
| Self closed and conn is closed | CloseCallback | Conn is destroyed
Execution Order:
OnPrepare => OnConnect => OnRequest => CloseCallback
OnDisconnect
Note: only OnRequest and OnDisconnect will be executed in parallel
*/
// OnPrepare is used to inject custom preparation at connection initialization,
// which is optional but important in some scenarios. For example, a qps limiter
// can be set by closing overloaded connections directly in OnPrepare.
//
// Return:
// context will become the argument of OnRequest.
// Usually, custom resources can be initialized in OnPrepare and used in OnRequest.
//
// PLEASE NOTE:
// OnPrepare is executed without any data in the connection,
// so Reader() or Writer() cannot be used here, but may be supported in the future.
type OnPrepare func(connection Connection) context.Context
// OnConnect is called once connection created.
// It supports read/write/close connection, and could return a ctx which will be passed to OnRequest.
// OnConnect will not block the poller since it's executed asynchronously.
// Only after OnConnect finished the OnRequest could be executed.
//
// An example usage in TCP Proxy scenario:
//
// func onConnect(ctx context.Context, upstream netpoll.Connection) context.Context {
// downstream, _ := netpoll.DialConnection("tcp", downstreamAddr, time.Second)
// return context.WithValue(ctx, downstreamKey, downstream)
// }
//
// func onRequest(ctx context.Context, upstream netpoll.Connection) error {
// downstream := ctx.Value(downstreamKey).(netpoll.Connection)
// }
type OnConnect func(ctx context.Context, connection Connection) context.Context
// OnDisconnect is called once connection is going to be closed.
// OnDisconnect must return as quick as possible because it will block poller.
// OnDisconnect is different from CloseCallback, you could check with "The Connection Callback Sequence Diagram" section.
type OnDisconnect func(ctx context.Context, connection Connection)
// OnRequest defines the function for handling connection. When data is sent from the connection peer,
// netpoll actively reads the data in LT mode and places it in the connection's input buffer.
// Generally, OnRequest starts handling the data in the following way:
//
// func OnRequest(ctx context, connection Connection) error {
// input := connection.Reader().Next(n)
// handling input data...
// send, _ := connection.Writer().Malloc(l)
// copy(send, output)
// connection.Flush()
// return nil
// }
//
// OnRequest will run in a separate goroutine and
// it is guaranteed that there is one and only one OnRequest running at the same time.
// The underlying logic is similar to:
//
// go func() {
// for !connection.Reader().IsEmpty() {
// OnRequest(ctx, connection)
// }
// }()
//
// PLEASE NOTE:
// OnRequest must either eventually read all the input data or actively Close the connection,
// otherwise the goroutine will fall into a dead loop.
//
// Return: error is unused which will be ignored directly.
type OnRequest func(ctx context.Context, connection Connection) error
================================================
FILE: fd_operator.go
================================================
// Copyright 2022 CloudWeGo Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package netpoll
import (
"runtime"
"sync/atomic"
)
// FDOperator is a collection of operations on file descriptors.
type FDOperator struct {
// FD is file descriptor, poll will bind when register.
FD int
// The FDOperator provides three operations of reading, writing, and hanging.
// The poll actively fire the FDOperator when fd changes, no check the return value of FDOperator.
OnRead func(p Poll) error
OnWrite func(p Poll) error
OnHup func(p Poll) error
// The following is the required fn, which must exist when used, or directly panic.
// Fns are only called by the poll when handles connection events.
Inputs func(vs [][]byte) (rs [][]byte)
InputAck func(n int) (err error)
// Outputs will locked if len(rs) > 0, which need unlocked by OutputAck.
// supportZeroCopy is not implemented, and it will be ignored
Outputs func(vs [][]byte) (rs [][]byte, supportZeroCopy bool)
OutputAck func(n int) (err error)
// poll is the registered location of the file descriptor.
poll Poll
// protect only detach once
detached int32
// private, used by operatorCache
next *FDOperator
state int32 // CAS: 0(unused) 1(inuse) 2(do-done)
index int32 // index in operatorCache
}
func (op *FDOperator) Control(event PollEvent) error {
if event == PollDetach && atomic.AddInt32(&op.detached, 1) > 1 {
return nil
}
return op.poll.Control(op, event)
}
func (op *FDOperator) Free() {
op.poll.Free(op)
}
func (op *FDOperator) do() (can bool) {
return atomic.CompareAndSwapInt32(&op.state, 1, 2)
}
func (op *FDOperator) done() {
atomic.StoreInt32(&op.state, 1)
}
func (op *FDOperator) inuse() {
for !atomic.CompareAndSwapInt32(&op.state, 0, 1) {
if atomic.LoadInt32(&op.state) == 1 {
return
}
runtime.Gosched()
}
}
func (op *FDOperator) unused() {
for !atomic.CompareAndSwapInt32(&op.state, 1, 0) {
if atomic.LoadInt32(&op.state) == 0 {
return
}
runtime.Gosched()
}
}
func (op *FDOperator) isUnused() bool {
return atomic.LoadInt32(&op.state) == 0
}
func (op *FDOperator) reset() {
op.FD = 0
op.OnRead, op.OnWrite, op.OnHup = nil, nil, nil
op.Inputs, op.InputAck = nil, nil
op.Outputs, op.OutputAck = nil, nil
op.poll = nil
op.detached = 0
}
================================================
FILE: fd_operator_cache.go
================================================
// Copyright 2022 CloudWeGo Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package netpoll
import (
"runtime"
"sync/atomic"
"unsafe"
)
func newOperatorCache() *operatorCache {
return &operatorCache{
cache: make([]*FDOperator, 0, 1024),
freelist: make([]int32, 0, 1024),
}
}
type operatorCache struct {
first *FDOperator
cache []*FDOperator
locked int32
// freelist store the freeable operator
// to reduce GC pressure, we only store op index here
freelocked int32
freelist []int32
}
func (c *operatorCache) alloc() *FDOperator {
lock(&c.locked)
if c.first == nil {
const opSize = unsafe.Sizeof(FDOperator{})
n := block4k / opSize
if n == 0 {
n = 1
}
index := int32(len(c.cache))
for i := uintptr(0); i < n; i++ {
pd := &FDOperator{index: index}
c.cache = append(c.cache, pd)
pd.next = c.first
c.first = pd
index++
}
}
op := c.first
c.first = op.next
unlock(&c.locked)
return op
}
// freeable mark the operator that could be freed
// only poller could do the real free action
func (c *operatorCache) freeable(op *FDOperator) {
// reset all state
op.unused()
op.reset()
lock(&c.freelocked)
c.freelist = append(c.freelist, op.index)
unlock(&c.freelocked)
}
func (c *operatorCache) free() {
lock(&c.freelocked)
defer unlock(&c.freelocked)
if len(c.freelist) == 0 {
return
}
lock(&c.locked)
for _, idx := range c.freelist {
op := c.cache[idx]
op.next = c.first
c.first = op
}
c.freelist = c.freelist[:0]
unlock(&c.locked)
}
func lock(locked *int32) {
for !atomic.CompareAndSwapInt32(locked, 0, 1) {
runtime.Gosched()
}
}
func unlock(locked *int32) {
atomic.StoreInt32(locked, 0)
}
================================================
FILE: fd_operator_cache_test.go
================================================
// Copyright 2022 CloudWeGo Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//go:build !windows
// +build !windows
package netpoll
import (
"runtime"
"testing"
)
// go test -v -gcflags=-d=checkptr -run=TestPersistFDOperator
func TestPersistFDOperator(t *testing.T) {
opcache := newOperatorCache()
// init
size := 2048
ops := make([]*FDOperator, size)
for i := 0; i < size; i++ {
op := opcache.alloc()
op.FD = i
ops[i] = op
}
Equal(t, len(opcache.freelist), 0)
// gc
for i := 0; i < 4; i++ {
runtime.GC()
}
// check alloc
for i := range ops {
Equal(t, ops[i].FD, i)
opcache.freeable(ops[i])
Equal(t, len(opcache.freelist), i+1)
}
Equal(t, len(opcache.freelist), size)
opcache.free()
Equal(t, len(opcache.freelist), 0)
Assert(t, len(opcache.cache) >= size)
}
func BenchmarkPersistFDOperator1(b *testing.B) {
b.ReportAllocs()
b.ResetTimer()
opcache := newOperatorCache()
for i := 0; i < b.N; i++ {
op := opcache.alloc()
opcache.freeable(op)
opcache.free()
}
}
func BenchmarkPersistFDOperator2(b *testing.B) {
// benchmark
b.ReportAllocs()
b.SetParallelism(128)
b.ResetTimer()
opcache := newOperatorCache()
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
op := opcache.alloc()
opcache.freeable(op)
opcache.free()
}
})
}
================================================
FILE: go.mod
================================================
module github.com/cloudwego/netpoll
go 1.15
require (
github.com/bytedance/gopkg v0.1.1
github.com/cloudwego/gopkg v0.1.4
golang.org/x/sys v0.19.0
)
================================================
FILE: go.sum
================================================
github.com/bytedance/gopkg v0.1.1 h1:3azzgSkiaw79u24a+w9arfH8OfnQQ4MHUt9lJFREEaE=
github.com/bytedance/gopkg v0.1.1/go.mod h1:576VvJ+eJgyCzdjS+c4+77QF3p7ubbtiKARP3TxducM=
github.com/cloudwego/gopkg v0.1.4 h1:EoQiCG4sTonTPHxOGE0VlQs+sQR+Hsi2uN0qqwu8O50=
github.com/cloudwego/gopkg v0.1.4/go.mod h1:FQuXsRWRsSqJLsMVd5SYzp8/Z1y5gXKnVvRrWUOsCMI=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
golang.org/x/crypto v0.22.0/go.mod h1:vr6Su+7cTlO45qkww3VDJlzDn0ctJvRgYbC2NvXHt+M=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44=
golang.org/x/net v0.24.0/go.mod h1:2Q7sJY5mzlzWjKtYUEXSlBWCdyaioyXzRB2RtU8KVE8=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.8.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.19.0 h1:q5f1RH2jigJ1MoAWp2KTp3gm5zAGFUTarQZ5U386+4o=
golang.org/x/sys v0.19.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.8.0/go.mod h1:xPskH00ivmX89bAKVGSKKtLOWNx2+17Eiy94tnKShWo=
golang.org/x/term v0.17.0/go.mod h1:lLRBjIVuehSbZlaOtGMbcMncT+aqLLLmKrsjNrUguwk=
golang.org/x/term v0.19.0/go.mod h1:2CuTdWZ7KHSQwUzKva0cbMg6q2DMI3Mmxp+gKJbskEk=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
================================================
FILE: internal/runner/runner.go
================================================
/*
* Copyright 2025 CloudWeGo Authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package runner
import (
"context"
"os"
"strconv"
bgopool "github.com/bytedance/gopkg/util/gopool"
cgopool "github.com/cloudwego/gopkg/concurrency/gopool"
)
// RunTask runs the `f` in background, and `ctx` is optional.
// `ctx` is used to pass to underlying implementation
var RunTask func(ctx context.Context, f func())
func goRunTask(ctx context.Context, f func()) {
go f()
}
func init() {
// netpoll uses github.com/bytedance/gopkg/util/gopool by default
// if the env is set, change it to cloudwego/gopkg
// for most users, using the 'go' keyword directly is more suitable.
if yes, _ := strconv.ParseBool(os.Getenv("USE_CLOUDWEGO_GOPOOL")); yes {
RunTask = cgopool.CtxGo
} else {
RunTask = bgopool.CtxGo
}
}
// UseGoRunTask updates RunTask with goRunTask which creates
// a new goroutine for the given func, basically `go f()`
func UseGoRunTask() {
RunTask = goRunTask
}
// SetPanicHandler sets the panic handler for the global pool.
func SetPanicHandler(f func(context.Context, interface{})) {
bgopool.SetPanicHandler(f)
cgopool.SetPanicHandler(f)
}
================================================
FILE: internal/runner/runner_test.go
================================================
/*
* Copyright 2025 CloudWeGo Authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package runner
import (
"context"
"sync"
"testing"
)
func TestRunTask(t *testing.T) {
var wg sync.WaitGroup
wg.Add(2)
ctx := context.Background()
RunTask(ctx, func() {
wg.Done()
})
UseGoRunTask()
RunTask(ctx, func() {
wg.Done()
})
wg.Wait()
}
================================================
FILE: lint.sh
================================================
#!/usr/bin/env bash
golangci-lint run
================================================
FILE: mux/mux_test.go
================================================
// Copyright 2022 CloudWeGo Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//go:build !windows
// +build !windows
package mux
import (
"testing"
)
func MustNil(t *testing.T, val interface{}) {
t.Helper()
Assert(t, val == nil, val)
if val != nil {
t.Fatal("assertion nil failed, val=", val)
}
}
func MustTrue(t *testing.T, cond bool) {
t.Helper()
if !cond {
t.Fatal("assertion true failed.")
}
}
func Equal(t *testing.T, got, expect interface{}) {
t.Helper()
if got != expect {
t.Fatalf("assertion equal failed, got=[%v], expect=[%v]", got, expect)
}
}
func Assert(t *testing.T, cond bool, val ...interface{}) {
t.Helper()
if !cond {
if len(val) > 0 {
val = append([]interface{}{"assertion failed:"}, val...)
t.Fatal(val...)
} else {
t.Fatal("assertion failed")
}
}
}
================================================
FILE: mux/shard_queue.go
================================================
// Copyright 2022 CloudWeGo Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package mux
import (
"fmt"
"runtime"
"sync"
"sync/atomic"
"github.com/cloudwego/netpoll"
"github.com/cloudwego/netpoll/internal/runner"
)
/* DOC:
* ShardQueue uses the netpoll's nocopy API to merge and send data.
* The Data Flush is passively triggered by ShardQueue.Add and does not require user operations.
* If there is an error in the data transmission, the connection will be closed.
*
* ShardQueue.Add: add the data to be sent.
* NewShardQueue: create a queue with netpoll.Connection.
* ShardSize: the recommended number of shards is 32.
*/
var ShardSize int
func init() {
ShardSize = runtime.GOMAXPROCS(0)
}
// NewShardQueue .
func NewShardQueue(size int, conn netpoll.Connection) (queue *ShardQueue) {
queue = &ShardQueue{
conn: conn,
size: int32(size),
getters: make([][]WriterGetter, size),
swap: make([]WriterGetter, 0, 64),
locks: make([]int32, size),
}
for i := range queue.getters {
queue.getters[i] = make([]WriterGetter, 0, 64)
}
queue.list = make([]int32, size)
return queue
}
// WriterGetter is used to get a netpoll.Writer.
type WriterGetter func() (buf netpoll.Writer, isNil bool)
// ShardQueue uses the netpoll's nocopy API to merge and send data.
// The Data Flush is passively triggered by ShardQueue.Add and does not require user operations.
// If there is an error in the data transmission, the connection will be closed.
// ShardQueue.Add: add the data to be sent.
type ShardQueue struct {
conn netpoll.Connection
idx, size int32
getters [][]WriterGetter // len(getters) = size
swap []WriterGetter // use for swap
locks []int32 // len(locks) = size
queueTrigger
}
const (
// queueTrigger state
active = 0
closing = 1
closed = 2
)
// here for trigger
type queueTrigger struct {
trigger int32
state int32 // 0: active, 1: closing, 2: closed
runNum int32
w, r int32 // ptr of list
list []int32 // record the triggered shard
listLock sync.Mutex // list total lock
}
// Add adds to q.getters[shard]
func (q *ShardQueue) Add(gts ...WriterGetter) {
if atomic.LoadInt32(&q.state) != active {
return
}
shard := atomic.AddInt32(&q.idx, 1) % q.size
q.lock(shard)
trigger := len(q.getters[shard]) == 0
q.getters[shard] = append(q.getters[shard], gts...)
q.unlock(shard)
if trigger {
q.triggering(shard)
}
}
func (q *ShardQueue) Close() error {
if !atomic.CompareAndSwapInt32(&q.state, active, closing) {
return fmt.Errorf("shardQueue has been closed")
}
// wait for all tasks finished
for atomic.LoadInt32(&q.state) != closed {
if atomic.LoadInt32(&q.trigger) == 0 {
atomic.StoreInt32(&q.state, closed)
return nil
}
runtime.Gosched()
}
return nil
}
// triggering shard.
func (q *ShardQueue) triggering(shard int32) {
q.listLock.Lock()
q.w = (q.w + 1) % q.size
q.list[q.w] = shard
q.listLock.Unlock()
if atomic.AddInt32(&q.trigger, 1) > 1 {
return
}
q.foreach()
}
// foreach swap r & w. It's not concurrency safe.
func (q *ShardQueue) foreach() {
if atomic.AddInt32(&q.runNum, 1) > 1 {
return
}
runner.RunTask(nil, func() {
var negNum int32 // is negative number of triggerNum
for triggerNum := atomic.LoadInt32(&q.trigger); triggerNum > 0; {
q.r = (q.r + 1) % q.size
shared := q.list[q.r]
// lock & swap
q.lock(shared)
tmp := q.getters[shared]
q.getters[shared] = q.swap[:0]
q.swap = tmp
q.unlock(shared)
// deal
q.deal(q.swap)
negNum--
if triggerNum+negNum == 0 {
triggerNum = atomic.AddInt32(&q.trigger, negNum)
negNum = 0
}
}
q.flush()
// quit & check again
atomic.StoreInt32(&q.runNum, 0)
if atomic.LoadInt32(&q.trigger) > 0 {
q.foreach()
return
}
// if state is closing, change it to closed
atomic.CompareAndSwapInt32(&q.state, closing, closed)
})
}
// deal is used to get deal of netpoll.Writer.
func (q *ShardQueue) deal(gts []WriterGetter) {
if !q.conn.IsActive() {
return
}
writer := q.conn.Writer()
for _, gt := range gts {
buf, isNil := gt()
if !isNil {
err := writer.Append(buf)
if err != nil {
q.conn.Close()
return
}
}
}
}
// flush is used to flush netpoll.Writer.
func (q *ShardQueue) flush() {
err := q.conn.Writer().Flush()
if err != nil {
q.conn.Close()
return
}
}
// lock shard.
func (q *ShardQueue) lock(shard int32) {
for !atomic.CompareAndSwapInt32(&q.locks[shard], 0, 1) {
runtime.Gosched()
}
}
// unlock shard.
func (q *ShardQueue) unlock(shard int32) {
atomic.StoreInt32(&q.locks[shard], 0)
}
================================================
FILE: mux/shard_queue_test.go
================================================
// Copyright 2022 CloudWeGo Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//go:build !windows
// +build !windows
package mux
import (
"net"
"testing"
"time"
"github.com/cloudwego/netpoll"
)
func TestShardQueue(t *testing.T) {
var svrConn net.Conn
accepted := make(chan struct{})
network, address := "tcp", "localhost:12345"
ln, err := net.Listen("tcp", address)
MustNil(t, err)
stop := make(chan int, 1)
defer close(stop)
go func() {
var err error
for {
select {
case <-stop:
err = ln.Close()
MustNil(t, err)
return
default:
}
svrConn, err = ln.Accept()
MustNil(t, err)
accepted <- struct{}{}
}
}()
conn, err := netpoll.DialConnection(network, address, time.Second)
MustNil(t, err)
<-accepted
// test
queue := NewShardQueue(4, conn)
count, pkgsize := 16, 11
for i := 0; i < count; i++ {
var getter WriterGetter = func() (buf netpoll.Writer, isNil bool) {
buf = netpoll.NewLinkBuffer(pkgsize)
buf.Malloc(pkgsize)
return buf, false
}
queue.Add(getter)
}
err = queue.Close()
MustNil(t, err)
total := count * pkgsize
recv := make([]byte, total)
rn, err := svrConn.Read(recv)
MustNil(t, err)
Equal(t, rn, total)
}
// TODO: need mock flush
func BenchmarkShardQueue(b *testing.B) {
b.Skip()
}
================================================
FILE: net_dialer.go
================================================
// Copyright 2022 CloudWeGo Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//go:build !windows
// +build !windows
package netpoll
import (
"context"
"net"
"time"
)
// DialConnection is a default implementation of Dialer.
func DialConnection(network, address string, timeout time.Duration) (connection Connection, err error) {
return defaultDialer.DialConnection(network, address, timeout)
}
// NewFDConnection create a Connection initialized by any fd
// It's useful for writing unit tests for functions that have args with the type of netpoll.Connection
// The typical usage is like:
//
// rfd, wfd := netpoll.GetSysFdPairs()
// rconn, _ = netpoll.NewFDConnection(rfd)
// wconn, _ = netpoll.NewFDConnection(wfd)
func NewFDConnection(fd int) (Connection, error) {
conn := new(connection)
err := conn.init(&netFD{fd: fd}, nil)
if err != nil {
return nil, err
}
return conn, nil
}
// NewDialer only support TCP and unix socket now.
func NewDialer() Dialer {
return &dialer{}
}
var defaultDialer = NewDialer()
type dialer struct{}
// DialTimeout implements Dialer.
func (d *dialer) DialTimeout(network, address string, timeout time.Duration) (net.Conn, error) {
return d.DialConnection(network, address, timeout)
}
// DialConnection implements Dialer.
func (d *dialer) DialConnection(network, address string, timeout time.Duration) (connection Connection, err error) {
ctx := context.Background()
if timeout > 0 {
subCtx, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
ctx = subCtx
}
switch network {
case "tcp", "tcp4", "tcp6":
return d.dialTCP(ctx, network, address)
case "unix", "unixgram", "unixpacket":
raddr := &UnixAddr{
UnixAddr: net.UnixAddr{Name: address, Net: network},
}
return DialUnix(network, nil, raddr)
default:
return nil, net.UnknownNetworkError(network)
}
}
func (d *dialer) dialTCP(ctx context.Context, network, address string) (connection *TCPConnection, err error) {
host, port, err := net.SplitHostPort(address)
if err != nil {
return nil, err
}
var portnum int
if portnum, err = net.DefaultResolver.LookupPort(ctx, network, port); err != nil {
return nil, err
}
var ipaddrs []net.IPAddr
// host maybe empty if address is :12345
if host == "" {
ipaddrs = []net.IPAddr{{}}
} else {
ipaddrs, err = net.DefaultResolver.LookupIPAddr(ctx, host)
if err != nil {
return nil, err
}
if len(ipaddrs) == 0 {
return nil, &net.DNSError{Err: "no such host", Name: host, IsNotFound: true}
}
}
var firstErr error // The error from the first address is most relevant.
tcpAddr := &TCPAddr{}
for _, ipaddr := range ipaddrs {
tcpAddr.IP = ipaddr.IP
tcpAddr.Port = portnum
tcpAddr.Zone = ipaddr.Zone
if ipaddr.IP != nil && ipaddr.IP.To4() == nil {
connection, err = DialTCP(ctx, "tcp6", nil, tcpAddr)
} else {
connection, err = DialTCP(ctx, "tcp", nil, tcpAddr)
}
if err == nil {
return connection, nil
}
select {
case <-ctx.Done(): // check timeout error
return nil, err
default:
}
if firstErr == nil {
firstErr = err
}
}
if firstErr == nil {
firstErr = &net.OpError{Op: "dial", Net: network, Source: nil, Addr: nil, Err: errMissingAddress}
}
return nil, firstErr
}
// sysDialer contains a Dial's parameters and configuration.
type sysDialer struct {
net.Dialer
network, address string
}
================================================
FILE: net_dialer_test.go
================================================
// Copyright 2022 CloudWeGo Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//go:build !windows
// +build !windows
package netpoll
import (
"context"
"fmt"
"runtime"
"strconv"
"strings"
"sync"
"syscall"
"testing"
"time"
)
func TestDialerTCP(t *testing.T) {
dialer := NewDialer()
address := getTestAddress()
conn, err := dialer.DialTimeout("tcp", address, time.Second)
MustTrue(t, err != nil)
MustTrue(t, conn.(*TCPConnection) == nil)
ln, err := CreateListener("tcp", address)
MustNil(t, err)
stop := make(chan int, 1)
defer close(stop)
go func() {
for {
select {
case <-stop:
err := ln.Close()
MustNil(t, err)
return
default:
}
conn, err := ln.Accept()
if conn == nil && err == nil {
continue
}
}
}()
conn, err = dialer.DialTimeout("tcp", address, time.Second)
MustNil(t, err)
MustTrue(t, strings.HasPrefix(conn.LocalAddr().String(), "127.0.0.1:"))
Equal(t, conn.RemoteAddr().String(), address)
}
func TestDialerUnix(t *testing.T) {
dialer := NewDialer()
conn, err := dialer.DialTimeout("unix", "tmp.sock", time.Second)
MustTrue(t, err != nil)
MustTrue(t, conn.(*UnixConnection) == nil)
ln, err := CreateListener("unix", "tmp.sock")
MustNil(t, err)
defer ln.Close()
stop := make(chan int, 1)
defer func() {
close(stop)
time.Sleep(time.Millisecond)
}()
go func() {
for {
select {
case <-stop:
err := ln.Close()
MustNil(t, err)
return
default:
}
conn, err := ln.Accept()
if conn == nil && err == nil {
continue
}
}
}()
conn, err = dialer.DialTimeout("unix", "tmp.sock", time.Second)
MustNil(t, err)
if runtime.GOOS == "linux" {
Equal(t, conn.LocalAddr().String(), "@")
} else {
Equal(t, conn.LocalAddr().String(), "")
}
Equal(t, conn.RemoteAddr().String(), "tmp.sock")
}
func TestDialerFdAlloc(t *testing.T) {
address := getTestAddress()
ln, err := CreateListener("tcp", address)
MustNil(t, err)
defer ln.Close()
el1, _ := NewEventLoop(func(ctx context.Context, connection Connection) error {
connection.Close()
return nil
})
go func() {
el1.Serve(ln)
}()
ctx1, cancel1 := context.WithTimeout(context.Background(), time.Second)
defer cancel1()
defer el1.Shutdown(ctx1)
for i := 0; i < 100; i++ {
conn, err := DialConnection("tcp", address, time.Second)
MustNil(t, err)
fd := conn.(*TCPConnection).fd
conn.Write([]byte("hello world"))
for conn.IsActive() {
runtime.Gosched()
}
time.Sleep(time.Millisecond)
syscall.SetNonblock(fd, true)
}
}
func TestFDClose(t *testing.T) {
address := getTestAddress()
ln, err := CreateListener("tcp", address)
MustNil(t, err)
defer ln.Close()
el1, _ := NewEventLoop(func(ctx context.Context, connection Connection) error {
connection.Close()
return nil
})
go func() {
el1.Serve(ln)
}()
ctx1, cancel1 := context.WithTimeout(context.Background(), time.Second)
defer cancel1()
defer el1.Shutdown(ctx1)
var fd int
var conn Connection
conn, err = DialConnection("tcp", address, time.Second)
MustNil(t, err)
fd = conn.(*TCPConnection).fd
syscall.SetNonblock(fd, true)
conn.Close()
conn, err = DialConnection("tcp", address, time.Second)
MustNil(t, err)
fd = conn.(*TCPConnection).fd
syscall.SetNonblock(fd, true)
time.Sleep(time.Second)
conn.Close()
}
// fd data package race test, use two servers and two dialers.
func TestDialerThenClose(t *testing.T) {
address1 := getTestAddress()
address2 := getTestAddress()
// server 1
ln1, _ := createTestListener("tcp", address1)
el1 := mockDialerEventLoop(1)
go func() {
el1.Serve(ln1)
}()
ctx1, cancel1 := context.WithTimeout(context.Background(), time.Second)
defer cancel1()
defer el1.Shutdown(ctx1)
// server 2
ln2, _ := createTestListener("tcp", address2)
el2 := mockDialerEventLoop(2)
go func() {
el2.Serve(ln2)
}()
ctx2, cancel2 := context.WithTimeout(context.Background(), time.Second)
defer cancel2()
defer el2.Shutdown(ctx2)
size := 20
var wg sync.WaitGroup
wg.Add(size)
for i := 0; i < size; i++ {
go func() {
defer wg.Done()
for i := 0; i < 50; i++ {
// send server 1
conn, err := DialConnection("tcp", address1, time.Second)
if err == nil {
mockDialerSend(1, &conn.(*TCPConnection).connection)
}
// send server 2
conn, err = DialConnection("tcp", address2, time.Second)
if err == nil {
mockDialerSend(2, &conn.(*TCPConnection).connection)
}
}
}()
}
wg.Wait()
}
func TestNewFDConnection(t *testing.T) {
r, w := GetSysFdPairs()
rconn, err := NewFDConnection(r)
MustNil(t, err)
wconn, err := NewFDConnection(w)
MustNil(t, err)
_, err = rconn.Writer().WriteString("hello")
MustNil(t, err)
err = rconn.Writer().Flush()
MustNil(t, err)
buf, err := wconn.Reader().Next(5)
MustNil(t, err)
Equal(t, string(buf), "hello")
}
func mockDialerEventLoop(idx int) EventLoop {
el, _ := NewEventLoop(func(ctx context.Context, conn Connection) (err error) {
defer func() {
if err != nil {
fmt.Printf("Error: server%d conn closed: %s", idx, err.Error())
conn.Close()
}
}()
operator := conn.(*connection)
fd := operator.fd
msg := make([]byte, 15)
n, err := operator.Read(msg)
if err != nil {
fmt.Printf("Error: conn[%d] server%d-read fail: %s", operator.fd, idx, err.Error())
return err
}
if n < 1 {
return nil
}
if string(msg[0]) != strconv.Itoa(idx) {
panic(fmt.Sprintf("msg[%s] != [%d-xxx]", msg, idx))
}
ss := strings.Split(string(msg[:n]), "-")
rfd, _ := strconv.Atoi(ss[1])
_, err = operator.Write([]byte(fmt.Sprintf("%d-%d", idx, fd)))
if err != nil {
fmt.Printf("Error: conn[%d] rfd[%d] server%d-write fail: %s", operator.fd, rfd, idx, err.Error())
return err
}
return nil
})
return el
}
func mockDialerSend(idx int, conn *connection) {
defer func() {
conn.Close()
}()
randID1 := []byte(fmt.Sprintf("%d-%d", idx, conn.fd))
_, err := conn.Write(randID1)
if err != nil {
fmt.Printf("Error: conn[%d] client%d write fail: %s", conn.fd, idx, err.Error())
}
msg := make([]byte, 15)
_, err = conn.Read(msg)
if err != nil {
fmt.Printf("Error: conn[%d] client%d Next fail: %s", conn.fd, idx, err.Error())
}
}
================================================
FILE: net_io.go
================================================
// Copyright 2023 CloudWeGo Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//go:build darwin || netbsd || freebsd || openbsd || dragonfly || linux
// +build darwin netbsd freebsd openbsd dragonfly linux
package netpoll
import "syscall"
// return value:
// - n: n == 0 but err == nil, retry syscall
// - err: if not nil, connection should be closed.
func ioread(fd int, bs [][]byte, ivs []syscall.Iovec) (n int, err error) {
n, err = readv(fd, bs, ivs)
if n == 0 && err == nil { // means EOF
return 0, Exception(ErrEOF, "")
}
if err == syscall.EINTR || err == syscall.EAGAIN {
return 0, nil
}
return n, err
}
// return value:
// - n: n == 0 but err == nil, retry syscall
// - err: if not nil, connection should be closed.
func iosend(fd int, bs [][]byte, ivs []syscall.Iovec, zerocopy bool) (n int, err error) {
n, err = sendmsg(fd, bs, ivs, zerocopy)
if err == syscall.EAGAIN {
return 0, nil
}
return n, err
}
================================================
FILE: net_listener.go
================================================
// Copyright 2022 CloudWeGo Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//go:build darwin || netbsd || freebsd || openbsd || dragonfly || linux
// +build darwin netbsd freebsd openbsd dragonfly linux
package netpoll
import (
"errors"
"net"
"os"
"syscall"
)
// CreateListener return a new Listener.
func CreateListener(network, addr string) (l Listener, err error) {
if network == "udp" || network == "udp4" || network == "udp6" {
return nil, Exception(ErrUnsupported, "UDP")
}
// tcp, tcp4, tcp6, unix
ln, err := net.Listen(network, addr)
if err != nil {
return nil, err
}
return ConvertListener(ln)
}
// ConvertListener converts net.Listener to Listener
func ConvertListener(l net.Listener) (nl Listener, err error) {
if tmp, ok := l.(Listener); ok {
return tmp, nil
}
ln := &listener{}
ln.ln = l
ln.addr = l.Addr()
err = ln.parseFD()
if err != nil {
return nil, err
}
return ln, syscall.SetNonblock(ln.fd, true)
}
var _ net.Listener = &listener{}
type listener struct {
fd int
addr net.Addr // listener's local addr
ln net.Listener // tcp|unix listener
file *os.File
}
// Accept implements Listener.
func (ln *listener) Accept() (net.Conn, error) {
fd, sa, err := syscall.Accept(ln.fd)
if err != nil {
/* https://man7.org/linux/man-pages/man2/accept.2.html
EAGAIN or EWOULDBLOCK
The socket is marked nonblocking and no connections are
present to be accepted. POSIX.1-2001 and POSIX.1-2008
allow either error to be returned for this case, and do
not require these constants to have the same value, so a
portable application should check for both possibilities.
*/
if err == syscall.EAGAIN || err == syscall.EWOULDBLOCK {
return nil, nil
}
return nil, err
}
nfd := &netFD{}
nfd.fd = fd
nfd.localAddr = ln.addr
nfd.network = ln.addr.Network()
nfd.remoteAddr = sockaddrToAddr(sa)
return nfd, nil
}
// Close implements Listener.
func (ln *listener) Close() error {
if ln.fd != 0 {
syscall.Close(ln.fd)
}
if ln.file != nil {
ln.file.Close()
}
if ln.ln != nil {
ln.ln.Close()
}
return nil
}
// Addr implements Listener.
func (ln *listener) Addr() net.Addr {
return ln.addr
}
// Fd implements Listener.
func (ln *listener) Fd() (fd int) {
return ln.fd
}
func (ln *listener) parseFD() (err error) {
switch netln := ln.ln.(type) {
case *net.TCPListener:
ln.file, err = netln.File()
case *net.UnixListener:
ln.file, err = netln.File()
default:
return errors.New("listener type can't support")
}
if err != nil {
return err
}
ln.fd = int(ln.file.Fd())
return nil
}
================================================
FILE: net_listener_test.go
================================================
// Copyright 2022 CloudWeGo Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//go:build darwin || netbsd || freebsd || openbsd || dragonfly || linux
// +build darwin netbsd freebsd openbsd dragonfly linux
package netpoll
import (
"context"
"net"
"sync/atomic"
"testing"
"time"
)
func TestListenerDialer(t *testing.T) {
network := "tcp"
addr := getTestAddress()
ln, err := CreateListener(network, addr)
MustNil(t, err)
defer ln.Close()
trigger := make(chan int)
msg := []byte("0123456789")
go func() {
for {
conn, err := ln.Accept()
if conn == nil && err == nil {
continue
}
if err != nil {
return
}
go func(conn net.Conn) {
<-trigger
buf := make([]byte, 10)
n, err := conn.Read(buf)
MustNil(t, err)
Equal(t, n, len(msg))
Equal(t, string(buf[:n]), string(msg))
n, err = conn.Write(buf)
MustNil(t, err)
Equal(t, n, len(msg))
}(conn)
}
}()
// trigger
var closed, read int32
dialer := NewDialer()
callback := func(connection Connection) error {
atomic.StoreInt32(&closed, 1)
return nil
}
onRequest := func(ctx context.Context, connection Connection) error {
atomic.StoreInt32(&read, 1)
err := connection.Close()
MustNil(t, err)
return err
}
for i := 0; i < 10; i++ {
conn, err := dialer.DialConnection(network, addr, time.Second)
if err != nil {
continue
}
conn.AddCloseCallback(callback)
conn.SetOnRequest(onRequest)
MustNil(t, err)
n, err := conn.Write(msg)
MustNil(t, err)
Equal(t, n, len(msg))
time.Sleep(10 * time.Millisecond)
trigger <- 1
time.Sleep(10 * time.Millisecond)
Equal(t, atomic.LoadInt32(&read), int32(1))
Equal(t, atomic.LoadInt32(&closed), int32(1))
}
}
func TestConvertListener(t *testing.T) {
network, address := "unix", "mock.test.sock"
ln, err := net.Listen(network, address)
if err != nil {
panic(err)
}
udsln, _ := ln.(*net.UnixListener)
// udsln.SetUnlinkOnClose(false)
nln, err := ConvertListener(udsln)
if err != nil {
panic(err)
}
err = nln.Close()
if err != nil {
panic(err)
}
}
================================================
FILE: net_netfd.go
================================================
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//
// This file may have been modified by CloudWeGo authors. (“CloudWeGo Modifications”).
// All CloudWeGo Modifications are Copyright 2022 CloudWeGo authors.
//go:build aix || darwin || dragonfly || freebsd || linux || nacl || netbsd || openbsd || solaris
// +build aix darwin dragonfly freebsd linux nacl netbsd openbsd solaris
package netpoll
import (
"context"
"errors"
"net"
"os"
"runtime"
"syscall"
"time"
)
// nonDeadline and noCancel are just zero values for
// readability with functions taking too many parameters.
var noDeadline = time.Time{}
type netFD struct {
// file descriptor
fd int
// When calling netFD.dial(), fd will be registered into poll in some scenarios, such as dialing tcp socket,
// but not in other scenarios, such as dialing unix socket.
// This leads to a different behavior in register poller at after, so use this field to mark it.
pd *pollDesc
// closed marks whether fd has expired
closed uint32
// Whether this is a streaming descriptor. Immutable.
isStream bool
// Whether a zero byte read indicates EOF. This is false for a
// message based socket connection.
zeroReadIsEOF bool
family int // AF_INET, AF_INET6, syscall.AF_UNIX
sotype int // syscall.SOCK_STREAM, syscall.SOCK_DGRAM, syscall.SOCK_RAW
isConnected bool // handshake completed or use of association with peer
network string // tcp, tcp4, tcp6, unix, unixgram, unixpacket
localAddr net.Addr
remoteAddr net.Addr
// for detaching conn from poller
detaching bool
}
func newNetFD(fd, family, sotype int, net string) *netFD {
ret := &netFD{}
ret.fd = fd
ret.network = net
ret.family = family
ret.sotype = sotype
ret.isStream = sotype == syscall.SOCK_STREAM
ret.zeroReadIsEOF = sotype != syscall.SOCK_DGRAM && sotype != syscall.SOCK_RAW
return ret
}
// if dial connection error, you need exec netFD.Close actively
func (c *netFD) dial(ctx context.Context, laddr, raddr sockaddr) (err error) {
var lsa syscall.Sockaddr
if laddr != nil {
if lsa, err = laddr.sockaddr(c.family); err != nil {
return err
} else if lsa != nil {
// bind local address
if err = syscall.Bind(c.fd, lsa); err != nil {
return os.NewSyscallError("bind", err)
}
}
}
var rsa syscall.Sockaddr // remote address from the user
var crsa syscall.Sockaddr // remote address we actually connected to
if raddr != nil {
if rsa, err = raddr.sockaddr(c.family); err != nil {
return err
}
}
// remote address we actually connected to
if crsa, err = c.connect(ctx, lsa, rsa); err != nil {
return err
}
c.isConnected = true
// Record the local and remote addresses from the actual socket.
// Get the local address by calling Getsockname.
// For the remote address, use
// 1) the one returned by the connect method, if any; or
// 2) the one from Getpeername, if it succeeds; or
// 3) the one passed to us as the raddr parameter.
lsa, _ = syscall.Getsockname(c.fd)
c.localAddr = sockaddrToAddr(lsa)
if crsa != nil {
c.remoteAddr = sockaddrToAddr(crsa)
} else if crsa, _ = syscall.Getpeername(c.fd); crsa != nil {
c.remoteAddr = sockaddrToAddr(crsa)
} else {
c.remoteAddr = sockaddrToAddr(rsa)
}
return nil
}
func (c *netFD) connect(ctx context.Context, la, ra syscall.Sockaddr) (rsa syscall.Sockaddr, retErr error) {
// Do not need to call c.writing here,
// because c is not yet accessible to user,
// so no concurrent operations are possible.
switch err := syscall.Connect(c.fd, ra); err {
case syscall.EINPROGRESS, syscall.EALREADY, syscall.EINTR:
case nil, syscall.EISCONN:
select {
case <-ctx.Done():
return nil, mapErr(ctx.Err())
default:
}
return nil, nil
case syscall.EINVAL:
// On Solaris we can see EINVAL if the socket has
// already been accepted and closed by the server.
// Treat this as a successful connection--writes to
// the socket will see EOF. For details and a test
// case in C see https://golang.org/issue/6828.
if runtime.GOOS == "solaris" {
return nil, nil
}
fallthrough
default:
return nil, os.NewSyscallError("connect", err)
}
c.pd = newPollDesc(c.fd)
defer func() {
// free operator to avoid leak
c.pd.operator.Free()
c.pd = nil
}()
for {
// Performing multiple connect system calls on a
// non-blocking socket under Unix variants does not
// necessarily result in earlier errors being
// returned. Instead, once runtime-integrated network
// poller tells us that the socket is ready, get the
// SO_ERROR socket option to see if the connection
// succeeded or failed. See issue 7474 for further
// details.
if err := c.pd.WaitWrite(ctx); err != nil {
return nil, err
}
nerr, err := syscall.GetsockoptInt(c.fd, syscall.SOL_SOCKET, syscall.SO_ERROR)
if err != nil {
return nil, os.NewSyscallError("getsockopt", err)
}
switch err := syscall.Errno(nerr); err {
case syscall.EINPROGRESS, syscall.EALREADY, syscall.EINTR:
case syscall.EISCONN:
return nil, nil
case syscall.Errno(0):
// The runtime poller can wake us up spuriously;
// see issues 14548 and 19289. Check that we are
// really connected; if not, wait again.
if rsa, err := syscall.Getpeername(c.fd); err == nil {
return rsa, nil
}
default:
return nil, os.NewSyscallError("connect", err)
}
}
}
// Various errors contained in OpError.
var (
errMissingAddress = errors.New("missing address")
errCanceled = errors.New("operation was canceled")
errIOTimeout = errors.New("i/o timeout")
)
// mapErr maps from the context errors to the historical internal net
// error values.
//
// TODO(bradfitz): get rid of this after adjusting tests and making
// context.DeadlineExceeded implement net.Error?
func mapErr(err error) error {
switch err {
case context.Canceled:
return errCanceled
case context.DeadlineExceeded:
return errIOTimeout
default:
return err
}
}
================================================
FILE: net_netfd_conn.go
================================================
// Copyright 2022 CloudWeGo Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//go:build darwin || netbsd || freebsd || openbsd || dragonfly || linux
// +build darwin netbsd freebsd openbsd dragonfly linux
package netpoll
import (
"net"
"strings"
"sync/atomic"
"syscall"
"time"
)
var _ Conn = &netFD{}
// Fd implements Conn.
func (c *netFD) Fd() (fd int) {
return c.fd
}
// Read implements Conn.
func (c *netFD) Read(b []byte) (n int, err error) {
n, err = syscall.Read(c.fd, b)
if err != nil {
if err == syscall.EAGAIN || err == syscall.EINTR {
return 0, nil
}
}
return n, err
}
// Write implements Conn.
func (c *netFD) Write(b []byte) (n int, err error) {
n, err = syscall.Write(c.fd, b)
if err != nil {
if err == syscall.EAGAIN {
return 0, nil
}
}
return n, err
}
// Close will be executed only once.
func (c *netFD) Close() (err error) {
if atomic.AddUint32(&c.closed, 1) != 1 {
return nil
}
if !c.detaching && c.fd > 2 {
err = syscall.Close(c.fd)
if err != nil {
logger.Printf("NETPOLL: netFD[%d] close error: %s", c.fd, err.Error())
}
}
return err
}
// LocalAddr implements Conn.
func (c *netFD) LocalAddr() (addr net.Addr) {
return c.localAddr
}
// RemoteAddr implements Conn.
func (c *netFD) RemoteAddr() (addr net.Addr) {
return c.remoteAddr
}
// SetKeepAlive implements Conn.
// TODO: only tcp conn is ok.
func (c *netFD) SetKeepAlive(second int) error {
if !strings.HasPrefix(c.network, "tcp") {
return nil
}
if second > 0 {
return SetKeepAlive(c.fd, second)
}
return nil
}
// SetDeadline implements Conn.
func (c *netFD) SetDeadline(t time.Time) error {
return Exception(ErrUnsupported, "SetDeadline")
}
// SetReadDeadline implements Conn.
func (c *netFD) SetReadDeadline(t time.Time) error {
return Exception(ErrUnsupported, "SetReadDeadline")
}
// SetWriteDeadline implements Conn.
func (c *netFD) SetWriteDeadline(t time.Time) error {
return Exception(ErrUnsupported, "SetWriteDeadline")
}
================================================
FILE: net_polldesc.go
================================================
// Copyright 2022 CloudWeGo Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//go:build !windows
// +build !windows
package netpoll
import (
"context"
)
func newPollDesc(fd int) *pollDesc {
pd := &pollDesc{}
poll := pollmanager.Pick()
pd.operator = poll.Alloc()
pd.operator.poll = poll
pd.operator.FD = fd
pd.operator.OnWrite = pd.onwrite
pd.operator.OnHup = pd.onhup
pd.writeTrigger = make(chan struct{})
pd.closeTrigger = make(chan struct{})
return pd
}
type pollDesc struct {
operator *FDOperator
// The write event is OneShot, then mark the writable to skip duplicate calling.
writeTrigger chan struct{}
closeTrigger chan struct{}
}
// WaitWrite .
func (pd *pollDesc) WaitWrite(ctx context.Context) (err error) {
if pd.operator.isUnused() {
// add ET|Write|Hup
if err = pd.operator.Control(PollWritable); err != nil {
logger.Printf("NETPOLL: pollDesc register operator failed: %v", err)
return err
}
}
select {
case <-pd.writeTrigger: // triggered by poller
case <-pd.closeTrigger: // triggered by poller
// no need to detach, since poller has done it in OnHup.
return Exception(ErrConnClosed, "by peer")
case <-ctx.Done(): // triggered by ctx
// deregister from poller, upper caller function will close fd
pd.detach()
return mapErr(ctx.Err())
}
// double check close trigger
select {
case <-pd.closeTrigger:
return Exception(ErrConnClosed, "by peer")
default:
return nil
}
}
func (pd *pollDesc) onwrite(p Poll) error {
select {
case <-pd.writeTrigger:
default:
pd.detach()
close(pd.writeTrigger)
}
return nil
}
func (pd *pollDesc) onhup(p Poll) error {
select {
case <-pd.closeTrigger:
default:
close(pd.closeTrigger)
}
return nil
}
func (pd *pollDesc) detach() {
if err := pd.operator.Control(PollDetach); err != nil {
logger.Printf("NETPOLL: pollDesc detach operator failed: %v", err)
}
}
================================================
FILE: net_polldesc_test.go
================================================
// Copyright 2022 CloudWeGo Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//go:build !windows
// +build !windows
package netpoll
import (
"testing"
"time"
)
func TestZeroTimer(t *testing.T) {
MustTrue(t, noDeadline.IsZero())
}
func TestRuntimePoll(t *testing.T) {
address := getTestAddress()
ln, err := CreateListener("tcp", address)
MustNil(t, err)
stop := make(chan int, 1)
defer close(stop)
go func() {
for {
select {
case <-stop:
err := ln.Close()
MustNil(t, err)
return
default:
}
conn, err := ln.Accept()
if conn == nil && err == nil {
continue
}
}
}()
for i := 0; i < 10; i++ {
conn, err := DialConnection("tcp", address, time.Second)
MustNil(t, err)
conn.Close()
}
}
================================================
FILE: net_sock.go
================================================
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//
// This file may have been modified by CloudWeGo authors. (“CloudWeGo Modifications”).
// All CloudWeGo Modifications are Copyright 2022 CloudWeGo authors.
//go:build !windows
// +build !windows
package netpoll
import (
"context"
"net"
"runtime"
"syscall"
)
// A sockaddr represents a TCP, IP or Unix network endpoint
// address that can be converted into a syscall.Sockaddr.
type sockaddr interface {
net.Addr
// family returns the platform-dependent address family
// identifier.
family() int
// isWildcard reports whether the address is a wildcard
// address.
isWildcard() bool
// sockaddr returns the address converted into a syscall
// sockaddr type that implements syscall.Sockaddr
// interface. It returns a nil interface when the address is nil.
sockaddr(family int) (syscall.Sockaddr, error)
// toLocal maps the zero address to a local system address (127.0.0.1 or ::1)
toLocal(net string) sockaddr
}
func internetSocket(ctx context.Context, net string, laddr, raddr sockaddr, sotype, proto int, mode string) (conn *netFD, err error) {
if (runtime.GOOS == "aix" || runtime.GOOS == "openbsd" || runtime.GOOS == "nacl") && raddr.isWildcard() {
raddr = raddr.toLocal(net)
}
family, ipv6only := favoriteAddrFamily(net, laddr, raddr)
return socket(ctx, net, family, sotype, proto, ipv6only, laddr, raddr)
}
// favoriteAddrFamily returns the appropriate address family for the
// given network, laddr, raddr and mode.
//
// If mode indicates "listen" and laddr is a wildcard, we assume that
// the user wants to make a passive-open connection with a wildcard
// address family, both AF_INET and AF_INET6, and a wildcard address
// like the following:
//
// - A listen for a wildcard communication domain, "tcp",
// with a wildcard address: If the platform supports
// both IPv6 and IPv4-mapped IPv6 communication capabilities,
// or does not support IPv4, we use a dual stack, AF_INET6 and
// IPV6_V6ONLY=0, wildcard address listen. The dual stack
// wildcard address listen may fall back to an IPv6-only,
// AF_INET6 and IPV6_V6ONLY=1, wildcard address listen.
// Otherwise we prefer an IPv4-only, AF_INET, wildcard address
// listen.
//
// - A listen for a wildcard communication domain, "tcp",
// with an IPv4 wildcard address: same as above.
//
// - A listen for a wildcard communication domain, "tcp",
// with an IPv6 wildcard address: same as above.
//
// - A listen for an IPv4 communication domain, "tcp4",
// with an IPv4 wildcard address: We use an IPv4-only, AF_INET,
// wildcard address listen.
//
// - A listen for an IPv6 communication domain, "tcp6",
// with an IPv6 wildcard address: We use an IPv6-only, AF_INET6
// and IPV6_V6ONLY=1, wildcard address listen.
//
// Otherwise guess: If the addresses are IPv4 then returns AF_INET,
// or else returns AF_INET6. It also returns a boolean value what
// designates IPV6_V6ONLY option.
//
// Note that the latest DragonFly BSD and OpenBSD kernels allow
// neither "net.inet6.ip6.v6only=1" change nor IPPROTO_IPV6 level
// IPV6_V6ONLY socket option setting.
func favoriteAddrFamily(network string, laddr, raddr sockaddr) (family int, ipv6only bool) {
switch network[len(network)-1] {
case '4':
return syscall.AF_INET, false
case '6':
return syscall.AF_INET6, true
}
if (laddr == nil || laddr.family() == syscall.AF_INET) &&
(raddr == nil || raddr.family() == syscall.AF_INET) {
return syscall.AF_INET, false
}
return syscall.AF_INET6, false
}
// socket returns a network file descriptor that is ready for
// asynchronous I/O using the network poller.
func socket(ctx context.Context, net string, family, sotype, proto int, ipv6only bool, laddr, raddr sockaddr) (netfd *netFD, err error) {
// syscall.Socket & set socket options
var fd int
fd, err = sysSocket(family, sotype, proto)
if err != nil {
return nil, err
}
err = setDefaultSockopts(fd, family, sotype, ipv6only)
if err != nil {
syscall.Close(fd)
return nil, err
}
netfd = newNetFD(fd, family, sotype, net)
err = netfd.dial(ctx, laddr, raddr)
if err != nil {
netfd.Close()
return nil, err
}
return netfd, nil
}
// sockaddrToAddr returns a go/net friendly address
func sockaddrToAddr(sa syscall.Sockaddr) net.Addr {
var a net.Addr
switch sa := sa.(type) {
case *syscall.SockaddrInet4:
a = &net.TCPAddr{
IP: sa.Addr[0:],
Port: sa.Port,
}
case *syscall.SockaddrInet6:
var zone string
if sa.ZoneId != 0 {
if ifi, err := net.InterfaceByIndex(int(sa.ZoneId)); err == nil {
zone = ifi.Name
}
}
// if zone == "" && sa.ZoneId != 0 {
// }
a = &net.TCPAddr{
IP: sa.Addr[0:],
Port: sa.Port,
Zone: zone,
}
case *syscall.SockaddrUnix:
a = &net.UnixAddr{Net: "unix", Name: sa.Name}
}
return a
}
================================================
FILE: net_tcpsock.go
================================================
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//
// This file may have been modified by CloudWeGo authors. (“CloudWeGo Modifications”).
// All CloudWeGo Modifications are Copyright 2022 CloudWeGo authors.
//go:build !windows
// +build !windows
package netpoll
import (
"context"
"net"
"os"
"syscall"
)
// TCPAddr represents the address of a TCP end point.
type TCPAddr struct {
net.TCPAddr
}
func (a *TCPAddr) isWildcard() bool {
if a == nil || a.IP == nil {
return true
}
return a.IP.IsUnspecified()
}
func (a *TCPAddr) opAddr() net.Addr {
if a == nil {
return nil
}
return a
}
func (a *TCPAddr) family() int {
if a == nil || len(a.IP) <= net.IPv4len {
return syscall.AF_INET
}
if a.IP.To4() != nil {
return syscall.AF_INET
}
return syscall.AF_INET6
}
func (a *TCPAddr) sockaddr(family int) (syscall.Sockaddr, error) {
if a == nil {
return nil, nil
}
return ipToSockaddr(family, a.IP, a.Port, a.Zone)
}
func (a *TCPAddr) toLocal(network string) sockaddr {
addr := &TCPAddr{}
addr.IP = loopbackIP(network)
addr.Port = a.Port
addr.Zone = a.Zone
return addr
}
func loopbackIP(network string) net.IP {
if network != "" && network[len(network)-1] == '6' {
return net.IPv6loopback
}
return net.IP{127, 0, 0, 1}
}
func ipToSockaddr(family int, ip net.IP, port int, zone string) (syscall.Sockaddr, error) {
switch family {
case syscall.AF_INET:
if len(ip) == 0 {
ip = net.IPv4zero
}
ip4 := ip.To4()
if ip4 == nil {
return nil, &net.AddrError{Err: "non-IPv4 address", Addr: ip.String()}
}
sa := &syscall.SockaddrInet4{Port: port}
copy(sa.Addr[:], ip4)
return sa, nil
case syscall.AF_INET6:
// In general, an IP wildcard address, which is either
// "0.0.0.0" or "::", means the entire IP addressing
// space. For some historical reason, it is used to
// specify "any available address" on some operations
// of IP node.
//
// When the IP node supports IPv4-mapped IPv6 address,
// we allow an listener to listen to the wildcard
// address of both IP addressing spaces by specifying
// IPv6 wildcard address.
if len(ip) == 0 || ip.Equal(net.IPv4zero) {
ip = net.IPv6zero
}
// We accept any IPv6 address including IPv4-mapped
// IPv6 address.
ip6 := ip.To16()
if ip6 == nil {
return nil, &net.AddrError{Err: "non-IPv6 address", Addr: ip.String()}
}
// TODO: sa := &syscall.SockaddrInet6{Port: port, ZoneId: uint32(zoneCache.index(zone))}
sa := &syscall.SockaddrInet6{Port: port}
copy(sa.Addr[:], ip6)
return sa, nil
}
return nil, &net.AddrError{Err: "invalid address family", Addr: ip.String()}
}
// ResolveTCPAddr returns an address of TCP end point.
//
// The network must be a TCP network name.
//
// If the host in the address parameter is not a literal IP address or
// the port is not a literal port number, ResolveTCPAddr resolves the
// address to an address of TCP end point.
// Otherwise, it parses the address as a pair of literal IP address
// and port number.
// The address parameter can use a host name, but this is not
// recommended, because it will return at most one of the host name's
// IP addresses.
//
// See func Dial for a description of the network and address
// parameters.
func ResolveTCPAddr(network, address string) (*TCPAddr, error) {
addr, err := net.ResolveTCPAddr(network, address)
if err != nil {
return nil, err
}
return &TCPAddr{*addr}, nil
}
// TCPConnection implements Connection.
type TCPConnection struct {
connection
}
// newTCPConnection wraps *TCPConnection.
func newTCPConnection(conn Conn) (connection *TCPConnection, err error) {
connection = &TCPConnection{}
err = connection.init(conn, nil)
if err != nil {
return nil, err
}
return connection, nil
}
// DialTCP acts like Dial for TCP networks.
//
// The network must be a TCP network name; see func Dial for details.
//
// If laddr is nil, a local address is automatically chosen.
// If the IP field of raddr is nil or an unspecified IP address, the
// local system is assumed.
func DialTCP(ctx context.Context, network string, laddr, raddr *TCPAddr) (*TCPConnection, error) {
switch network {
case "tcp", "tcp4", "tcp6":
default:
return nil, &net.OpError{Op: "dial", Net: network, Source: laddr.opAddr(), Addr: raddr.opAddr(), Err: net.UnknownNetworkError(network)}
}
if raddr == nil {
return nil, &net.OpError{Op: "dial", Net: network, Source: laddr.opAddr(), Addr: nil, Err: errMissingAddress}
}
if ctx == nil {
ctx = context.Background()
}
sd := &sysDialer{network: network, address: raddr.String()}
c, err := sd.dialTCP(ctx, laddr, raddr)
if err != nil {
return nil, &net.OpError{Op: "dial", Net: network, Source: laddr.opAddr(), Addr: raddr.opAddr(), Err: err}
}
return c, nil
}
func (sd *sysDialer) dialTCP(ctx context.Context, laddr, raddr *TCPAddr) (*TCPConnection, error) {
conn, err := internetSocket(ctx, sd.network, laddr, raddr, syscall.SOCK_STREAM, 0, "dial")
// TCP has a rarely used mechanism called a 'simultaneous connection' in
// which Dial("tcp", addr1, addr2) run on the machine at addr1 can
// connect to a simultaneous Dial("tcp", addr2, addr1) run on the machine
// at addr2, without either machine executing Listen. If laddr == nil,
// it means we want the kernel to pick an appropriate originating local
// address. Some Linux kernels cycle blindly through a fixed range of
// local ports, regardless of destination port. If a kernel happens to
// pick local port 50001 as the source for a Dial("tcp", "", "localhost:50001"),
// then the Dial will succeed, having simultaneously connected to itself.
// This can only happen when we are letting the kernel pick a port (laddr == nil)
// and when there is no listener for the destination address.
// It's hard to argue this is anything other than a kernel bug. If we
// see this happen, rather than expose the buggy effect to users, we
// close the conn and try again. If it happens twice more, we relent and
// use the result. See also:
// https://golang.org/issue/2690
// https://stackoverflow.com/questions/4949858/
//
// The opposite can also happen: if we ask the kernel to pick an appropriate
// originating local address, sometimes it picks one that is already in use.
// So if the error is EADDRNOTAVAIL, we have to try again too, just for
// a different reason.
//
// The kernel socket code is no doubt enjoying watching us squirm.
for i := 0; i < 2 && (laddr == nil || laddr.Port == 0) && (selfConnect(conn, err) || spuriousENOTAVAIL(err)); i++ {
if err == nil {
conn.Close()
}
conn, err = internetSocket(ctx, sd.network, laddr, raddr, syscall.SOCK_STREAM, 0, "dial")
}
if err != nil {
return nil, err
}
return newTCPConnection(conn)
}
func selfConnect(conn *netFD, err error) bool {
// If the connect failed, we clearly didn't connect to ourselves.
if err != nil {
return false
}
// The socket constructor can return an conn with raddr nil under certain
// unknown conditions. The errors in the calls there to Getpeername
// are discarded, but we can't catch the problem there because those
// calls are sometimes legally erroneous with a "socket not connected".
// Since this code (selfConnect) is already trying to work around
// a problem, we make sure if this happens we recognize trouble and
// ask the DialTCP routine to try again.
// TODO: try to understand what's really going on.
if conn.localAddr == nil || conn.remoteAddr == nil {
return true
}
l := conn.localAddr.(*net.TCPAddr)
r := conn.remoteAddr.(*net.TCPAddr)
return l.Port == r.Port && l.IP.Equal(r.IP)
}
func spuriousENOTAVAIL(err error) bool {
if op, ok := err.(*net.OpError); ok {
err = op.Err
}
if sys, ok := err.(*os.SyscallError); ok {
err = sys.Err
}
return err == syscall.EADDRNOTAVAIL
}
================================================
FILE: net_unixsock.go
================================================
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//
// This file may have been modified by CloudWeGo authors. (“CloudWeGo Modifications”).
// All CloudWeGo Modifications are Copyright 2022 CloudWeGo authors.
//go:build !windows
// +build !windows
package netpoll
import (
"context"
"errors"
"net"
"syscall"
)
// BUG(mikio): On JS, NaCl and Plan 9, methods and functions related
// to UnixConn and UnixListener are not implemented.
// BUG(mikio): On Windows, methods and functions related to UnixConn
// and UnixListener don't work for "unixgram" and "unixpacket".
// UnixAddr represents the address of a Unix domain socket end point.
type UnixAddr struct {
net.UnixAddr
}
func (a *UnixAddr) isWildcard() bool {
return a == nil || a.Name == ""
}
func (a *UnixAddr) opAddr() net.Addr {
if a == nil {
return nil
}
return a
}
func (a *UnixAddr) family() int {
return syscall.AF_UNIX
}
func (a *UnixAddr) sockaddr(family int) (syscall.Sockaddr, error) {
if a == nil {
return nil, nil
}
return &syscall.SockaddrUnix{Name: a.Name}, nil
}
func (a *UnixAddr) toLocal(net string) sockaddr {
return a
}
// ResolveUnixAddr returns an address of Unix domain socket end point.
//
// The network must be a Unix network name.
//
// See func Dial for a description of the network and address
// parameters.
func ResolveUnixAddr(network, address string) (*UnixAddr, error) {
ad
gitextract_ncyzmgn6/ ├── .github/ │ ├── CODEOWNERS │ ├── ISSUE_TEMPLATE/ │ │ ├── bug_report.md │ │ └── feature_request.md │ ├── PULL_REQUEST_TEMPLATE.md │ └── workflows/ │ └── pr-check.yml ├── .gitignore ├── .golangci.yaml ├── .licenserc.yaml ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── CREDITS ├── LICENSE ├── NOTICE ├── README.md ├── README_CN.md ├── _typos.toml ├── connection.go ├── connection_errors.go ├── connection_errors_test.go ├── connection_impl.go ├── connection_lock.go ├── connection_onevent.go ├── connection_reactor.go ├── connection_test.go ├── docs/ │ ├── guide/ │ │ ├── guide_cn.md │ │ └── guide_en.md │ └── reference/ │ ├── design_cn.md │ ├── design_en.md │ └── explain.md ├── eventloop.go ├── fd_operator.go ├── fd_operator_cache.go ├── fd_operator_cache_test.go ├── go.mod ├── go.sum ├── internal/ │ └── runner/ │ ├── runner.go │ └── runner_test.go ├── lint.sh ├── mux/ │ ├── mux_test.go │ ├── shard_queue.go │ └── shard_queue_test.go ├── net_dialer.go ├── net_dialer_test.go ├── net_io.go ├── net_listener.go ├── net_listener_test.go ├── net_netfd.go ├── net_netfd_conn.go ├── net_polldesc.go ├── net_polldesc_test.go ├── net_sock.go ├── net_tcpsock.go ├── net_unixsock.go ├── netpoll_config.go ├── netpoll_options.go ├── netpoll_server.go ├── netpoll_unix.go ├── netpoll_unix_test.go ├── netpoll_windows.go ├── nocopy.go ├── nocopy_linkbuffer.go ├── nocopy_linkbuffer_norace.go ├── nocopy_linkbuffer_race.go ├── nocopy_linkbuffer_test.go ├── nocopy_readwriter.go ├── nocopy_readwriter_test.go ├── poll.go ├── poll_default.go ├── poll_default_bsd.go ├── poll_default_bsd_norace.go ├── poll_default_bsd_race.go ├── poll_default_linux.go ├── poll_default_linux_norace.go ├── poll_default_linux_race.go ├── poll_default_linux_test.go ├── poll_loadbalance.go ├── poll_manager.go ├── poll_manager_test.go ├── poll_test.go ├── sys_epoll_linux.go ├── sys_epoll_linux_arm64.go ├── sys_epoll_linux_loong64.go ├── sys_exec.go ├── sys_exec_test.go ├── sys_keepalive_darwin.go ├── sys_keepalive_openbsd.go ├── sys_keepalive_unix.go ├── sys_sendmsg_bsd.go ├── sys_sendmsg_linux.go ├── sys_sockopt_bsd.go ├── sys_sockopt_linux.go └── test_conns.sh
SYMBOL INDEX (617 symbols across 66 files)
FILE: connection.go
type CloseCallback (line 24) | type CloseCallback
type Connection (line 29) | type Connection interface
type Conn (line 75) | type Conn interface
type Listener (line 83) | type Listener interface
type Dialer (line 93) | type Dialer interface
FILE: connection_errors.go
constant ErrConnClosed (line 26) | ErrConnClosed = syscall.Errno(0x101)
constant ErrReadTimeout (line 28) | ErrReadTimeout = syscall.Errno(0x102)
constant ErrDialTimeout (line 30) | ErrDialTimeout = syscall.Errno(0x103)
constant ErrDialNoDeadline (line 32) | ErrDialNoDeadline = syscall.Errno(0x104)
constant ErrUnsupported (line 34) | ErrUnsupported = syscall.Errno(0x105)
constant ErrEOF (line 36) | ErrEOF = syscall.Errno(0x106)
constant ErrWriteTimeout (line 38) | ErrWriteTimeout = syscall.Errno(0x107)
constant ErrConcurrentAccess (line 40) | ErrConcurrentAccess = syscall.Errno(0x108)
constant ErrnoMask (line 43) | ErrnoMask = 0xFF
function Exception (line 46) | func Exception(err error, suffix string) error {
type exception (line 59) | type exception struct
method Error (line 64) | func (e *exception) Error() string {
method Is (line 78) | func (e *exception) Is(target error) bool {
method Unwrap (line 92) | func (e *exception) Unwrap() error {
method Timeout (line 96) | func (e *exception) Timeout() bool {
method Temporary (line 104) | func (e *exception) Temporary() bool {
FILE: connection_errors_test.go
function TestErrno (line 26) | func TestErrno(t *testing.T) {
FILE: connection_impl.go
constant connStateNone (line 30) | connStateNone = 0
constant connStateConnected (line 31) | connStateConnected = 1
constant connStateDisconnected (line 32) | connStateDisconnected = 2
type connection (line 36) | type connection struct
method Reader (line 65) | func (c *connection) Reader() Reader {
method Writer (line 70) | func (c *connection) Writer() Writer {
method IsActive (line 75) | func (c *connection) IsActive() bool {
method SetIdleTimeout (line 80) | func (c *connection) SetIdleTimeout(timeout time.Duration) error {
method SetReadTimeout (line 88) | func (c *connection) SetReadTimeout(timeout time.Duration) error {
method SetWriteTimeout (line 97) | func (c *connection) SetWriteTimeout(timeout time.Duration) error {
method SetDeadline (line 106) | func (c *connection) SetDeadline(t time.Time) error {
method SetReadDeadline (line 117) | func (c *connection) SetReadDeadline(t time.Time) error {
method SetWriteDeadline (line 127) | func (c *connection) SetWriteDeadline(t time.Time) error {
method Next (line 139) | func (c *connection) Next(n int) (p []byte, err error) {
method Peek (line 147) | func (c *connection) Peek(n int) (buf []byte, err error) {
method Skip (line 155) | func (c *connection) Skip(n int) (err error) {
method Release (line 163) | func (c *connection) Release() (err error) {
method Slice (line 186) | func (c *connection) Slice(n int) (r Reader, err error) {
method Len (line 194) | func (c *connection) Len() (length int) {
method Until (line 199) | func (c *connection) Until(delim byte) (line []byte, err error) {
method ReadString (line 219) | func (c *connection) ReadString(n int) (s string, err error) {
method ReadBinary (line 227) | func (c *connection) ReadBinary(n int) (p []byte, err error) {
method ReadByte (line 235) | func (c *connection) ReadByte() (b byte, err error) {
method Malloc (line 245) | func (c *connection) Malloc(n int) (buf []byte, err error) {
method MallocLen (line 253) | func (c *connection) MallocLen() (length int) {
method Flush (line 263) | func (c *connection) Flush() error {
method MallocAck (line 278) | func (c *connection) MallocAck(n int) (err error) {
method Append (line 286) | func (c *connection) Append(w Writer) (err error) {
method WriteString (line 294) | func (c *connection) WriteString(s string) (n int, err error) {
method WriteBinary (line 302) | func (c *connection) WriteBinary(b []byte) (n int, err error) {
method WriteDirect (line 310) | func (c *connection) WriteDirect(p []byte, remainCap int) (err error) {
method WriteByte (line 318) | func (c *connection) WriteByte(b byte) (err error) {
method Read (line 328) | func (c *connection) Read(p []byte) (n int, err error) {
method Write (line 339) | func (c *connection) Write(p []byte) (n int, err error) {
method Close (line 357) | func (c *connection) Close() error {
method Detach (line 362) | func (c *connection) Detach() error {
method init (line 379) | func (c *connection) init(conn Conn, opts *options) (err error) {
method initNetFD (line 403) | func (c *connection) initNetFD(conn Conn) {
method initFDOperator (line 415) | func (c *connection) initFDOperator() {
method initFinalizer (line 425) | func (c *connection) initFinalizer() {
method triggerRead (line 437) | func (c *connection) triggerRead(err error) {
method triggerWrite (line 444) | func (c *connection) triggerWrite(err error) {
method waitRead (line 452) | func (c *connection) waitRead(n int) (err error) {
method waitReadWithTimeout (line 485) | func (c *connection) waitReadWithTimeout(n int, timeout time.Duration)...
method flush (line 527) | func (c *connection) flush() error {
method waitFlush (line 555) | func (c *connection) waitFlush() (err error) {
method getState (line 594) | func (c *connection) getState() connState {
method setState (line 598) | func (c *connection) setState(newState connState) {
method changeState (line 602) | func (c *connection) changeState(from, to connState) bool {
FILE: connection_lock.go
constant none (line 25) | none who = iota
constant user (line 26) | user
constant poller (line 27) | poller
type key (line 30) | type key
constant closing (line 47) | closing key = iota
constant connecting (line 48) | connecting
constant processing (line 49) | processing
constant flushing (line 50) | flushing
constant total (line 52) | total
type locker (line 55) | type locker struct
method closeBy (line 61) | func (l *locker) closeBy(w who) (success bool) {
method isCloseBy (line 65) | func (l *locker) isCloseBy(w who) (yes bool) {
method status (line 69) | func (l *locker) status(k key) int32 {
method force (line 73) | func (l *locker) force(k key, v int32) {
method lock (line 77) | func (l *locker) lock(k key) (success bool) {
method unlock (line 81) | func (l *locker) unlock(k key) {
method stop (line 85) | func (l *locker) stop(k key) {
method isUnlock (line 91) | func (l *locker) isUnlock(k key) bool {
FILE: connection_onevent.go
type gracefulExit (line 29) | type gracefulExit interface
type onEvent (line 37) | type onEvent struct
type callbackNode (line 45) | type callbackNode struct
method SetOnConnect (line 51) | func (c *connection) SetOnConnect(onConnect OnConnect) error {
method SetOnDisconnect (line 59) | func (c *connection) SetOnDisconnect(onDisconnect OnDisconnect) error {
method SetOnRequest (line 67) | func (c *connection) SetOnRequest(onRequest OnRequest) error {
method AddCloseCallback (line 80) | func (c *connection) AddCloseCallback(callback CloseCallback) error {
method onPrepare (line 95) | func (c *connection) onPrepare(opts *options) (err error) {
method onConnect (line 121) | func (c *connection) onConnect() {
method onDisconnect (line 136) | func (c *connection) onDisconnect() {
method onRequest (line 163) | func (c *connection) onRequest() (needTrigger bool) {
method onProcess (line 180) | func (c *connection) onProcess(onConnect OnConnect, onRequest OnRequest)...
method closeCallback (line 269) | func (c *connection) closeCallback(needLock, needDetach bool) (err error) {
method register (line 290) | func (c *connection) register() (err error) {
method isIdle (line 301) | func (c *connection) isIdle() (yes bool) {
FILE: connection_reactor.go
method onHup (line 27) | func (c *connection) onHup(p Poll) error {
method onClose (line 51) | func (c *connection) onClose() error {
method closeBuffer (line 71) | func (c *connection) closeBuffer() {
method inputs (line 86) | func (c *connection) inputs(vs [][]byte) (rs [][]byte) {
method inputAck (line 92) | func (c *connection) inputAck(n int) (err error) {
method outputs (line 122) | func (c *connection) outputs(vs [][]byte) (rs [][]byte, _ bool) {
method outputAck (line 132) | func (c *connection) outputAck(n int) (err error) {
method rw2r (line 144) | func (c *connection) rw2r() {
FILE: connection_test.go
function BenchmarkConnectionIO (line 35) | func BenchmarkConnectionIO(b *testing.B) {
function TestConnectionWrite (line 59) | func TestConnectionWrite(t *testing.T) {
function TestConnectionLargeWrite (line 91) | func TestConnectionLargeWrite(t *testing.T) {
function TestConnectionRead (line 125) | func TestConnectionRead(t *testing.T) {
function TestConnectionIOReader (line 158) | func TestConnectionIOReader(t *testing.T) {
function TestConnectionReadAfterClosed (line 197) | func TestConnectionReadAfterClosed(t *testing.T) {
function TestConnectionWaitReadHalfPacket (line 217) | func TestConnectionWaitReadHalfPacket(t *testing.T) {
function TestReadTimer (line 251) | func TestReadTimer(t *testing.T) {
function TestReadTrigger (line 258) | func TestReadTrigger(t *testing.T) {
function writeAll (line 267) | func writeAll(fd int, buf []byte) error {
function createTestTCPListener (line 278) | func createTestTCPListener(t *testing.T) net.Listener {
function TestLargeBufferWrite (line 286) | func TestLargeBufferWrite(t *testing.T) {
function TestConnectionTimeout (line 344) | func TestConnectionTimeout(t *testing.T) {
function TestConnectionLargeMemory (line 523) | func TestConnectionLargeMemory(t *testing.T) {
function TestSetTCPNoDelay (line 558) | func TestSetTCPNoDelay(t *testing.T) {
function TestConnectionUntil (line 572) | func TestConnectionUntil(t *testing.T) {
function TestBookSizeLargerThanMaxSize (line 603) | func TestBookSizeLargerThanMaxSize(t *testing.T) {
function TestConnDetach (line 649) | func TestConnDetach(t *testing.T) {
function TestParallelShortConnection (line 709) | func TestParallelShortConnection(t *testing.T) {
function TestConnectionServerClose (line 761) | func TestConnectionServerClose(t *testing.T) {
function TestWriterAfterClose (line 861) | func TestWriterAfterClose(t *testing.T) {
function TestConnectionDailTimeoutAndClose (line 900) | func TestConnectionDailTimeoutAndClose(t *testing.T) {
FILE: eventloop.go
type EventLoop (line 23) | type EventLoop interface
type OnPrepare (line 62) | type OnPrepare
type OnConnect (line 79) | type OnConnect
type OnDisconnect (line 84) | type OnDisconnect
type OnRequest (line 114) | type OnRequest
FILE: fd_operator.go
type FDOperator (line 23) | type FDOperator struct
method Control (line 55) | func (op *FDOperator) Control(event PollEvent) error {
method Free (line 62) | func (op *FDOperator) Free() {
method do (line 66) | func (op *FDOperator) do() (can bool) {
method done (line 70) | func (op *FDOperator) done() {
method inuse (line 74) | func (op *FDOperator) inuse() {
method unused (line 83) | func (op *FDOperator) unused() {
method isUnused (line 92) | func (op *FDOperator) isUnused() bool {
method reset (line 96) | func (op *FDOperator) reset() {
FILE: fd_operator_cache.go
function newOperatorCache (line 23) | func newOperatorCache() *operatorCache {
type operatorCache (line 30) | type operatorCache struct
method alloc (line 40) | func (c *operatorCache) alloc() *FDOperator {
method freeable (line 65) | func (c *operatorCache) freeable(op *FDOperator) {
method free (line 74) | func (c *operatorCache) free() {
function lock (line 91) | func lock(locked *int32) {
function unlock (line 97) | func unlock(locked *int32) {
FILE: fd_operator_cache_test.go
function TestPersistFDOperator (line 26) | func TestPersistFDOperator(t *testing.T) {
function BenchmarkPersistFDOperator1 (line 53) | func BenchmarkPersistFDOperator1(b *testing.B) {
function BenchmarkPersistFDOperator2 (line 64) | func BenchmarkPersistFDOperator2(b *testing.B) {
FILE: internal/runner/runner.go
function goRunTask (line 32) | func goRunTask(ctx context.Context, f func()) {
function init (line 36) | func init() {
function UseGoRunTask (line 49) | func UseGoRunTask() {
function SetPanicHandler (line 54) | func SetPanicHandler(f func(context.Context, interface{})) {
FILE: internal/runner/runner_test.go
function TestRunTask (line 25) | func TestRunTask(t *testing.T) {
FILE: mux/mux_test.go
function MustNil (line 24) | func MustNil(t *testing.T, val interface{}) {
function MustTrue (line 32) | func MustTrue(t *testing.T, cond bool) {
function Equal (line 39) | func Equal(t *testing.T, got, expect interface{}) {
function Assert (line 46) | func Assert(t *testing.T, cond bool, val ...interface{}) {
FILE: mux/shard_queue.go
function init (line 38) | func init() {
function NewShardQueue (line 43) | func NewShardQueue(size int, conn netpoll.Connection) (queue *ShardQueue) {
type WriterGetter (line 59) | type WriterGetter
type ShardQueue (line 65) | type ShardQueue struct
method Add (line 92) | func (q *ShardQueue) Add(gts ...WriterGetter) {
method Close (line 106) | func (q *ShardQueue) Close() error {
method triggering (line 122) | func (q *ShardQueue) triggering(shard int32) {
method foreach (line 135) | func (q *ShardQueue) foreach() {
method deal (line 174) | func (q *ShardQueue) deal(gts []WriterGetter) {
method flush (line 192) | func (q *ShardQueue) flush() {
method lock (line 201) | func (q *ShardQueue) lock(shard int32) {
method unlock (line 208) | func (q *ShardQueue) unlock(shard int32) {
constant active (line 76) | active = 0
constant closing (line 77) | closing = 1
constant closed (line 78) | closed = 2
type queueTrigger (line 82) | type queueTrigger struct
FILE: mux/shard_queue_test.go
function TestShardQueue (line 28) | func TestShardQueue(t *testing.T) {
function BenchmarkShardQueue (line 79) | func BenchmarkShardQueue(b *testing.B) {
FILE: net_dialer.go
function DialConnection (line 27) | func DialConnection(network, address string, timeout time.Duration) (con...
function NewFDConnection (line 38) | func NewFDConnection(fd int) (Connection, error) {
function NewDialer (line 48) | func NewDialer() Dialer {
type dialer (line 54) | type dialer struct
method DialTimeout (line 57) | func (d *dialer) DialTimeout(network, address string, timeout time.Dur...
method DialConnection (line 62) | func (d *dialer) DialConnection(network, address string, timeout time....
method dialTCP (line 83) | func (d *dialer) dialTCP(ctx context.Context, network, address string)...
type sysDialer (line 137) | type sysDialer struct
FILE: net_dialer_test.go
function TestDialerTCP (line 32) | func TestDialerTCP(t *testing.T) {
function TestDialerUnix (line 67) | func TestDialerUnix(t *testing.T) {
function TestDialerFdAlloc (line 109) | func TestDialerFdAlloc(t *testing.T) {
function TestFDClose (line 138) | func TestFDClose(t *testing.T) {
function TestDialerThenClose (line 171) | func TestDialerThenClose(t *testing.T) {
function TestNewFDConnection (line 217) | func TestNewFDConnection(t *testing.T) {
function mockDialerEventLoop (line 232) | func mockDialerEventLoop(idx int) EventLoop {
function mockDialerSend (line 267) | func mockDialerSend(idx int, conn *connection) {
FILE: net_io.go
function ioread (line 25) | func ioread(fd int, bs [][]byte, ivs []syscall.Iovec) (n int, err error) {
function iosend (line 39) | func iosend(fd int, bs [][]byte, ivs []syscall.Iovec, zerocopy bool) (n ...
FILE: net_listener.go
function CreateListener (line 28) | func CreateListener(network, addr string) (l Listener, err error) {
function ConvertListener (line 41) | func ConvertListener(l net.Listener) (nl Listener, err error) {
type listener (line 57) | type listener struct
method Accept (line 65) | func (ln *listener) Accept() (net.Conn, error) {
method Close (line 90) | func (ln *listener) Close() error {
method Addr (line 104) | func (ln *listener) Addr() net.Addr {
method Fd (line 109) | func (ln *listener) Fd() (fd int) {
method parseFD (line 113) | func (ln *listener) parseFD() (err error) {
FILE: net_listener_test.go
function TestListenerDialer (line 28) | func TestListenerDialer(t *testing.T) {
function TestConvertListener (line 94) | func TestConvertListener(t *testing.T) {
FILE: net_netfd.go
type netFD (line 27) | type netFD struct
method dial (line 63) | func (c *netFD) dial(ctx context.Context, laddr, raddr sockaddr) (err ...
method connect (line 106) | func (c *netFD) connect(ctx context.Context, la, ra syscall.Sockaddr) ...
function newNetFD (line 51) | func newNetFD(fd, family, sotype int, net string) *netFD {
function mapErr (line 184) | func mapErr(err error) error {
FILE: net_netfd_conn.go
method Fd (line 31) | func (c *netFD) Fd() (fd int) {
method Read (line 36) | func (c *netFD) Read(b []byte) (n int, err error) {
method Write (line 47) | func (c *netFD) Write(b []byte) (n int, err error) {
method Close (line 58) | func (c *netFD) Close() (err error) {
method LocalAddr (line 72) | func (c *netFD) LocalAddr() (addr net.Addr) {
method RemoteAddr (line 77) | func (c *netFD) RemoteAddr() (addr net.Addr) {
method SetKeepAlive (line 83) | func (c *netFD) SetKeepAlive(second int) error {
method SetDeadline (line 94) | func (c *netFD) SetDeadline(t time.Time) error {
method SetReadDeadline (line 99) | func (c *netFD) SetReadDeadline(t time.Time) error {
method SetWriteDeadline (line 104) | func (c *netFD) SetWriteDeadline(t time.Time) error {
FILE: net_polldesc.go
function newPollDesc (line 24) | func newPollDesc(fd int) *pollDesc {
type pollDesc (line 37) | type pollDesc struct
method WaitWrite (line 45) | func (pd *pollDesc) WaitWrite(ctx context.Context) (err error) {
method onwrite (line 73) | func (pd *pollDesc) onwrite(p Poll) error {
method onhup (line 83) | func (pd *pollDesc) onhup(p Poll) error {
method detach (line 92) | func (pd *pollDesc) detach() {
FILE: net_polldesc_test.go
function TestZeroTimer (line 25) | func TestZeroTimer(t *testing.T) {
function TestRuntimePoll (line 29) | func TestRuntimePoll(t *testing.T) {
FILE: net_sock.go
type sockaddr (line 22) | type sockaddr interface
function internetSocket (line 42) | func internetSocket(ctx context.Context, net string, laddr, raddr sockad...
function favoriteAddrFamily (line 89) | func favoriteAddrFamily(network string, laddr, raddr sockaddr) (family i...
function socket (line 105) | func socket(ctx context.Context, net string, family, sotype, proto int, ...
function sockaddrToAddr (line 128) | func sockaddrToAddr(sa syscall.Sockaddr) net.Addr {
FILE: net_tcpsock.go
type TCPAddr (line 21) | type TCPAddr struct
method isWildcard (line 25) | func (a *TCPAddr) isWildcard() bool {
method opAddr (line 32) | func (a *TCPAddr) opAddr() net.Addr {
method family (line 39) | func (a *TCPAddr) family() int {
method sockaddr (line 49) | func (a *TCPAddr) sockaddr(family int) (syscall.Sockaddr, error) {
method toLocal (line 56) | func (a *TCPAddr) toLocal(network string) sockaddr {
function loopbackIP (line 64) | func loopbackIP(network string) net.IP {
function ipToSockaddr (line 71) | func ipToSockaddr(family int, ip net.IP, port int, zone string) (syscall...
function ResolveTCPAddr (line 127) | func ResolveTCPAddr(network, address string) (*TCPAddr, error) {
type TCPConnection (line 136) | type TCPConnection struct
function newTCPConnection (line 141) | func newTCPConnection(conn Conn) (connection *TCPConnection, err error) {
function DialTCP (line 157) | func DialTCP(ctx context.Context, network string, laddr, raddr *TCPAddr)...
method dialTCP (line 177) | func (sd *sysDialer) dialTCP(ctx context.Context, laddr, raddr *TCPAddr)...
function selfConnect (line 217) | func selfConnect(conn *netFD, err error) bool {
function spuriousENOTAVAIL (line 239) | func spuriousENOTAVAIL(err error) bool {
FILE: net_unixsock.go
type UnixAddr (line 27) | type UnixAddr struct
method isWildcard (line 31) | func (a *UnixAddr) isWildcard() bool {
method opAddr (line 35) | func (a *UnixAddr) opAddr() net.Addr {
method family (line 42) | func (a *UnixAddr) family() int {
method sockaddr (line 46) | func (a *UnixAddr) sockaddr(family int) (syscall.Sockaddr, error) {
method toLocal (line 53) | func (a *UnixAddr) toLocal(net string) sockaddr {
function ResolveUnixAddr (line 63) | func ResolveUnixAddr(network, address string) (*UnixAddr, error) {
type UnixConnection (line 72) | type UnixConnection struct
function newUnixConnection (line 77) | func newUnixConnection(conn Conn) (connection *UnixConnection, err error) {
function DialUnix (line 92) | func DialUnix(network string, laddr, raddr *UnixAddr) (*UnixConnection, ...
method dialUnix (line 106) | func (sd *sysDialer) dialUnix(ctx context.Context, laddr, raddr *UnixAdd...
function unixSocket (line 114) | func unixSocket(ctx context.Context, network string, laddr, raddr sockad...
FILE: netpoll_config.go
type Config (line 29) | type Config struct
type Feature (line 39) | type Feature struct
FILE: netpoll_options.go
type Option (line 20) | type Option struct
type options (line 24) | type options struct
function WithOnPrepare (line 35) | func WithOnPrepare(onPrepare OnPrepare) Option {
function WithOnConnect (line 42) | func WithOnConnect(onConnect OnConnect) Option {
function WithOnDisconnect (line 49) | func WithOnDisconnect(onDisconnect OnDisconnect) Option {
function WithReadTimeout (line 56) | func WithReadTimeout(timeout time.Duration) Option {
function WithWriteTimeout (line 63) | func WithWriteTimeout(timeout time.Duration) Option {
function WithIdleTimeout (line 70) | func WithIdleTimeout(timeout time.Duration) Option {
FILE: netpoll_server.go
function newServer (line 30) | func newServer(ln Listener, opts *options, onQuit func(err error)) *serv...
type server (line 38) | type server struct
method Run (line 47) | func (s *server) Run() (err error) {
method Close (line 62) | func (s *server) Close(ctx context.Context) error {
method OnRead (line 99) | func (s *server) OnRead(p Poll) error {
method OnHup (line 158) | func (s *server) OnHup(p Poll) error {
method onAccept (line 163) | func (s *server) onAccept(conn Conn) {
function isOutOfFdErr (line 181) | func isOutOfFdErr(err error) bool {
FILE: netpoll_unix.go
function Initialize (line 39) | func Initialize() {
function Configure (line 46) | func Configure(config Config) (err error) {
function SetNumLoops (line 84) | func SetNumLoops(numLoops int) error {
function SetLoadBalance (line 92) | func SetLoadBalance(lb LoadBalance) error {
function SetLoggerOutput (line 98) | func SetLoggerOutput(w io.Writer) {
function SetRunner (line 105) | func SetRunner(f func(ctx context.Context, f func())) {
function DisableGopool (line 116) | func DisableGopool() error {
function NewEventLoop (line 122) | func NewEventLoop(onRequest OnRequest, ops ...Option) (EventLoop, error) {
type eventLoop (line 135) | type eventLoop struct
method Serve (line 143) | func (evl *eventLoop) Serve(ln net.Listener) error {
method Shutdown (line 160) | func (evl *eventLoop) Shutdown(ctx context.Context) error {
method waitQuit (line 174) | func (evl *eventLoop) waitQuit() error {
method quit (line 178) | func (evl *eventLoop) quit(err error) {
FILE: netpoll_unix_test.go
function MustNil (line 35) | func MustNil(t *testing.T, val interface{}) {
function MustTrue (line 43) | func MustTrue(t *testing.T, cond bool) {
function Equal (line 50) | func Equal(t *testing.T, got, expect interface{}) {
function Assert (line 57) | func Assert(t *testing.T, cond bool, val ...interface{}) {
function getTestAddress (line 72) | func getTestAddress() string {
function TestEqual (line 76) | func TestEqual(t *testing.T) {
function TestOnConnect (line 84) | func TestOnConnect(t *testing.T) {
function TestOnConnectWrite (line 128) | func TestOnConnectWrite(t *testing.T) {
function TestOnDisconnect (line 150) | func TestOnDisconnect(t *testing.T) {
function TestOnDisconnectWhenOnConnect (line 210) | func TestOnDisconnectWhenOnConnect(t *testing.T) {
function TestGracefulExit (line 260) | func TestGracefulExit(t *testing.T) {
function TestCloseCallbackWhenOnRequest (line 322) | func TestCloseCallbackWhenOnRequest(t *testing.T) {
function TestCloseCallbackWhenOnConnect (line 353) | func TestCloseCallbackWhenOnConnect(t *testing.T) {
function TestCloseConnWhenOnConnect (line 380) | func TestCloseConnWhenOnConnect(t *testing.T) {
function TestServerReadAndClose (line 415) | func TestServerReadAndClose(t *testing.T) {
function TestServerPanicAndClose (line 445) | func TestServerPanicAndClose(t *testing.T) {
function TestClientWriteAndClose (line 486) | func TestClientWriteAndClose(t *testing.T) {
function TestServerAcceptWhenTooManyOpenFiles (line 530) | func TestServerAcceptWhenTooManyOpenFiles(t *testing.T) {
function createTestListener (line 602) | func createTestListener(network, address string) (Listener, error) {
function newTestEventLoop (line 612) | func newTestEventLoop(network, address string, onRequest OnRequest, opts...
FILE: netpoll_windows.go
function Configure (line 26) | func Configure(config Config) (err error) {
function NewDialer (line 31) | func NewDialer() Dialer {
function NewEventLoop (line 36) | func NewEventLoop(onRequest OnRequest, ops ...Option) (EventLoop, error) {
function ConvertListener (line 41) | func ConvertListener(l net.Listener) (nl Listener, err error) {
function CreateListener (line 46) | func CreateListener(network, addr string) (l Listener, err error) {
FILE: nocopy.go
type Reader (line 32) | type Reader interface
type Writer (line 123) | type Writer interface
type ReadWriter (line 201) | type ReadWriter interface
function NewReader (line 207) | func NewReader(r io.Reader) Reader {
function NewWriter (line 212) | func NewWriter(w io.Writer) Writer {
function NewReadWriter (line 217) | func NewReadWriter(rw io.ReadWriter) ReadWriter {
function NewIOReader (line 225) | func NewIOReader(r Reader) io.Reader {
function NewIOWriter (line 233) | func NewIOWriter(w Writer) io.Writer {
function NewIOReadWriter (line 241) | func NewIOReadWriter(rw ReadWriter) io.ReadWriter {
constant block1k (line 252) | block1k = 1 * 1024
constant block2k (line 253) | block2k = 2 * 1024
constant block4k (line 254) | block4k = 4 * 1024
constant block8k (line 255) | block8k = 8 * 1024
constant block32k (line 256) | block32k = 32 * 1024
constant pagesize (line 258) | pagesize = block8k
constant mallocMax (line 259) | mallocMax = block8k * block1k
constant defaultLinkBufferMode (line 261) | defaultLinkBufferMode = 0
constant flagUnmanaged (line 265) | flagUnmanaged uint8 = 1 << 0
constant flagReadExposed (line 269) | flagReadExposed uint8 = 1 << 1
function unsafeSliceToString (line 273) | func unsafeSliceToString(b []byte) string {
function unsafeStringToSlice (line 278) | func unsafeStringToSlice(s string) (b []byte) {
function malloc (line 288) | func malloc(size, capacity int) []byte {
function free (line 296) | func free(buf []byte) {
FILE: nocopy_linkbuffer.go
constant BinaryInplaceThreshold (line 29) | BinaryInplaceThreshold = block4k
function NewLinkBuffer (line 42) | func NewLinkBuffer(size ...int) *LinkBuffer {
type UnsafeLinkBuffer (line 54) | type UnsafeLinkBuffer struct
method Len (line 72) | func (b *UnsafeLinkBuffer) Len() int {
method IsEmpty (line 78) | func (b *UnsafeLinkBuffer) IsEmpty() (ok bool) {
method readCopy (line 88) | func (b *UnsafeLinkBuffer) readCopy(p []byte) (n int) {
method Next (line 149) | func (b *UnsafeLinkBuffer) Next(n int) (p []byte, err error) {
method Peek (line 189) | func (b *UnsafeLinkBuffer) Peek(n int) (p []byte, err error) {
method Skip (line 249) | func (b *UnsafeLinkBuffer) Skip(n int) (err error) {
method Release (line 273) | func (b *UnsafeLinkBuffer) Release() (err error) {
method ReadString (line 295) | func (b *UnsafeLinkBuffer) ReadString(n int) (s string, err error) {
method ReadBinary (line 307) | func (b *UnsafeLinkBuffer) ReadBinary(n int) (p []byte, err error) {
method readBinary (line 319) | func (b *UnsafeLinkBuffer) readBinary(n int) (p []byte) {
method ReadByte (line 347) | func (b *UnsafeLinkBuffer) ReadByte() (p byte, err error) {
method Until (line 362) | func (b *UnsafeLinkBuffer) Until(delim byte) (line []byte, err error) {
method Slice (line 374) | func (b *UnsafeLinkBuffer) Slice(n int) (r Reader, err error) {
method Malloc (line 428) | func (b *UnsafeLinkBuffer) Malloc(n int) (buf []byte, err error) {
method MallocLen (line 438) | func (b *UnsafeLinkBuffer) MallocLen() (length int) {
method MallocAck (line 443) | func (b *UnsafeLinkBuffer) MallocAck(n int) (err error) {
method Flush (line 467) | func (b *UnsafeLinkBuffer) Flush() (err error) {
method Append (line 489) | func (b *UnsafeLinkBuffer) Append(w Writer) (err error) {
method WriteBuffer (line 500) | func (b *UnsafeLinkBuffer) WriteBuffer(buf *LinkBuffer) (err error) {
method WriteString (line 539) | func (b *UnsafeLinkBuffer) WriteString(s string) (n int, err error) {
method WriteBinary (line 548) | func (b *UnsafeLinkBuffer) WriteBinary(p []byte) (n int, err error) {
method WriteDirect (line 570) | func (b *UnsafeLinkBuffer) WriteDirect(extra []byte, remainLen int) er...
method WriteByte (line 624) | func (b *UnsafeLinkBuffer) WriteByte(p byte) (err error) {
method Close (line 633) | func (b *UnsafeLinkBuffer) Close() (err error) {
method Bytes (line 650) | func (b *UnsafeLinkBuffer) Bytes() []byte {
method GetBytes (line 668) | func (b *UnsafeLinkBuffer) GetBytes(p [][]byte) (vs [][]byte) {
method book (line 700) | func (b *UnsafeLinkBuffer) book(bookSize, maxSize int) (p []byte) {
method bookAck (line 717) | func (b *UnsafeLinkBuffer) bookAck(n int) (length int, err error) {
method calcMaxSize (line 728) | func (b *UnsafeLinkBuffer) calcMaxSize() (sum int) {
method resetTail (line 738) | func (b *UnsafeLinkBuffer) resetTail(maxSize int) {
method indexByte (line 750) | func (b *UnsafeLinkBuffer) indexByte(c byte, skip int) int {
method recalLen (line 784) | func (b *UnsafeLinkBuffer) recalLen(delta int) (length int) {
method growth (line 794) | func (b *UnsafeLinkBuffer) growth(n int) {
method isSingleNode (line 812) | func (b *UnsafeLinkBuffer) isSingleNode(readN int) (single bool) {
method memorySize (line 825) | func (b *LinkBuffer) memorySize() (bytes int) {
function newLinkBufferNode (line 840) | func newLinkBufferNode(size int) *linkBufferNode {
type linkBufferNode (line 863) | type linkBufferNode struct
method Len (line 873) | func (node *linkBufferNode) Len() (l int) {
method IsEmpty (line 877) | func (node *linkBufferNode) IsEmpty() (ok bool) {
method Reset (line 881) | func (node *linkBufferNode) Reset() {
method Next (line 889) | func (node *linkBufferNode) Next(n int) (p []byte) {
method Peek (line 895) | func (node *linkBufferNode) Peek(n int) (p []byte) {
method Malloc (line 899) | func (node *linkBufferNode) Malloc(n int) (buf []byte) {
method Refer (line 907) | func (node *linkBufferNode) Refer(n int) (p *linkBufferNode) {
method Release (line 923) | func (node *linkBufferNode) Release() (err error) {
method getFlag (line 939) | func (node *linkBufferNode) getFlag(flag uint8) bool {
method setFlag (line 943) | func (node *linkBufferNode) setFlag(flag uint8) {
method unsetFlag (line 947) | func (node *linkBufferNode) unsetFlag(flag uint8) {
method reusable (line 953) | func (node *linkBufferNode) reusable() bool {
method readExposed (line 959) | func (node *linkBufferNode) readExposed() bool {
FILE: nocopy_linkbuffer_race.go
type SafeLinkBuffer (line 27) | type SafeLinkBuffer struct
method readCopy (line 34) | func (b *SafeLinkBuffer) readCopy(p []byte) int {
method Next (line 43) | func (b *SafeLinkBuffer) Next(n int) (p []byte, err error) {
method Peek (line 50) | func (b *SafeLinkBuffer) Peek(n int) (p []byte, err error) {
method Skip (line 57) | func (b *SafeLinkBuffer) Skip(n int) (err error) {
method Until (line 64) | func (b *SafeLinkBuffer) Until(delim byte) (line []byte, err error) {
method Release (line 71) | func (b *SafeLinkBuffer) Release() (err error) {
method ReadString (line 78) | func (b *SafeLinkBuffer) ReadString(n int) (s string, err error) {
method ReadBinary (line 85) | func (b *SafeLinkBuffer) ReadBinary(n int) (p []byte, err error) {
method ReadByte (line 92) | func (b *SafeLinkBuffer) ReadByte() (p byte, err error) {
method Slice (line 99) | func (b *SafeLinkBuffer) Slice(n int) (r Reader, err error) {
method Malloc (line 108) | func (b *SafeLinkBuffer) Malloc(n int) (buf []byte, err error) {
method MallocLen (line 115) | func (b *SafeLinkBuffer) MallocLen() (length int) {
method MallocAck (line 122) | func (b *SafeLinkBuffer) MallocAck(n int) (err error) {
method Flush (line 129) | func (b *SafeLinkBuffer) Flush() (err error) {
method Append (line 136) | func (b *SafeLinkBuffer) Append(w Writer) (err error) {
method WriteBuffer (line 143) | func (b *SafeLinkBuffer) WriteBuffer(buf *LinkBuffer) (err error) {
method WriteString (line 150) | func (b *SafeLinkBuffer) WriteString(s string) (n int, err error) {
method WriteBinary (line 157) | func (b *SafeLinkBuffer) WriteBinary(p []byte) (n int, err error) {
method WriteDirect (line 164) | func (b *SafeLinkBuffer) WriteDirect(p []byte, remainLen int) error {
method WriteByte (line 171) | func (b *SafeLinkBuffer) WriteByte(p byte) (err error) {
method Close (line 178) | func (b *SafeLinkBuffer) Close() (err error) {
method Bytes (line 187) | func (b *SafeLinkBuffer) Bytes() []byte {
method GetBytes (line 194) | func (b *SafeLinkBuffer) GetBytes(p [][]byte) (vs [][]byte) {
method book (line 206) | func (b *SafeLinkBuffer) book(bookSize, maxSize int) (p []byte) {
method bookAck (line 215) | func (b *SafeLinkBuffer) bookAck(n int) (length int, err error) {
method calcMaxSize (line 222) | func (b *SafeLinkBuffer) calcMaxSize() (sum int) {
method resetTail (line 228) | func (b *SafeLinkBuffer) resetTail(maxSize int) {
method indexByte (line 234) | func (b *SafeLinkBuffer) indexByte(c byte, skip int) int {
FILE: nocopy_linkbuffer_test.go
function TestLinkBuffer (line 29) | func TestLinkBuffer(t *testing.T) {
function TestLinkBufferGetBytes (line 91) | func TestLinkBufferGetBytes(t *testing.T) {
function TestLinkBufferWithInvalid (line 116) | func TestLinkBufferWithInvalid(t *testing.T) {
function TestLinkBufferMultiNode (line 200) | func TestLinkBufferMultiNode(t *testing.T) {
function TestLinkBufferRefer (line 331) | func TestLinkBufferRefer(t *testing.T) {
function TestLinkBufferResetTail (line 395) | func TestLinkBufferResetTail(t *testing.T) {
function TestLinkBufferWriteBuffer (line 416) | func TestLinkBufferWriteBuffer(t *testing.T) {
function TestLinkBufferCheckSingleNode (line 432) | func TestLinkBufferCheckSingleNode(t *testing.T) {
function TestLinkBufferWriteMultiFlush (line 453) | func TestLinkBufferWriteMultiFlush(t *testing.T) {
function TestLinkBufferWriteBinary (line 483) | func TestLinkBufferWriteBinary(t *testing.T) {
function TestLinkBufferWriteDirect (line 504) | func TestLinkBufferWriteDirect(t *testing.T) {
function TestLinkBufferBufferMode (line 531) | func TestLinkBufferBufferMode(t *testing.T) {
function TestLinkBufferReadCopy (line 543) | func TestLinkBufferReadCopy(t *testing.T) {
function BenchmarkLinkBufferConcurrentReadWrite (line 699) | func BenchmarkLinkBufferConcurrentReadWrite(b *testing.B) {
function TestUnsafeStringToSlice (line 758) | func TestUnsafeStringToSlice(t *testing.T) {
function TestLinkBufferIndexByte (line 766) | func TestLinkBufferIndexByte(t *testing.T) {
function TestLinkBufferPeekOutOfMemory (line 799) | func TestLinkBufferPeekOutOfMemory(t *testing.T) {
function TestMallocAck (line 846) | func TestMallocAck(t *testing.T) {
function BenchmarkStringToSliceByte (line 874) | func BenchmarkStringToSliceByte(b *testing.B) {
function BenchmarkStringToCopy (line 891) | func BenchmarkStringToCopy(b *testing.B) {
function BenchmarkLinkBufferPoolGet (line 906) | func BenchmarkLinkBufferPoolGet(b *testing.B) {
function BenchmarkCopyString (line 924) | func BenchmarkCopyString(b *testing.B) {
function BenchmarkLinkBufferNoCopyRead (line 939) | func BenchmarkLinkBufferNoCopyRead(b *testing.B) {
FILE: nocopy_readwriter.go
constant maxReadCycle (line 22) | maxReadCycle = 16
function newZCReader (line 24) | func newZCReader(r io.Reader) *zcReader {
type zcReader (line 34) | type zcReader struct
method Next (line 40) | func (r *zcReader) Next(n int) (p []byte, err error) {
method Peek (line 48) | func (r *zcReader) Peek(n int) (buf []byte, err error) {
method Skip (line 56) | func (r *zcReader) Skip(n int) (err error) {
method Release (line 64) | func (r *zcReader) Release() (err error) {
method Slice (line 69) | func (r *zcReader) Slice(n int) (reader Reader, err error) {
method Len (line 77) | func (r *zcReader) Len() (length int) {
method ReadString (line 82) | func (r *zcReader) ReadString(n int) (s string, err error) {
method ReadBinary (line 90) | func (r *zcReader) ReadBinary(n int) (p []byte, err error) {
method ReadByte (line 98) | func (r *zcReader) ReadByte() (b byte, err error) {
method Until (line 105) | func (r *zcReader) Until(delim byte) (line []byte, err error) {
method waitRead (line 109) | func (r *zcReader) waitRead(n int) (err error) {
method fill (line 123) | func (r *zcReader) fill(n int) (err error) {
function newZCWriter (line 147) | func newZCWriter(w io.Writer) *zcWriter {
type zcWriter (line 157) | type zcWriter struct
method Malloc (line 163) | func (w *zcWriter) Malloc(n int) (buf []byte, err error) {
method MallocLen (line 168) | func (w *zcWriter) MallocLen() (length int) {
method Flush (line 173) | func (w *zcWriter) Flush() (err error) {
method MallocAck (line 184) | func (w *zcWriter) MallocAck(n int) (err error) {
method Append (line 189) | func (w *zcWriter) Append(w2 Writer) (err error) {
method WriteString (line 194) | func (w *zcWriter) WriteString(s string) (n int, err error) {
method WriteBinary (line 199) | func (w *zcWriter) WriteBinary(b []byte) (n int, err error) {
method WriteDirect (line 204) | func (w *zcWriter) WriteDirect(p []byte, remainCap int) error {
method WriteByte (line 209) | func (w *zcWriter) WriteByte(b byte) (err error) {
type zcReadWriter (line 214) | type zcReadWriter struct
function newIOReader (line 219) | func newIOReader(r Reader) *ioReader {
type ioReader (line 231) | type ioReader struct
method Read (line 240) | func (r *ioReader) Read(p []byte) (n int, err error) {
function newIOWriter (line 264) | func newIOWriter(w Writer) *ioWriter {
type ioWriter (line 273) | type ioWriter struct
method Write (line 278) | func (w *ioWriter) Write(p []byte) (n int, err error) {
type ioReadWriter (line 292) | type ioReadWriter struct
FILE: nocopy_readwriter_test.go
function TestZCReader (line 27) | func TestZCReader(t *testing.T) {
function TestZCWriter (line 53) | func TestZCWriter(t *testing.T) {
function TestZCEOF (line 84) | func TestZCEOF(t *testing.T) {
type MockIOReadWriter (line 96) | type MockIOReadWriter struct
method Read (line 101) | func (rw *MockIOReadWriter) Read(p []byte) (n int, err error) {
method Write (line 108) | func (rw *MockIOReadWriter) Write(p []byte) (n int, err error) {
function TestIOReadWriter (line 115) | func TestIOReadWriter(t *testing.T) {
function TestIOReadWriter2 (line 129) | func TestIOReadWriter2(t *testing.T) {
FILE: poll.go
type Poll (line 20) | type Poll interface
type PollEvent (line 46) | type PollEvent
constant PollReadable (line 51) | PollReadable PollEvent = 0x1
constant PollWritable (line 55) | PollWritable PollEvent = 0x2
constant PollDetach (line 58) | PollDetach PollEvent = 0x3
constant PollR2RW (line 62) | PollR2RW PollEvent = 0x5
constant PollRW2R (line 65) | PollRW2R PollEvent = 0x6
FILE: poll_default.go
method Alloc (line 20) | func (p *defaultPoll) Alloc() (operator *FDOperator) {
method Free (line 26) | func (p *defaultPoll) Free(operator *FDOperator) {
method appendHup (line 30) | func (p *defaultPoll) appendHup(operator *FDOperator) {
method detach (line 36) | func (p *defaultPoll) detach(operator *FDOperator) {
method onhups (line 42) | func (p *defaultPoll) onhups() {
function readall (line 58) | func readall(op *FDOperator, br barrier) (total int, err error) {
FILE: poll_default_bsd.go
function openPoll (line 28) | func openPoll() (Poll, error) {
function openDefaultPoll (line 32) | func openDefaultPoll() (*defaultPoll, error) {
type defaultPoll (line 52) | type defaultPoll struct
method Wait (line 61) | func (p *defaultPoll) Wait() error {
method Close (line 159) | func (p *defaultPoll) Close() error {
method Trigger (line 165) | func (p *defaultPoll) Trigger() error {
method Control (line 178) | func (p *defaultPoll) Control(operator *FDOperator, event PollEvent) e...
FILE: poll_default_bsd_norace.go
method getOperator (line 23) | func (p *defaultPoll) getOperator(fd int, ptr unsafe.Pointer) *FDOperator {
method setOperator (line 27) | func (p *defaultPoll) setOperator(ptr unsafe.Pointer, operator *FDOperat...
method delOperator (line 31) | func (p *defaultPoll) delOperator(operator *FDOperator) {
FILE: poll_default_bsd_race.go
method getOperator (line 23) | func (p *defaultPoll) getOperator(fd int, ptr unsafe.Pointer) *FDOperator {
method setOperator (line 31) | func (p *defaultPoll) setOperator(ptr unsafe.Pointer, operator *FDOperat...
method delOperator (line 35) | func (p *defaultPoll) delOperator(operator *FDOperator) {
FILE: poll_default_linux.go
function openPoll (line 26) | func openPoll() (Poll, error) {
function openDefaultPoll (line 30) | func openDefaultPoll() (*defaultPoll, error) {
type defaultPoll (line 60) | type defaultPoll struct
method Wait (line 91) | func (p *defaultPoll) Wait() (err error) {
method handler (line 118) | func (p *defaultPoll) handler(events []epollevent) (closed bool) {
method Close (line 223) | func (p *defaultPoll) Close() error {
method Trigger (line 229) | func (p *defaultPoll) Trigger() error {
method Control (line 239) | func (p *defaultPoll) Control(operator *FDOperator, event PollEvent) e...
type pollArgs (line 73) | type pollArgs struct
method reset (line 81) | func (a *pollArgs) reset(size, caps int) {
FILE: poll_default_linux_norace.go
method getOperator (line 22) | func (p *defaultPoll) getOperator(fd int, ptr unsafe.Pointer) *FDOperator {
method setOperator (line 26) | func (p *defaultPoll) setOperator(ptr unsafe.Pointer, operator *FDOperat...
method delOperator (line 30) | func (p *defaultPoll) delOperator(operator *FDOperator) {
FILE: poll_default_linux_race.go
type eventdata (line 22) | type eventdata struct
method getOperator (line 27) | func (p *defaultPoll) getOperator(fd int, ptr unsafe.Pointer) *FDOperator {
method setOperator (line 36) | func (p *defaultPoll) setOperator(ptr unsafe.Pointer, operator *FDOperat...
method delOperator (line 41) | func (p *defaultPoll) delOperator(operator *FDOperator) {
FILE: poll_default_linux_test.go
function TestEpollEvent (line 29) | func TestEpollEvent(t *testing.T) {
function TestEpollWait (line 97) | func TestEpollWait(t *testing.T) {
function TestEpollETClose (line 166) | func TestEpollETClose(t *testing.T) {
function TestEpollETDel (line 218) | func TestEpollETDel(t *testing.T) {
function TestEpollConnectSameFD (line 246) | func TestEpollConnectSameFD(t *testing.T) {
function epollWaitUntil (line 345) | func epollWaitUntil(epfd int, events []epollevent, msec int) (n int, err...
FILE: poll_loadbalance.go
type LoadBalance (line 24) | type LoadBalance
constant RoundRobin (line 29) | RoundRobin LoadBalance = iota
constant Random (line 31) | Random
type loadbalance (line 35) | type loadbalance interface
function newLoadbalance (line 43) | func newLoadbalance(lb LoadBalance, polls []Poll) loadbalance {
function newRandomLB (line 53) | func newRandomLB(polls []Poll) loadbalance {
type randomLB (line 57) | type randomLB struct
method LoadBalance (line 62) | func (b *randomLB) LoadBalance() LoadBalance {
method Pick (line 66) | func (b *randomLB) Pick() (poll Poll) {
method Rebalance (line 71) | func (b *randomLB) Rebalance(polls []Poll) {
function newRoundRobinLB (line 75) | func newRoundRobinLB(polls []Poll) loadbalance {
type roundRobinLB (line 79) | type roundRobinLB struct
method LoadBalance (line 85) | func (b *roundRobinLB) LoadBalance() LoadBalance {
method Pick (line 89) | func (b *roundRobinLB) Pick() (poll Poll) {
method Rebalance (line 94) | func (b *roundRobinLB) Rebalance(polls []Poll) {
FILE: poll_manager.go
constant managerUninitialized (line 27) | managerUninitialized = iota
constant managerInitializing (line 28) | managerInitializing
constant managerInitialized (line 29) | managerInitialized
function newManager (line 32) | func newManager(numLoops int) *manager {
type manager (line 41) | type manager struct
method SetNumLoops (line 49) | func (m *manager) SetNumLoops(numLoops int) (err error) {
method SetLoadBalance (line 60) | func (m *manager) SetLoadBalance(lb LoadBalance) error {
method Close (line 69) | func (m *manager) Close() (err error) {
method Run (line 80) | func (m *manager) Run() (err error) {
method Reset (line 122) | func (m *manager) Reset() error {
method Pick (line 131) | func (m *manager) Pick() Poll {
FILE: poll_manager_test.go
function TestPollManager (line 26) | func TestPollManager(t *testing.T) {
function TestPollManagerReset (line 50) | func TestPollManagerReset(t *testing.T) {
function TestPollManagerSetNumLoops (line 57) | func TestPollManagerSetNumLoops(t *testing.T) {
FILE: poll_test.go
function TestPollTrigger (line 30) | func TestPollTrigger(t *testing.T) {
function TestPollMod (line 55) | func TestPollMod(t *testing.T) {
function TestPollClose (line 117) | func TestPollClose(t *testing.T) {
function BenchmarkPollMod (line 130) | func BenchmarkPollMod(b *testing.B) {
FILE: sys_epoll_linux.go
constant EPOLLET (line 25) | EPOLLET = -syscall.EPOLLET
type epollevent (line 27) | type epollevent struct
function EpollCreate (line 33) | func EpollCreate(flag int) (fd int, err error) {
function EpollCtl (line 43) | func EpollCtl(epfd, op, fd int, event *epollevent) (err error) {
function EpollWait (line 52) | func EpollWait(epfd int, events []epollevent, msec int) (n int, err erro...
FILE: sys_epoll_linux_arm64.go
constant EPOLLET (line 22) | EPOLLET = syscall.EPOLLET
type epollevent (line 24) | type epollevent struct
function EpollCreate (line 31) | func EpollCreate(flag int) (fd int, err error) {
function EpollCtl (line 41) | func EpollCtl(epfd int, op int, fd int, event *epollevent) (err error) {
function EpollWait (line 50) | func EpollWait(epfd int, events []epollevent, msec int) (n int, err erro...
FILE: sys_epoll_linux_loong64.go
constant EPOLLET (line 25) | EPOLLET = syscall.EPOLLET
type epollevent (line 27) | type epollevent struct
function EpollCreate (line 34) | func EpollCreate(flag int) (fd int, err error) {
function EpollCtl (line 44) | func EpollCtl(epfd int, op int, fd int, event *epollevent) (err error) {
function EpollWait (line 53) | func EpollWait(epfd int, events []epollevent, msec int) (n int, err erro...
FILE: sys_exec.go
function GetSysFdPairs (line 28) | func GetSysFdPairs() (r, w int) {
function setTCPNoDelay (line 34) | func setTCPNoDelay(fd int, b bool) (err error) {
function sysSocket (line 40) | func sysSocket(family, sotype, proto int) (int, error) {
constant barriercap (line 58) | barriercap = 32
type barrier (line 60) | type barrier struct
function writev (line 66) | func writev(fd int, bs [][]byte, ivs []syscall.Iovec) (n int, err error) {
function readv (line 82) | func readv(fd int, bs [][]byte, ivs []syscall.Iovec) (n int, err error) {
function iovecs (line 102) | func iovecs(bs [][]byte, ivs []syscall.Iovec) (iovLen int) {
function resetIovecs (line 126) | func resetIovecs(bs [][]byte, ivs []syscall.Iovec) {
function boolint (line 136) | func boolint(b bool) int {
FILE: sys_exec_test.go
function TestIovecs (line 26) | func TestIovecs(t *testing.T) {
function TestWritev (line 76) | func TestWritev(t *testing.T) {
function TestReadv (line 96) | func TestReadv(t *testing.T) {
function TestSendmsg (line 127) | func TestSendmsg(t *testing.T) {
function BenchmarkWrite (line 147) | func BenchmarkWrite(b *testing.B) {
function BenchmarkWritev (line 173) | func BenchmarkWritev(b *testing.B) {
function BenchmarkSendmsg (line 200) | func BenchmarkSendmsg(b *testing.B) {
function BenchmarkRead (line 227) | func BenchmarkRead(b *testing.B) {
function BenchmarkReadv (line 253) | func BenchmarkReadv(b *testing.B) {
FILE: sys_keepalive_darwin.go
function SetKeepAlive (line 20) | func SetKeepAlive(fd, secs int) error {
FILE: sys_keepalive_openbsd.go
function SetKeepAlive (line 18) | func SetKeepAlive(fd, secs int) error {
FILE: sys_keepalive_unix.go
function SetKeepAlive (line 23) | func SetKeepAlive(fd, secs int) error {
FILE: sys_sendmsg_bsd.go
function sendmsg (line 27) | func sendmsg(fd int, bs [][]byte, ivs []syscall.Iovec, zerocopy bool) (n...
FILE: sys_sendmsg_linux.go
function sendmsg (line 34) | func sendmsg(fd int, bs [][]byte, ivs []syscall.Iovec, zerocopy bool) (n...
FILE: sys_sockopt_bsd.go
function setDefaultSockopts (line 19) | func setDefaultSockopts(s, family, sotype int, ipv6only bool) error {
FILE: sys_sockopt_linux.go
function setDefaultSockopts (line 15) | func setDefaultSockopts(s, family, sotype int, ipv6only bool) error {
Condensed preview — 92 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (397K chars).
[
{
"path": ".github/CODEOWNERS",
"chars": 268,
"preview": "# For more information, please refer to https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-f"
},
{
"path": ".github/ISSUE_TEMPLATE/bug_report.md",
"chars": 834,
"preview": "---\nname: Bug report\nabout: Create a report to help us improve\ntitle: ''\nlabels: ''\nassignees: ''\n\n---\n\n**Describe the b"
},
{
"path": ".github/ISSUE_TEMPLATE/feature_request.md",
"chars": 595,
"preview": "---\nname: Feature request\nabout: Suggest an idea for this project\ntitle: ''\nlabels: ''\nassignees: ''\n\n---\n\n**Is your fea"
},
{
"path": ".github/PULL_REQUEST_TEMPLATE.md",
"chars": 0,
"preview": ""
},
{
"path": ".github/workflows/pr-check.yml",
"chars": 1696,
"preview": "name: Push and Pull Request Check\n\non: [ push, pull_request ]\n\njobs:\n compatibility-test:\n strategy:\n matrix:\n "
},
{
"path": ".gitignore",
"chars": 275,
"preview": "# Binaries for programs and plugins\n*.exe\n*.exe~\n*.dll\n*.so\n*.dylib\n\n# Test binary, built with `go test -c`\n*.test\n\n# Ou"
},
{
"path": ".golangci.yaml",
"chars": 636,
"preview": "# Options for analysis running.\nrun:\n timeout: 3m\n\nlinters: # https://golangci-lint.run/usage/linters/\n disable-all: t"
},
{
"path": ".licenserc.yaml",
"chars": 306,
"preview": "header:\n license:\n spdx-id: Apache-2.0\n copyright-owner: CloudWeGo Authors\n\n paths:\n - '**/*.go'\n - '**/*."
},
{
"path": "CODE_OF_CONDUCT.md",
"chars": 5222,
"preview": "# Contributor Covenant Code of Conduct\n\n## Our Pledge\n\nWe as members, contributors, and leaders pledge to make participa"
},
{
"path": "CONTRIBUTING.md",
"chars": 3296,
"preview": "# How to Contribute\n\n## Your First Pull Request\nWe use github for our codebase. You can start by reading [How To Pull Re"
},
{
"path": "CREDITS",
"chars": 0,
"preview": ""
},
{
"path": "LICENSE",
"chars": 11357,
"preview": " Apache License\n Version 2.0, January 2004\n "
},
{
"path": "NOTICE",
"chars": 82,
"preview": "CloudWeGO\nCopyright 2022 CloudWeGO authors.\n\nGo\nCopyright (c) 2009 The Go Authors."
},
{
"path": "README.md",
"chars": 4647,
"preview": "# CloudWeGo-Netpoll\n\n[中文](README_CN.md)\n\n[](https:/"
},
{
"path": "README_CN.md",
"chars": 3361,
"preview": "# CloudWeGo-Netpoll\n\n[English](README.md)\n\n[](https"
},
{
"path": "_typos.toml",
"chars": 213,
"preview": "# Typo check: https://github.com/crate-ci/typos\n\n[files]\nextend-exclude = [\"go.mod\", \"go.sum\"]\n\n[default.extend-identifi"
},
{
"path": "connection.go",
"chars": 4135,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "connection_errors.go",
"chars": 3016,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "connection_errors_test.go",
"chars": 1100,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "connection_impl.go",
"chars": 14732,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "connection_lock.go",
"chars": 2259,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "connection_onevent.go",
"chars": 9573,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "connection_reactor.go",
"chars": 4203,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "connection_test.go",
"chars": 22205,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "docs/guide/guide_cn.md",
"chars": 10494,
"preview": "# 快速开始\n\n本教程通过一些简单的 [示例][Examples] 帮助您开始使用 [Netpoll][Netpoll]，包括如何使用 [Server](#1-使用-sever)、[Client](#2-使用-dialer) 和 [noco"
},
{
"path": "docs/guide/guide_en.md",
"chars": 14723,
"preview": "# Tutorial\n\nThis tutorial gets you started with [Netpoll][Netpoll] through some simple [examples][Examples], includes ho"
},
{
"path": "docs/reference/design_cn.md",
"chars": 6,
"preview": "# TODO"
},
{
"path": "docs/reference/design_en.md",
"chars": 6,
"preview": "# TODO"
},
{
"path": "docs/reference/explain.md",
"chars": 387,
"preview": "# DATA RACE EXPLAIN\n`Netpoll` declare different files by `//+build !race` and `//+build race` to avoid `DATA RACE` detec"
},
{
"path": "eventloop.go",
"chars": 4937,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "fd_operator.go",
"chars": 2802,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "fd_operator_cache.go",
"chars": 2194,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "fd_operator_cache_test.go",
"chars": 1808,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "go.mod",
"chars": 154,
"preview": "module github.com/cloudwego/netpoll\n\ngo 1.15\n\nrequire (\n\tgithub.com/bytedance/gopkg v0.1.1\n\tgithub.com/cloudwego/gopkg v"
},
{
"path": "go.sum",
"chars": 6807,
"preview": "github.com/bytedance/gopkg v0.1.1 h1:3azzgSkiaw79u24a+w9arfH8OfnQQ4MHUt9lJFREEaE=\ngithub.com/bytedance/gopkg v0.1.1/go.m"
},
{
"path": "internal/runner/runner.go",
"chars": 1682,
"preview": "/*\n * Copyright 2025 CloudWeGo Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may "
},
{
"path": "internal/runner/runner_test.go",
"chars": 863,
"preview": "/*\n * Copyright 2025 CloudWeGo Authors\n *\n * Licensed under the Apache License, Version 2.0 (the \"License\");\n * you may "
},
{
"path": "lint.sh",
"chars": 39,
"preview": "#!/usr/bin/env bash\n\ngolangci-lint run\n"
},
{
"path": "mux/mux_test.go",
"chars": 1327,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "mux/shard_queue.go",
"chars": 5121,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "mux/shard_queue_test.go",
"chars": 1798,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "net_dialer.go",
"chars": 3861,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "net_dialer_test.go",
"chars": 6691,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "net_io.go",
"chars": 1449,
"preview": "// Copyright 2023 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "net_listener.go",
"chars": 3104,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "net_listener_test.go",
"chars": 2584,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "net_netfd.go",
"chars": 6042,
"preview": "// Copyright 2009 The Go Authors. All rights reserved.\n// Use of this source code is governed by a BSD-style\n// license "
},
{
"path": "net_netfd_conn.go",
"chars": 2493,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "net_polldesc.go",
"chars": 2395,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "net_polldesc_test.go",
"chars": 1261,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "net_sock.go",
"chars": 4960,
"preview": "// Copyright 2009 The Go Authors. All rights reserved.\n// Use of this source code is governed by a BSD-style\n// license "
},
{
"path": "net_tcpsock.go",
"chars": 7894,
"preview": "// Copyright 2009 The Go Authors. All rights reserved.\n// Use of this source code is governed by a BSD-style\n// license "
},
{
"path": "net_unixsock.go",
"chars": 3729,
"preview": "// Copyright 2009 The Go Authors. All rights reserved.\n// Use of this source code is governed by a BSD-style\n// license "
},
{
"path": "netpoll_config.go",
"chars": 1662,
"preview": "// Copyright 2024 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "netpoll_options.go",
"chars": 1989,
"preview": "// Copyright 2024 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "netpoll_server.go",
"chars": 4525,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "netpoll_unix.go",
"chars": 5005,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "netpoll_unix_test.go",
"chars": 16271,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "netpoll_windows.go",
"chars": 1329,
"preview": "// Copyright 2024 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "nocopy.go",
"chars": 9748,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "nocopy_linkbuffer.go",
"chars": 25200,
"preview": "// Copyright 2024 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "nocopy_linkbuffer_norace.go",
"chars": 680,
"preview": "// Copyright 2024 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "nocopy_linkbuffer_race.go",
"chars": 6159,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "nocopy_linkbuffer_test.go",
"chars": 21368,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "nocopy_readwriter.go",
"chars": 5949,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "nocopy_readwriter_test.go",
"chars": 2943,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "poll.go",
"chars": 2368,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "poll_default.go",
"chars": 1855,
"preview": "// Copyright 2023 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "poll_default_bsd.go",
"chars": 5511,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "poll_default_bsd_norace.go",
"chars": 1051,
"preview": "// Copyright 2023 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "poll_default_bsd_race.go",
"chars": 1129,
"preview": "// Copyright 2023 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "poll_default_linux.go",
"chars": 7445,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "poll_default_linux_norace.go",
"chars": 959,
"preview": "// Copyright 2023 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "poll_default_linux_race.go",
"chars": 1180,
"preview": "// Copyright 2023 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "poll_default_linux_test.go",
"chars": 10736,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "poll_loadbalance.go",
"chars": 2283,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "poll_manager.go",
"chars": 3958,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "poll_manager_test.go",
"chars": 2182,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "poll_test.go",
"chars": 3344,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "sys_epoll_linux.go",
"chars": 1873,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "sys_epoll_linux_arm64.go",
"chars": 1839,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "sys_epoll_linux_loong64.go",
"chars": 1892,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "sys_exec.go",
"chars": 3451,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "sys_exec_test.go",
"chars": 5785,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "sys_keepalive_darwin.go",
"chars": 1112,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "sys_keepalive_openbsd.go",
"chars": 786,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "sys_keepalive_unix.go",
"chars": 1310,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "sys_sendmsg_bsd.go",
"chars": 1290,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "sys_sendmsg_linux.go",
"chars": 1291,
"preview": "// Copyright 2022 CloudWeGo Authors\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not"
},
{
"path": "sys_sockopt_bsd.go",
"chars": 1248,
"preview": "// Copyright 2009 The Go Authors. All rights reserved.\n// Use of this source code is governed by a BSD-style\n// license "
},
{
"path": "sys_sockopt_linux.go",
"chars": 859,
"preview": "// Copyright 2009 The Go Authors. All rights reserved.\n// Use of this source code is governed by a BSD-style\n// license "
},
{
"path": "test_conns.sh",
"chars": 148,
"preview": "#!/usr/bin/env bash\n\nip=\"$1\"\nport=\"$2\"\nconns=\"$3\"\ntimeout=\"$4\"\n\nfor i in $(seq 1 $conns);\ndo\n nc -v -w $timeout $ip $po"
}
]
About this extraction
This page contains the full source code of the cloudwego/netpoll GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 92 files (356.8 KB), approximately 106.2k tokens, and a symbol index with 617 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.